Logo
Machine Learning (2026-2027) - k-Means Clustering

In the print dialog, choose "Save as PDF" as the destination.

Machine Learning, Week 8

k-Means Clustering

Group customers that carry no labels at all: assign, update, repeat. Then segment 200 customers for a campaign.

Objectives

  • Tell supervised learning from clustering
  • Run k-means by hand: assign to the nearest centroid, move to the mean, repeat
  • Compute the WCSS, which scikit-learn calls inertia_
  • Fit KMeans, read cluster_centers_ and labels_, choose k with the elbow method
  • Segment customers and pick the segment a campaign should target
  • Explain why a different start can give a different result, and what n_init does

Week 8 of the plan

Where This Sits in the Course

  • Two notebooks: Clustering (blobs, then 200 mall customers) and K_Means_Clustering_Practice (777 US colleges)
  • Real-world scenario: segmenting customers from their data to decide who gets a campaign for a new product or a promotion to increase their purchasing
  • Project milestone this week: Project Teams Presentations and Discussions

Plan for the Two Hours

PartWhat we doTime
1Clustering and k-means, then five customers by hand40 min
2Notebook 1: blobs and the elbow method15 min
3Notebook 1: segmenting customers20 min
4Notebook 2: colleges, and the project milestone15 min
5Practice with answers, then takeaways30 min

Part 1

The Idea of Clustering

Find groups of similar rows, with no labels to learn from

Supervised Learning vs Clustering

Supervised (weeks 3 to 7)Clustering (this week)
Training datafeatures X and a label yfeatures X only
The model learnshow to predict ygroups of similar rows
ExampleKNN predicts the iris speciesk-means finds customer segments
Checking the resultcompare with the true yusually no true answer exists

What Is k-Means?

k-means
Divides the samples into k clusters that do not overlap. Each cluster is described by the mean of its samples: its centroid.
  • You choose k, the number of clusters
  • A centroid is usually not one of the data points, but it lives in the same space
  • For customers described by (income, spending), a centroid is also an (income, spending) pair

The Algorithm

  1. Choose k and place k starting centroids, for example on k random data points
  2. Assign: every point joins its nearest centroid (squared Euclidean distance)
  3. Update: every centroid moves to the mean of its points
  4. Repeat assign and update until no point changes its cluster

Two Formulas

A point (x, y), a centroid (a, b), a cluster with n points

d2 = (x − a)2 + (y − b)2
a = x1 + x2 + ... + xnn, b = y1 + y2 + ... + ynn
  • The assign step uses d²: no square root is needed, it never changes which centroid is nearest
  • The update step uses the two means: the new centroid is the average point of its cluster
  • Same distance as KNN in week 5, now measured to centroids

By hand

Worked Example: Five Customers

Five real customers from notebook 1: annual income (k$) and spending score (1 to 100)

CustomerIncome (k$)Spending score
#111914
#494042
#624655
#1367388
#19010385

Iteration 1: Assign

μ1 = (19, 14), μ2 = (46, 55); difference = customer minus centroid

Customerd² to μ1d² to μ2Joins
#110 + 0 = 0729 + 1681 = 2410μ1
#49441 + 784 = 122536 + 169 = 205μ2
#62729 + 1681 = 24100 + 0 = 0μ2
#1362916 + 5476 = 8392729 + 1089 = 1818μ2
#1907056 + 5041 = 120973249 + 900 = 4149μ2

Iteration 1: Update

Cluster 1: #11. Cluster 2: #49, #62, #136, #190

μ1 = (19, 14)
μ2 = (40 + 46 + 73 + 1034, 42 + 55 + 88 + 854) = (65.5, 67.5)

Iteration 2: Assign

μ1 = (19, 14), μ2 = (65.5, 67.5)

Customerd² to μ1d² to μ2Joins
#110 + 0 = 02162.25 + 2862.25 = 5024.5μ1
#49441 + 784 = 1225650.25 + 650.25 = 1300.5μ1
#62729 + 1681 = 2410380.25 + 156.25 = 536.5μ2
#1362916 + 5476 = 839256.25 + 420.25 = 476.5μ2
#1907056 + 5041 = 120971406.25 + 306.25 = 1712.5μ2

Iteration 2: Update

Cluster 1: #11, #49. Cluster 2: #62, #136, #190

μ1 = (19 + 402, 14 + 422) = (29.5, 28)
μ2 = (46 + 73 + 1033, 55 + 88 + 853) = (74, 76)

Iteration 3: Assign

μ1 = (29.5, 28), μ2 = (74, 76)

Customerd² to μ1d² to μ2Joins
#11110.25 + 196 = 306.253025 + 3844 = 6869μ1
#49110.25 + 196 = 306.251156 + 1156 = 2312μ1
#62272.25 + 729 = 1001.25784 + 441 = 1225μ1
#1361892.25 + 3600 = 5492.251 + 144 = 145μ2
#1905402.25 + 3249 = 8651.25841 + 81 = 922μ2

Iteration 3: Update

Cluster 1: #11, #49, #62. Cluster 2: #136, #190

μ1 = (19 + 40 + 463, 14 + 42 + 553) = (35, 37)
μ2 = (73 + 1032, 88 + 852) = (88, 86.5)

Iteration 4: Nothing Moves

μ1 = (35, 37), μ2 = (88, 86.5)

Customerd² to μ1d² to μ2Joins
#11256 + 529 = 7854761 + 5256.25 = 10017.25μ1
#4925 + 25 = 502304 + 1980.25 = 4284.25μ1
#62121 + 324 = 4451764 + 992.25 = 2756.25μ1
#1361444 + 2601 = 4045225 + 2.25 = 227.25μ2
#1904624 + 2304 = 6928225 + 2.25 = 227.25μ2

WCSS: How Tight Are the Clusters?

N points, d_i = distance from point i to its own centroid

J = d12 + d22 + ... + dN2
IterationJ after assignJ after update
161724026
23950.52904.5
32680.751734.5
41734.5converged

Try It: k-Means, One Step at a Time

Part 2

Notebook 1: Blobs and the Elbow

Generated points first, so we know the right answer

Open the Notebooks in Colab

text
https://colab.research.google.com/github/
jeffprosise/Machine-Learning/blob/master/
Clustering.ipynb
Notebook 1. Join the lines into one address, or click "Open in Colab" on the lesson page.
text
https://colab.research.google.com/github/
tirthajyoti/Machine-Learning-with-Python/blob/master/
Clustering-Dimensionality-Reduction/
K_Means_Clustering_Practice.ipynb
Notebook 2, for Part 4.

Four Blobs: Ask for 3, 4 or 5 Clusters

make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=0), then KMeans(n_clusters=k, random_state=0)

The blobs clustered with k = 3, 4 and 5, with the centroids in red; inertia 708.8, 369.5 and 328.6
  • k = 3 merges two blobs into one cluster of 151 points
  • k = 4 finds the four blobs: 81, 74, 73 and 72 points
  • k = 5 cuts one real blob into 42 and 31 points

A New Point: Which Cluster?

The k = 4 model predicts (-0.5, 5) with kmeans.predict(point), then prints the distance to each centroid

text
[array([1.46570934]), array([4.8470332]),
 array([2.93542029]), array([2.52147763])]
The printed distances to centroids 0, 1, 2 and 3

Choosing k: the Elbow Method

Inertia against the number of clusters for the blobs, with the elbow at 4
  • Loop k from 1 to 9 and record kmeans.inertia_
  • Inertia always falls as k grows, so the smallest value is not the answer
  • Pick the elbow: 3 to 4 drops 339.24, 4 to 5 only 40.88
  • So k = 4, the number of blobs generated

Try It: Run the Elbow Loop

Part 3

Segmenting Customers

The week 8 scenario: who gets the campaign?

The Scenario

Segmenting customers based on customer data to identify which customers to target with a campaign for a new product or a promotion to increase their purchasing activity.
Week 8 real-world scenario
ColumnIn the 200 customers
GenderMale or Female (88 and 112)
Age18 to 70
Annual Income (k$)15 to 137
Spending Score (1-100)1 to 99: how much the customer spends

Load and Look

customers.shape is (200, 5); info() shows no missing values

python
url = ("https://raw.githubusercontent.com/"
       "jeffprosise/Machine-Learning/master/"
       "Data/customers.csv")
customers = pd.read_csv(url)

points = customers.iloc[:, 3:5].values
x = points[:, 0]
y = points[:, 1]
plt.scatter(x, y, s=50, alpha=0.7)
The notebook reads Data/customers.csv, which is not there in Colab: read it from the repository.
200 customers by annual income and spending score: a dense group in the middle and four groups in the corners

The Elbow Says 5

Inertia against the number of clusters for income and spending, with the elbow at 5
  • Same loop, k from 1 to 9, on income and spending
  • 4 to 5 drops 29231.33 (73679.79 to 44448.46)
  • 5 to 6 drops only 5589.50
  • The notebook segments the customers into five clusters

Five Segments

KMeans(n_clusters=5, random_state=0); red dots are the centroids

The 200 customers in five segments with their centroids; the star at (120, 20) falls in cluster 2

Which Segment Gets the Promotion?

The centroids of the five segments

ClusterAvg income (k$)Avg spendingCustomers
055.3049.5281
186.5482.1339
288.2017.1135
326.3020.9123
425.7379.3622

Build the Campaign List

python
df = customers.copy()
df['Cluster'] = kmeans.predict(points)

cluster = kmeans.predict(np.array([[120, 20]]))[0]
clustered_df = df[df['Cluster'] == cluster]
clustered_df['CustomerID'].values

Segment on All Four Attributes

Gender, age, income and spending; CustomerID is left out

python
from sklearn.preprocessing import LabelEncoder

df = customers.copy()
encoder = LabelEncoder()
df['Gender'] = encoder.fit_transform(df['Gender'])
points = df.iloc[:, 1:5].values

kmeans = KMeans(n_clusters=5, random_state=0)
kmeans.fit(points)
encoder.classes_ is ['Female' 'Male']: Female becomes 0, Male becomes 1
Inertia against the number of clusters for four features: the elbow is less distinct

Four-Feature Segments

The report loop of the notebook, one row per cluster

ClusterAvg ageAvg incomeAvg spending
054.0640.4636.72
132.6986.5482.13
225.2525.8376.92
341.6588.7416.76
433.4058.0648.77

A Different Start, a Different Result

The saved notebook shows other ages for the same cell: 45.22, 32.69, 43.09, 40.67, 25.52

  • The start is random (init='k-means++'), fixed by random_state
  • n_init = how many starts to run; the run with the smallest inertia is kept
  • Since scikit-learn 1.4 the default n_init='auto' means one start
SettingInertiaCluster sizes
one start (default)82657.0550, 39, 24, 34, 53
n_init=1075399.6223, 39, 79, 23, 36

Part 4

Notebook 2: The Colleges

Can k-means find private and public colleges without the labels?

777 Colleges, Two Clusters

  • Private (Yes or No) and 17 numbers: Apps, F.Undergrad, Outstate, Expend, ...
  • 565 private and 212 public colleges
  • Cluster on the 17 numbers only, then compare with Private
  • Real clustering has no labels; here they let us score the result
Full-time undergraduates against out-of-state tuition, coloured by Private

Three Cells to Fix

python
url = ("https://raw.githubusercontent.com/"
       "tirthajyoti/Machine-Learning-with-Python/"
       "master/Datasets/College_Data")
df = pd.read_csv(url, index_col=0)
df.loc['Cazenovia College', 'Grad.Rate'] = 100
  • College_Data is not next to the notebook in Colab: read it from Datasets
  • df['Grad.Rate']['Cazenovia College'] = 100 is chained assignment: with pandas 3 it only warns and the rate stays 118. Use .loc
  • seaborn now needs x= and y= by name, and height= instead of size= in lmplot and FacetGrid

Fit Two Clusters

We added random_state=0 so the run is repeatable

python
kmeans = KMeans(n_clusters=2, verbose=0, tol=1e-3, max_iter=300,
                n_init=20, random_state=0)
kmeans.fit(df.drop('Private', axis=1))
FeatureCluster 0 (669)Cluster 1 (108)
Apps1813.210363.1
F.Undergrad2188.513061.9
Outstate10395.710719.2
Expend8932.014170.5

Compare the Clusters with the Labels

Cluster = 1 for a private college, 0 for a public one; A = accuracy

python
print(confusion_matrix(df['Cluster'], kmeans.labels_))
# [[138  74]
#  [531  34]]
Rows: true label (0 public, 1 private). Columns: the cluster.
A = 138 + 34777 = 0.2214

Swap the Cluster Numbers

1 - kmeans.labels_: the matrix saved in the notebook

Cluster 0Cluster 1
Public (0)74138
Private (1)34531
A = 74 + 531777 = 0.7786

Before you practise

Common Mistakes

  • Treating cluster numbers as meaningful: read the centroids
  • Picking the k with the smallest inertia instead of the elbow
  • Trusting one start: use n_init=10 or more
  • Clustering on an ID column, or passing text columns without encoding them
  • Scoring clusters against labels before matching the numbers
  • Writing df1 = df when you mean df1 = df.copy(): the notebook's last cell leaks the labels this way

Your team project

Project Milestone: Presentations and Discussions

  1. The problem and the dataset: rows, features, target
  2. How you cleaned and explored the data, with one or two plots
  3. The models you built since week 3, scored on the same test split
  4. The model you keep, and why
  5. What is still missing, and your plan

Part 5

Practice: Your Turn

About 30 minutes, answers after each task

About 10 minutes

Practice 1: A Different Start

  1. Same five customers, k = 2, start at μ1 = #11 = (19, 14) and μ2 = #136 = (73, 88)
  2. Assign and update until no customer changes cluster
  3. Write the final clusters, centroids and J
  4. How many assign steps did you need, compared with the worked example?

Answers

Practice 1: Answer

Iteration 1: distances to μ1 = (19, 14) and μ2 = (73, 88)

Customerd² to μ1d² to μ2Joins
#1102916 + 5476 = 8392μ1
#49441 + 784 = 12251089 + 2116 = 3205μ1
#62729 + 1681 = 2410729 + 1089 = 1818μ2
#13683920μ2
#1907056 + 5041 = 12097900 + 9 = 909μ2

About 5 minutes

Practice 2: Which Segment?

A new customer: income 100, spending 50

ClusterCentroid (income, spending)
0(55.3, 49.5)
1(86.5, 82.1)
2(88.2, 17.1)
3(26.3, 20.9)
4(25.7, 79.4)

About 5 minutes

Practice 3: Score the College Clusters

After the swap; A accuracy, P_1 precision and R_1 recall of the private class

Cluster 0Cluster 1
Public (0)74138
Private (1)34531
A = 74 + 531777 = 0.7786
P1 = 531531 + 138 = 0.7937, R1 = 531565 = 0.9398

About 10 minutes

Practice 4: In Colab

  1. Notebook 1, five segments on income and spending: predict [[100, 50], [20, 90]] in one call
  2. Re-run the four-feature model with n_init=10: print inertia_ and np.bincount(kmeans.labels_)
  3. Notebook 2 with random_state=2: print the confusion matrix and accuracy_score
TaskResult
1[1 4]: high/high and low income, high spending
275399.62, sizes [23 39 79 23 36]
3[[74 138] [34 531]], accuracy 0.7786: only the names changed

Key Takeaways

  1. Clustering finds groups in data with no labels
  2. k-means repeats assign (nearest centroid) and update (mean) until nothing changes
  3. The WCSS (inertia_) measures tightness; neither step can raise it
  4. Choose k at the elbow, not at the smallest inertia
  5. The result depends on the start: use n_init
  6. Cluster numbers are arbitrary: read the centroids and name the segments

Open this lesson

Mahmoud Abask-Means Clustering