Mahmoud Abas, Full-Stack Software Engineer & AI

Machine Learning & AI - Lab #1: K-Means Clustering with Synthetic Data

In the print dialog, choose "Save as PDF" as the destination.

Machine Learning and AI - Lab 1

K-Means Clustering with Synthetic Data

Unsupervised learning with scikit-learn: generate, cluster, evaluate, and optimize

Lab Objectives

Generate synthetic, multi-modal data with sklearn.datasets.make_blobs
Apply K-Means clustering via sklearn.cluster.KMeans
Understand key parameters: n_clusters, init, max_iter, n_init
Visualize data points colored by cluster, plus centroids
Evaluate cluster quality with the Silhouette Score
Find the optimal k using the Elbow Method (WCSS)

Generating Synthetic Data

make_blobs creates isotropic Gaussian blobs for clustering experiments
Key parameters: n_samples, n_features, centers, cluster_std, random_state

python

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

X, y_true = make_blobs(n_samples=300, n_features=2,
                        centers=4, cluster_std=0.8,
                        random_state=42)

Visualizing the Ground Truth

Scatter plot of X colored by y_true shows the true clustering
Each color represents one generated cluster
This is the target structure K-Means will try to recover
Useful baseline before running any clustering algorithm

Applying K-Means Clustering

init='k-means++': smart seeding, new centroids favor points far from existing ones

Inertia: The sum of squared distances between each data point and the center of its assigned cluster.

Initialize centroids

k-means++

Run n_init times

different seeds

Compute inertia

per run

Keep lowest inertia

python

kmeans = KMeans(n_clusters=4, init='k-means++',
                 max_iter=300, n_init=10, random_state=42)
kmeans.fit(X)
labels = kmeans.labels_
centroids = kmeans.cluster_centers_

Visualizing Clustering Results

Plot data points colored by assigned cluster (labels)
Overlay centroids as red X markers
Compare visually against the ground-truth plot from the previous step

Evaluating with the Silhouette Score

Silhouette Score: Measures how similar a point is to its own cluster compared to other clusters.

For each point: a = avg distance to same-cluster points, b = avg distance to nearest other cluster

Silhouette = (b - a) / max(a, b)

Score	Meaning
+1	Well-clustered
0	On the boundary
-1	Likely misassigned

The Elbow Method

Plots WCSS against k from 1 to 10 to reveal a bend where improvement slows

WCSS = ∑c∑p d(p,c)²

Try k = 1 to 10

Fit K-Means

record inertia

Plot WCSS vs k

Locate the elbow

python

wcss, silhouette_scores = [], []
for i in range(1, 11):
    km = KMeans(n_clusters=i, init='k-means++',
                max_iter=300, n_init=10, random_state=42)
    km.fit(X)
    wcss.append(km.inertia_)

True vs. Predicted Clusters

Side-by-side plot: ground truth (left) vs. K-Means output (right)
Cluster labels may differ (true cluster 0 could become predicted cluster 2)
What matters is whether points that belong together are grouped together
K-Means recovered the underlying structure closely

Lab Summary

Metric	Value
Samples / Features / True clusters	`300` / `2` / `4`
cluster_std	`0.8`
k used / Initialization	`4` / `k-means++`
WCSS (inertia)	`362.472`
Silhouette Score	`0.834` (excellent)
Best k by silhouette	`4`

Key Takeaways

K-Means works best on spherical, well-separated clusters
k-means++ initialization plus multiple n_init runs avoid poor local minima
The Elbow Method and Silhouette Score are complementary ways to pick k
Visualization is essential to sanity-check both data and results
In real, high-dimensional data, evaluation metrics matter even more since you cannot just "look" at the clusters

Open this lesson

Mahmoud AbasLab #1: K-Means Clustering with Synthetic Data

from sklearn.datasets import make_blobs from sklearn.cluster import KMeans from sklearn.metrics import silhouette_score X, y_true = make_blobs(n_samples=300, n_features=2, centers=4, cluster_std=0.8, random_state=42)

Score

Meaning

Well-clustered

On the boundary

-1

Likely misassigned

wcss, silhouette_scores = [], [] for i in range(1, 11): km = KMeans(n_clusters=i, init='k-means++', max_iter=300, n_init=10, random_state=42) km.fit(X) wcss.append(km.inertia_)

Metric

Value

Samples / Features / True clusters

300 / 2 / 4

cluster_std

0.8

k used / Initialization

4 / k-means++

WCSS (inertia)

362.472

Silhouette Score

0.834 (excellent)

Best k by silhouette

4