Mahmoud Abas, Full-Stack Software Engineer & AI

Machine Learning & AI - Fuzzy C-Means Clustering

In the print dialog, choose "Save as PDF" as the destination.

Machine Learning & AI - Lesson 4

Fuzzy C-Means Clustering

Soft clustering where points belong to every cluster by degree

Introduction

Fuzzy C-Means (FCM): A soft clustering algorithm: each point can belong to multiple clusters with a degree of membership, instead of being forced into exactly one cluster.

Unlike K-Means, no point is forced into exactly one cluster
Iteratively updates centroids and memberships until convergence

Algorithm Overview

Start

Initialize Membership Matrix

random values

Calculate Centroids

Compute Distances

Update Memberships

Converged?

converged

End

Membership matrix starts random: rows = clusters, columns = points
Constraint: for each point, memberships across all clusters sum to 1
Stops when memberships stabilize or a max iteration count is hit

Membership Matrix and Centroids

Membership matrix: random values in [0,1], each column (point) sums to 1
Centroid = weighted average of all points, weighted by membership raised to the power m

V_ij = ∑k γ_ik^m · x_k∑k γ_ik^m

m: The fuzziness parameter, typically set to 2.

Calculating Distances

Euclidean distance from every point to every cluster centroid

D_ki = √((x_k - V_i1)² + (y_k - V_i2)²)

These distances drive the next membership update

Updating Membership Values

Points closer to a centroid get higher membership in that cluster

γ_ki = (∑j (d_ki² / d_kj²)^1/(m-1))^-1

Membership shifts each iteration as centroids move

Convergence Check

Compare updated memberships to the previous iteration
Stop when the maximum change is below a tolerance (e.g. 0.01)

Worked Example

Setup

Data points: (1,3), (2,5), (4,8), (7,9) - 2 clusters, m = 2

Cluster	(1,3)	(2,5)	(4,8)	(7,9)
1	0.8	0.7	0.2	0.1
2	0.2	0.3	0.8	0.9

Worked Example

Computing the Centroids

Find the two cluster centroids from the initial membership matrix

Cluster 1 centroid: (1.568, 4.051) - pulled toward the low-index points
Cluster 2 centroid: (5.35, 8.215) - pulled toward the high-index points
Each coordinate is a membership-squared-weighted average of all 4 points

Worked Example

Distances and Updated Memberships

Find the distance and updated membership for points (1,3) and (7,9)

Point	Distance to C1	Distance to C2	Membership
(1,3)	1.2	6.79	[0.97, 0.03]
(7,9)	7.34	1.82	[0.06, 0.94]

Memberships sharpen toward the nearer centroid after just one iteration
Iteration continues until changes fall below the tolerance

The Fuzziness Parameter (`m`)

Value	Effect
`m = 1`	Hard clustering - equivalent to `K-Means`
`m = 2`	Standard fuzzy clustering (most common choice)
`m > 2`	More fuzziness - points spread membership more equally

Applications, Advantages, and Limitations

Category	Details
Applications	Image segmentation, bioinformatics, customer segmentation, medical diagnosis
Advantages	Soft assignments, handles overlapping clusters, less sensitive to initialization
Limitations	May converge to local optima, needs the cluster count set upfront, costlier than `K-Means`, sensitive to noise

Key Takeaways

FCM assigns each point a membership degree to every cluster, not a hard label
Centroids are membership-weighted averages; memberships depend on distance
The fuzziness parameter m controls how soft the boundaries are
Iterate centroid, distance, and membership updates until convergence
Best suited for data with naturally overlapping or ambiguous categories

Open this lesson

Mahmoud AbasFuzzy C-Means Clustering

Cluster

(1,3)

(2,5)

(4,8)

(7,9)

0.8

0.7

0.2

0.1

0.2

0.3

0.8

0.9

Point

Distance to C1

Distance to C2

Membership

(1,3)

1.2

6.79

[0.97, 0.03]

(7,9)

7.34

1.82

[0.06, 0.94]

Value

Effect

m = 1

Hard clustering - equivalent to K-Means

m = 2

Standard fuzzy clustering (most common choice)

m > 2

More fuzziness - points spread membership more equally

Fuzzy C-Means Clustering

Introduction

Algorithm Overview

Membership Matrix and Centroids

Calculating Distances

Updating Membership Values

Convergence Check

Setup

Computing the Centroids

Distances and Updated Memberships

The Fuzziness Parameter (m)

Applications, Advantages, and Limitations

Key Takeaways

Open this lesson

Fuzzy C-Means Clustering

Introduction

Algorithm Overview

Membership Matrix and Centroids

Calculating Distances

Updating Membership Values

Convergence Check

Setup

Computing the Centroids

Distances and Updated Memberships

The Fuzziness Parameter (m)

Applications, Advantages, and Limitations

Key Takeaways

Open this lesson

The Fuzziness Parameter (`m`)

The Fuzziness Parameter (`m`)