Mahmoud Abas, Full-Stack Software Engineer & AI

Machine Learning & AI - Lab 2: Self-Organizing Maps (Kohonen Maps)

In the print dialog, choose "Save as PDF" as the destination.

Machine Learning & AI - Lab 2

Self-Organizing Maps (Kohonen Maps)

Competitive learning for clustering, dimensionality reduction, and visualization of high-dimensional data

Introduction to Self-Organizing Maps

Self-Organizing Map (SOM): An unsupervised neural network algorithm based on biological neural models from the 1970s. Also called a Kohonen Map.

Uses a competitive learning approach: neurons compete to represent each input
Clustering: groups similar data points together
Dimensionality reduction: maps high-dimensional data to a lower-dimensional space
Visualization: makes complex, high-dimensional datasets easier to interpret

Architecture

Input layer

Represents the features of the data

Output layer

A 2D grid of neurons; each neuron represents a cluster

How SOMs Work

Initialization and Competition

Random weights

Every output neuron starts with randomly initialized weights

Compute distances

Euclidean distance from the input to every neuron's weight vector

BMU wins

The neuron with the smallest distance becomes the Best Matching Unit

D(j) = ∑ni=1 (w_ij - x_i)²

BMU: Best Matching Unit, the neuron whose weight vector w_{ij} is closest to the input feature x_i.

Weight Update and Learning Rate Decay

Only the winning neuron's weights move toward the input vector
The learning rate alpha decays every epoch so the map converges

w_ij^new = w_ij^old + α · (x_i - w_ij^old)

α(t+1) = 0.9 · α(t)

Stopping Condition

Initialize weights

Competition

Find the BMU

Weight update

Decay alpha

Check stopping

Training stops when the maximum number of epochs is reached, or
Weight changes become negligible (convergence)

Implementation: The SOM Class

winner(): finds the BMU by comparing distances to each cluster's weights
update(): nudges the winning neuron's weights toward the sample

python

class SOM:
    def winner(self, weights, sample):
        D0 = sum((sample[i]-weights[0][i])**2 for i in range(len(sample)))
        D1 = sum((sample[i]-weights[1][i])**2 for i in range(len(sample)))
        return 0 if D0 < D1 else 1

    def update(self, weights, sample, J, alpha):
        for i in range(len(weights[0])):
            weights[J][i] += alpha * (sample[i] - weights[J][i])
        return weights

Preparing Training Data

Sample	Features
S1	[1, 1, 0, 0]
S2	[0, 0, 0, 1]
S3	[1, 0, 0, 0]
S4	[0, 0, 1, 1]

2 clusters, randomly initialized weights
Hyperparameters: 10 epochs, initial learning rate alpha = 0.5

Training the SOM Network

For each epoch, find the BMU for every sample
Update the winning neuron's weights toward that sample
Decay the learning rate by x0.9 after every epoch

python

for epoch in range(epochs):
    for j in range(m):
        sample = T[j]
        J = som.winner(weights, sample)
        weights = som.update(weights, sample, J, alpha)
    alpha *= 0.9

All 4 samples converge to a stable cluster assignment across the 10 epochs

Worked Example

Testing the Trained SOM

Classify test sample [0, 0, 0, 1] using the trained weights
Predict which cluster it belongs to

Result	Value
Predicted cluster	0
Cluster 0 weights (approx.)	[0, 0, 0.57, 1.0]
Cluster 1 weights (approx.)	[1.0, 0.43, 0, 0]

Demonstrates the SOM's ability to generalize to unseen input

Visualization and Interpretation

Weight convergence plot: shows each feature's weight stabilizing over epochs
Cluster plot: input samples vs. final weight vectors projected on 2 features
Similar input samples map to the same cluster; weights represent prototypes

Exercise Ideas

Change the decay rate (alpha *= 0.5 vs 0.9): how does convergence change?
Extend the SOM to support 3 or 4 clusters instead of 2
Add a neighborhood function so nearby neurons also update, not just the BMU
Add new training samples like [1, 1, 1, 1] and observe cluster formation

Key Takeaways

SOMs are unsupervised: they cluster and reduce dimensionality without labels
Competitive learning: only the winning neuron gets updated toward the input
Topology preservation: similar inputs map to nearby neurons in the output grid
Learning rate decay is essential for stable convergence
Applications: data visualization, pattern recognition, exploratory data analysis

Open this lesson

Mahmoud AbasLab 2: Self-Organizing Maps (Kohonen Maps)

class SOM: def winner(self, weights, sample): D0 = sum((sample[i]-weights[0][i])**2 for i in range(len(sample))) D1 = sum((sample[i]-weights[1][i])**2 for i in range(len(sample))) return 0 if D0 < D1 else 1 def update(self, weights, sample, J, alpha): for i in range(len(weights[0])): weights[J][i] += alpha * (sample[i] - weights[J][i]) return weights

Sample

Features

[1, 1, 0, 0]

[0, 0, 0, 1]

[1, 0, 0, 0]

[0, 0, 1, 1]

Result

Value

Predicted cluster

Cluster 0 weights (approx.)

[0, 0, 0.57, 1.0]

Cluster 1 weights (approx.)

[1.0, 0.43, 0, 0]