Mahmoud Abas, Full-Stack Software Engineer & AI

Software Construction - Building a CNN to classify images in the CIFAR-10 Dataset

In the print dialog, choose "Save as PDF" as the destination.

Building a CNN for CIFAR-10

Convolutional layers, a first CNN, and boosting it with data augmentation

The CIFAR-10 Dataset

60,000 32x32 color images across 10 classes, 6,000 images per class
50,000 training images, 10,000 test images
The 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck
Goal: walk through CNN-specific Keras layers, build a first CNN, then build a deeper one and compare performance

Loading and Preparing the Data

python

from keras.datasets import cifar10

(x_train, y_train), (x_test, y_test) = cifar10.load_data()
print('x_train shape:', x_train.shape)   # (50000, 32, 32, 3)

# Each image is a 32x32x3 array (height, width, RGB channels)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255   # scale pixel values to [0, 1]
x_test /= 255

Preparing the Labels

python

num_classes = 10
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

# Instead of an integer 0-9, each label becomes a one-hot vector
y_train[444]
# array([0., 0., 0., 0., 0., 0., 0., 0., 0., 1.])

One-hot encoding is required to train with categorical_crossentropy loss

Keras CNN Layers: Conv2D

python

keras.layers.Conv2D(filters, kernel_size, strides=(1, 1),
                     padding='valid', activation=None, ...)

Parameter	Meaning
`filters`	Number of filters applied per location, the depth of the output
`kernel_size`	`(x, y)` tuple giving the height/width of the convolution kernel
`strides`	`(x, y)` tuple giving the step in each dimension, default `(1, 1)`
`input_shape`	Required only for the first layer in the model

Output size is determined by kernel_size and strides

Keras CNN Layers: MaxPooling2D and Flatten

python

keras.layers.MaxPooling2D(pool_size=(2, 2), strides=None, padding='valid')

Parameter	Meaning
`pool_size`	The `(x, y)` grid size to pool over
`strides`	Defaults to `pool_size` if not specified

Flatten: Turns its input into a one-dimensional vector per instance, used when transitioning from convolutional layers to fully connected (Dense) layers

Building the First CNN

python

model_1 = Sequential()

# 5x5 convolution, stride 2, 32 filters
model_1.add(Conv2D(32, kernel_size=(5, 5), strides=(2, 2),
                    padding='same', input_shape=x_train.shape[1:]))
model_1.add(Activation('relu'))
model_1.add(Conv2D(32, (5, 5), strides=(2, 2), activation="relu"))

model_1.add(MaxPooling2D(pool_size=(2, 2)))   # reduces to 3x3x32
model_1.add(Dropout(0.25))

model_1.add(Flatten())                         # 3x3x32 -> 288x1
model_1.add(Dense(512))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.5))
model_1.add(Dense(num_classes))
model_1.add(Activation('softmax'))

Conv2D

5x5, stride 2

Conv2D

5x5, stride 2

MaxPool

2x2

Flatten

Dense

512

Softmax

10 classes

Model Summary and Parameter Count

python

model_1.summary()

Layer	Output Shape	Param #
conv2d	(None, 16, 16, 32)	2,432
conv2d_1	(None, 6, 6, 32)	25,632
max_pooling2d	(None, 3, 3, 32)	0
flatten	(None, 288)	0
dense	(None, 512)	147,968
dense_1	(None, 10)	5,130

Compiling and Training the CNN

python

model_1.compile(loss='categorical_crossentropy',
                 optimizer='rmsprop',
                 metrics=['accuracy'])

model_1.fit(x_train, y_train,
            batch_size=32,
            epochs=15,
            validation_data=(x_test, y_test),
            shuffle=True)

Training accuracy plateaus around ~57-58%, validation accuracy fluctuates around ~55-60% over 15 epochs, a shallow CNN has limited capacity

Exercise

A Deeper CNN

Conv2D

MaxPool

Conv2D

MaxPool

Flatten

Dense

Softmax

Build the deeper pattern above, using stride 1 for all convolutional layers
Compare: how many parameters does the deeper model have versus the first one?
Train it for 5 epochs and compare training time, loss, and accuracy on both training and validation sets
Try different architectures and training durations to see how high accuracy can go

Data Augmentation

Data augmentation: Artificially expands the training set by creating modified versions of existing images, improves robustness, generalization, and reduces overfitting

python

from tensorflow.keras.preprocessing.image import ImageDataGenerator

datagen = ImageDataGenerator(
    rotation_range=20,       # random rotation, degrees
    width_shift_range=0.1,   # random horizontal shift (fraction of width)
    height_shift_range=0.1,  # random vertical shift (fraction of height)
    shear_range=0.1,         # shear intensity
    zoom_range=0.1,          # random zoom
    fill_mode='nearest'      # how to fill new pixels
)

Training with Augmented Data

python

model_1.fit(
    datagen.flow(x_train, y_train, batch_size=batch_size),
    epochs=15,
    validation_data=(x_test, y_test)
)

With the same architecture, augmented training reaches ~62% training accuracy and ~63% validation accuracy, a meaningful improvement over the non-augmented run, though each epoch takes longer
Exercise: experiment with the augmentation parameters, can you make the model better, or worse?

Key Takeaways

CIFAR-10 is a 10-class, 32x32 color image classification benchmark (50,000 train / 10,000 test images)
Conv2D extracts spatial features via filters and strides; MaxPooling2D downsamples; Flatten bridges convolutional output into Dense layers
A shallow CNN (two conv layers + one dense layer) already has 100K+ parameters and reaches roughly 55-60% validation accuracy on CIFAR-10
Going deeper (more Conv/MaxPool blocks) changes parameter count and can improve accuracy, at the cost of more training time
Data augmentation (rotation, shift, shear, zoom) expands the effective training set and improves generalization without collecting new data
Always compare training vs validation metrics to catch overfitting early

Open this lesson

Mahmoud AbasBuilding a CNN to classify images in the CIFAR-10 Dataset

from keras.datasets import cifar10 (x_train, y_train), (x_test, y_test) = cifar10.load_data() print('x_train shape:', x_train.shape) # (50000, 32, 32, 3) # Each image is a 32x32x3 array (height, width, RGB channels) x_train = x_train.astype('float32') x_test = x_test.astype('float32') x_train /= 255 # scale pixel values to [0, 1] x_test /= 255

num_classes = 10 y_train = keras.utils.to_categorical(y_train, num_classes) y_test = keras.utils.to_categorical(y_test, num_classes) # Instead of an integer 0-9, each label becomes a one-hot vector y_train[444] # array([0., 0., 0., 0., 0., 0., 0., 0., 0., 1.])

Parameter

Meaning

filters

Number of filters applied per location, the depth of the output

kernel_size

(x, y) tuple giving the height/width of the convolution kernel

strides

(x, y) tuple giving the step in each dimension, default (1, 1)

input_shape

Required only for the first layer in the model

Parameter

Meaning

pool_size

The (x, y) grid size to pool over

strides

Defaults to pool_size if not specified

model_1 = Sequential() # 5x5 convolution, stride 2, 32 filters model_1.add(Conv2D(32, kernel_size=(5, 5), strides=(2, 2), padding='same', input_shape=x_train.shape[1:])) model_1.add(Activation('relu')) model_1.add(Conv2D(32, (5, 5), strides=(2, 2), activation="relu")) model_1.add(MaxPooling2D(pool_size=(2, 2))) # reduces to 3x3x32 model_1.add(Dropout(0.25)) model_1.add(Flatten()) # 3x3x32 -> 288x1 model_1.add(Dense(512)) model_1.add(Activation('relu')) model_1.add(Dropout(0.5)) model_1.add(Dense(num_classes)) model_1.add(Activation('softmax'))

Layer

Output Shape

Param #

conv2d

(None, 16, 16, 32)

2,432

conv2d_1

(None, 6, 6, 32)

25,632

max_pooling2d

(None, 3, 3, 32)

flatten

(None, 288)

dense

(None, 512)

147,968

dense_1

(None, 10)

5,130

model_1.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy']) model_1.fit(x_train, y_train, batch_size=32, epochs=15, validation_data=(x_test, y_test), shuffle=True)

from tensorflow.keras.preprocessing.image import ImageDataGenerator datagen = ImageDataGenerator( rotation_range=20, # random rotation, degrees width_shift_range=0.1, # random horizontal shift (fraction of width) height_shift_range=0.1, # random vertical shift (fraction of height) shear_range=0.1, # shear intensity zoom_range=0.1, # random zoom fill_mode='nearest' # how to fill new pixels )