Building a CNN to classify images in the CIFAR-10 Dataset

10 min read

Lesson slides

1 / 12

Building a CNN to Classify Images in the CIFAR-10 Dataset

Convolutional layers, a first CNN, and data augmentation

Building a CNN to classify images in the CIFAR-10 Dataset

We will work with the CIFAR-10 Dataset. This is a well-known dataset for image classification, which consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

The 10 classes are:

airplane
automobile
bird
cat
deer
dog
frog
horse
ship
truck

For details about CIFAR-10 see: https://www.cs.toronto.edu/~kriz/cifar.html

For a compilation of published performance results on CIFAR 10, see: http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html

Building Convolutional Neural Nets

In this exercise we will build and train our first convolutional neural networks. In the first part, we walk through the different layers and how they are configured. In the second part, you will build your own model, train it, and compare the performance.

import keras
from keras.datasets import cifar10
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D
 
import matplotlib.pyplot as plt
%matplotlib inline

# The data, shuffled and split between train and test sets:
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

x_train shape: (50000, 32, 32, 3)
50000 train samples
10000 test samples

## Each image is a 32 x 32 x 3 numpy array
x_train[444].shape

(32, 32, 3)

## Let's look at one of the images
 
print(y_train[444])
plt.imshow(x_train[444]);

[9]

num_classes = 10
 
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

# now instead of classes described by an integer between 0-9 we have a vector with a 1 in the (Pythonic) 9th position
y_train[444]

array([0., 0., 0., 0., 0., 0., 0., 0., 0., 1.])

# As before, let's make everything float and scale
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

Keras Layers for CNNs

Previously we built Neural Networks using primarily the Dense, Activation and Dropout Layers.
Here we will describe how to use some of the CNN-specific layers provided by Keras

Conv2D

keras.layers.convolutional.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, **kwargs)

A few parameters explained:

filters: the number of filter used per location. In other words, the depth of the output.
kernel_size: an (x,y) tuple giving the height and width of the kernel to be used
strides: and (x,y) tuple giving the stride in each dimension. Default is (1,1)
input_shape: required only for the first layer

Note, the size of the output will be determined by the kernel_size, strides

MaxPooling2D

keras.layers.pooling.MaxPooling2D(pool_size=(2, 2), strides=None, padding='valid', data_format=None)

pool_size: the (x,y) size of the grid to be pooled.
strides: Assumed to be the pool_size unless otherwise specified

Flatten

Turns its input into a one-dimensional vector (per instance). Usually used when transitioning between convolutional layers and fully connected layers.

First CNN

Below we will build our first CNN. For demonstration purposes (so that it will train quickly) it is not very deep and has relatively few parameters. We use strides of 2 in the first two convolutional layers which quickly reduces the dimensions of the output. After a MaxPooling layer, we flatten, and then have a single fully connected layer before our final classification layer.

# Let's build a CNN using Keras' Sequential capabilities
 
model_1 = Sequential()
 
 
## 5x5 convolution with 2x2 stride and 32 filters
model_1.add(Conv2D(32, kernel_size = (5, 5), strides = (2,2), padding='same',
                 input_shape=x_train.shape[1:]))
model_1.add(Activation('relu'))
 
## Another 5x5 convolution with 2x2 stride and 32 filters
model_1.add(Conv2D(32, (5, 5), strides = (2,2), activation="relu")) # another way to define activation
 
 
## 2x2 max pooling reduces to 3 x 3 x 32
model_1.add(MaxPooling2D(pool_size=(2, 2)))
model_1.add(Dropout(0.25))
 
## Flatten turns 3x3x32 into 288x1
model_1.add(Flatten())
model_1.add(Dense(512))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.5))
model_1.add(Dense(num_classes))
model_1.add(Activation('softmax'))
 
model_1.summary()

/usr/local/lib/python3.11/dist-packages/keras/src/layers/convolutional/base_conv.py:107: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.
  super().__init__(activity_regularizer=activity_regularizer, **kwargs)
[1mModel: "sequential"[0m
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃[1m [0m[1mLayer (type)                   [0m[1m [0m┃[1m [0m[1mOutput Shape          [0m[1m [0m┃[1m [0m[1m      Param #[0m[1m [0m┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d ([38;5;33mConv2D[0m)                 │ ([38;5;45mNone[0m, [38;5;34m16[0m, [38;5;34m16[0m, [38;5;34m32[0m)     │         [38;5;34m2,432[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ activation ([38;5;33mActivation[0m)         │ ([38;5;45mNone[0m, [38;5;34m16[0m, [38;5;34m16[0m, [38;5;34m32[0m)     │             [38;5;34m0[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_1 ([38;5;33mConv2D[0m)               │ ([38;5;45mNone[0m, [38;5;34m6[0m, [38;5;34m6[0m, [38;5;34m32[0m)       │        [38;5;34m25,632[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d ([38;5;33mMaxPooling2D[0m)    │ ([38;5;45mNone[0m, [38;5;34m3[0m, [38;5;34m3[0m, [38;5;34m32[0m)       │             [38;5;34m0[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout ([38;5;33mDropout[0m)               │ ([38;5;45mNone[0m, [38;5;34m3[0m, [38;5;34m3[0m, [38;5;34m32[0m)       │             [38;5;34m0[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten ([38;5;33mFlatten[0m)               │ ([38;5;45mNone[0m, [38;5;34m288[0m)            │             [38;5;34m0[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense ([38;5;33mDense[0m)                   │ ([38;5;45mNone[0m, [38;5;34m512[0m)            │       [38;5;34m147,968[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ activation_1 ([38;5;33mActivation[0m)       │ ([38;5;45mNone[0m, [38;5;34m512[0m)            │             [38;5;34m0[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_1 ([38;5;33mDropout[0m)             │ ([38;5;45mNone[0m, [38;5;34m512[0m)            │             [38;5;34m0[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 ([38;5;33mDense[0m)                 │ ([38;5;45mNone[0m, [38;5;34m10[0m)             │         [38;5;34m5,130[0m │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ activation_2 ([38;5;33mActivation[0m)       │ ([38;5;45mNone[0m, [38;5;34m10[0m)             │             [38;5;34m0[0m │
└─────────────────────────────────┴────────────────────────┴───────────────┘
[1m Total params: [0m[38;5;34m181,162[0m (707.66 KB)
[1m Trainable params: [0m[38;5;34m181,162[0m (707.66 KB)
[1m Non-trainable params: [0m[38;5;34m0[0m (0.00 B)

We still have 181K parameters, even though this is a "small" model.

batch_size = 32
 
# initiate RMSprop optimizer
opt = keras.optimizers.RMSprop(learning_rate=0.0005)
 
# Let's train the model using RMSprop
model_1.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])
 
model_1.fit(x_train, y_train,
              batch_size=batch_size,
              epochs=15,
              validation_data=(x_test, y_test),
              shuffle=True)

Epoch 1/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m13s[0m 6ms/step - accuracy: 0.3136 - loss: 1.8675 - val_accuracy: 0.4506 - val_loss: 1.5152
Epoch 2/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.4739 - loss: 1.4538 - val_accuracy: 0.5385 - val_loss: 1.2852
Epoch 3/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m5s[0m 3ms/step - accuracy: 0.5191 - loss: 1.3548 - val_accuracy: 0.5606 - val_loss: 1.2484
Epoch 4/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5435 - loss: 1.3027 - val_accuracy: 0.5620 - val_loss: 1.2574
Epoch 5/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5592 - loss: 1.2648 - val_accuracy: 0.5616 - val_loss: 1.2314
Epoch 6/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5674 - loss: 1.2424 - val_accuracy: 0.5452 - val_loss: 1.3458
Epoch 7/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m10s[0m 4ms/step - accuracy: 0.5718 - loss: 1.2406 - val_accuracy: 0.5917 - val_loss: 1.1967
Epoch 8/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m5s[0m 4ms/step - accuracy: 0.5748 - loss: 1.2402 - val_accuracy: 0.6114 - val_loss: 1.1200
Epoch 9/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5711 - loss: 1.2464 - val_accuracy: 0.5790 - val_loss: 1.2071
Epoch 10/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m10s[0m 4ms/step - accuracy: 0.5820 - loss: 1.2403 - val_accuracy: 0.4886 - val_loss: 1.5045
Epoch 11/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5753 - loss: 1.2391 - val_accuracy: 0.6053 - val_loss: 1.1649
Epoch 12/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m10s[0m 4ms/step - accuracy: 0.5779 - loss: 1.2530 - val_accuracy: 0.5396 - val_loss: 1.3305
Epoch 13/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5763 - loss: 1.2622 - val_accuracy: 0.5561 - val_loss: 1.2769
Epoch 14/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5788 - loss: 1.2512 - val_accuracy: 0.5679 - val_loss: 1.2698
Epoch 15/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m6s[0m 4ms/step - accuracy: 0.5731 - loss: 1.2575 - val_accuracy: 0.5863 - val_loss: 1.2251
<keras.src.callbacks.history.History at 0x7b0b01452b50>

Exercise

Our previous model had the structure:

Conv -> Conv -> MaxPool -> (Flatten) -> Dense -> Final Classification

(with appropriate activation functions and dropouts)

Build a more complicated model with the following pattern:

Conv -> Conv -> MaxPool -> Conv -> Conv -> MaxPool -> (Flatten) -> Dense -> Final Classification
Use strides of 1 for all convolutional layers.

How many parameters does your model have? How does that compare to the previous model?
Train it for 5 epochs. What do you notice about the training time, loss and accuracy numbers (on both the training and validation sets)?
Try different structures and run times, and see how accurate your model can be.

# Please provide your solution here
# Create model_2 as mentioned in the exercise

Classifying CIFAR-10 with Data Augmentation

Data augmentation is used to artificially expand your training dataset by creating modified versions of existing images.

These modifications help your model become more robust, generalize better, and avoid overfitting.

Next, we revisit CIFAR-10 and the networks we previously built. We will use real-time data augmentation to try to improve our results.

When you are done going through the notebook, experiment with different data augmentation parameters and see if they help (or hurt!) the performance of your classifier.

from tensorflow.keras.preprocessing.image import ImageDataGenerator
 
datagen = ImageDataGenerator(
    rotation_range=20,  # randomly rotate images in the range (degrees, 0 to 180)
    width_shift_range=0.1,  # randomly shift images horizontally (fraction of total width)
    height_shift_range=0.1,  # randomly shift images vertically (fraction of total height)
    horizontal_flip=False,  # randomly flip images
    shear_range=0.1,   # Shear intensity (shear angle in counter-clockwise direction)
    zoom_range=0.1,    # Zoom in/out on images
    fill_mode='nearest'   # Strategy to fill in new pixels
    )

# Let's build a CNN using Keras' Sequential capabilities
model_1 = Sequential()
 
## 5x5 convolution with 2x2 stride and 32 filters
model_1.add(Conv2D(32, kernel_size = (5, 5), strides = (2,2), padding='same',
                 input_shape=x_train.shape[1:]))
model_1.add(Activation('relu'))
 
## Another 5x5 convolution with 2x2 stride and 32 filters
model_1.add(Conv2D(32, (5, 5), strides = (2,2), activation="relu")) # another way to define activation
 
 
## 2x2 max pooling reduces to 3 x 3 x 32
model_1.add(MaxPooling2D(pool_size=(2, 2)))
#model_1.add(Dropout(0.25))
 
## Flatten turns 3x3x32 into 288x1
model_1.add(Flatten())
model_1.add(Dense(512))
model_1.add(Activation('relu'))
#model_1.add(Dropout(0.5))
model_1.add(Dense(num_classes))
model_1.add(Activation('softmax'))
 
batch_size = 32
 
# Let's train the model using RMSprop
model_1.compile(loss='categorical_crossentropy',
                optimizer='rmsprop',
                metrics=['accuracy'])
 
# Fit the model on the batches generated by datagen.flow().
model_1.fit(
    datagen.flow(x_train, y_train, batch_size=batch_size),
    epochs=15,
    validation_data=(x_test, y_test)
)

Epoch 1/15
/usr/local/lib/python3.11/dist-packages/keras/src/trainers/data_adapters/py_dataset_adapter.py:121: UserWarning: Your `PyDataset` class should call `super().__init__(**kwargs)` in its constructor. `**kwargs` can include `workers`, `use_multiprocessing`, `max_queue_size`. Do not pass these arguments to `fit()`, as they will be ignored.
  self._warn_if_super_not_called()
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m33s[0m 20ms/step - accuracy: 0.3195 - loss: 1.8580 - val_accuracy: 0.5205 - val_loss: 1.3476
Epoch 2/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m30s[0m 19ms/step - accuracy: 0.4840 - loss: 1.4376 - val_accuracy: 0.5217 - val_loss: 1.3634
Epoch 3/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m30s[0m 19ms/step - accuracy: 0.5274 - loss: 1.3324 - val_accuracy: 0.5750 - val_loss: 1.1765
Epoch 4/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m41s[0m 19ms/step - accuracy: 0.5594 - loss: 1.2471 - val_accuracy: 0.6083 - val_loss: 1.1206
Epoch 5/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m30s[0m 19ms/step - accuracy: 0.5745 - loss: 1.2118 - val_accuracy: 0.6015 - val_loss: 1.1356
Epoch 6/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m29s[0m 19ms/step - accuracy: 0.5858 - loss: 1.1726 - val_accuracy: 0.6154 - val_loss: 1.1121
Epoch 7/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m42s[0m 19ms/step - accuracy: 0.6007 - loss: 1.1575 - val_accuracy: 0.6197 - val_loss: 1.0995
Epoch 8/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m33s[0m 21ms/step - accuracy: 0.6035 - loss: 1.1363 - val_accuracy: 0.6273 - val_loss: 1.0985
Epoch 9/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m38s[0m 19ms/step - accuracy: 0.6074 - loss: 1.1399 - val_accuracy: 0.6604 - val_loss: 0.9989
Epoch 10/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m31s[0m 20ms/step - accuracy: 0.6181 - loss: 1.1066 - val_accuracy: 0.6279 - val_loss: 1.1005
Epoch 11/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m30s[0m 19ms/step - accuracy: 0.6132 - loss: 1.1133 - val_accuracy: 0.6527 - val_loss: 1.0322
Epoch 12/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m31s[0m 20ms/step - accuracy: 0.6201 - loss: 1.1032 - val_accuracy: 0.6485 - val_loss: 1.0506
Epoch 13/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m30s[0m 19ms/step - accuracy: 0.6232 - loss: 1.1023 - val_accuracy: 0.6452 - val_loss: 1.0841
Epoch 14/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m31s[0m 20ms/step - accuracy: 0.6258 - loss: 1.0899 - val_accuracy: 0.6548 - val_loss: 1.0765
Epoch 15/15
[1m1563/1563[0m [32m━━━━━━━━━━━━━━━━━━━━[0m[37m[0m [1m29s[0m 19ms/step - accuracy: 0.6221 - loss: 1.1078 - val_accuracy: 0.6285 - val_loss: 1.0848
<keras.src.callbacks.history.History at 0x7b0b01450390>

How does the performance compare with the non-augmented training?

Exercise

Your Turn

Experiment above with different settings of the data augmentation parameters. Can you make the model do better? Can you make it do worse?