Mahmoud Abas, Full-Stack Software Engineer & AI

Software Construction - Keras Sequential model

In the print dialog, choose "Save as PDF" as the destination.

Software Construction - Lesson 8

Keras Sequential Model

Building, inspecting, and training a plain stack of layers with the Sequential model

When to Use a Sequential Model

Sequential model: Fits a plain stack of layers where each layer has exactly one input tensor and one output tensor.

python

model = keras.Sequential([
    keras.Input(shape=(3, 1)),
    layers.Dense(2, activation="relu", name="layer1"),
    layers.Dense(3, activation="relu", name="layer2"),
    layers.Dense(4, name="layer3"),
])

Input

shape=(3, 1)

layer1

Dense(2, relu)

layer2

Dense(3, relu)

layer3

Dense(4)

Not appropriate when: the model has multiple inputs or outputs, any layer has multiple inputs or outputs, you need layer sharing, or you want non-linear topology (for example a residual connection or a multi-branch model)

Creating a Sequential Model

python

# Pass a list of layers to the constructor
model = keras.Sequential([
    layers.Dense(2, activation="relu"),
    layers.Dense(3, activation="relu"),
    layers.Dense(4),
])

# Or build incrementally
model = keras.Sequential()
model.add(layers.Dense(2, activation="relu"))
model.add(layers.Dense(3, activation="relu"))
model.add(layers.Dense(4))
model.pop()   # removes the last layer - behaves like a list of layers

Layers are accessible via model.layers
The constructor also accepts a name argument, useful for annotating TensorBoard graphs

Weights Are Created Lazily

Lazy weight creation: A layer has no weights until it is first called on an input; the weight shape depends on the input shape.

python

layer = layers.Dense(3)
layer.weights          # [] - empty

x = ops.ones((1, 4))
y = layer(x)
layer.weights           # now has weights: shape (4, 3) and (3,)

The same applies to Sequential models: without a specified input shape, model.weights is empty until the model first sees data

Specifying the Input Shape in Advance

python

model = keras.Sequential()
model.add(keras.Input(shape=(4,)))
model.add(layers.Dense(2, activation="relu"))

model.summary()   # works immediately - model is already "built"

The Input object itself is not part of model.layers, it is not a layer
Models built with a predefined input shape always have weights and a defined output shape, even before seeing data

Model Summary

python

model.summary()

Layer (type)	Output Shape	Param #
dense (Dense)	(1, 2)	10
dense_1 (Dense)	(1, 3)	9
dense_2 (Dense)	(1, 4)	16

Model "sequential", Total params: 35
Shows every layer's type, output shape, and parameter count, essential for sanity-checking an architecture before training

Training Workflow: Preparing Data

python

model = keras.Sequential()
model.add(keras.Input(shape=(784,)))
model.add(layers.Dense(64, activation="relu"))
model.add(layers.Dense(64, activation="relu"))
model.add(layers.Dense(10, activation="softmax"))

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype("float32") / 255
x_test = x_test.reshape(10000, 784).astype("float32") / 255

# Reserve 10,000 samples for validation
x_val, y_val = x_train[-10000:], y_train[-10000:]
x_train, y_train = x_train[:-10000], y_train[:-10000]

Training

fit on the training split

Validation

holdout set from training data

Evaluation

scored on the test set

End-to-end workflow: training, validation on a holdout set, evaluation on test data

compile(): Loss, Optimizer, Metrics

python

model.compile(
    optimizer=keras.optimizers.RMSprop(learning_rate=1e-3),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[keras.metrics.SparseCategoricalAccuracy()],
)

# Shortcut with string identifiers for default settings
model.compile(optimizer="rmsprop",
              loss="sparse_categorical_crossentropy",
              metrics=["sparse_categorical_accuracy"])

The string-identifier shortcut below uses defaults for the optimizer, loss, and metrics.

Loss / metric	When to use	Label format
`SparseCategoricalCrossentropy` / `SparseCategoricalAccuracy`	Integer labels	`0, 1, 2, ...`
`categorical_crossentropy` / `categorical_accuracy`	Multi-class, one-hot labels	`[0, 0, 1]`, `[1, 0, 0]`
`binary_crossentropy` / `BinaryAccuracy`	Binary classification	`[0, 1, 1, 0]`

metrics is a list; a model can track any number of metrics

fit(): Training the Model

python

history = model.fit(
    x_train, y_train,
    batch_size=64,
    epochs=2,
    validation_data=(x_val, y_val),
)

Parameter	Description
`x`	Input data (`NumPy` array, `Tensor`, or `Dataset`)
`y`	Target labels, same format as the model's output
`batch_size`	Number of samples per gradient update (default `32`)
`epochs`	Number of passes through the entire dataset
`validation_data`	Tuple `(val_x, val_y)` evaluated at the end of each epoch
`shuffle`	Whether to shuffle training data before each epoch (default `True`)
`verbose`	`0` silent, `1` progress bar, `2` one line per epoch

Slices data into batches of batch_size, iterating over the full dataset for epochs passes
The returned history object records loss and metric values at every epoch

evaluate() and predict()

python

results = model.evaluate(x_test, y_test, batch_size=128)
print("test loss, test acc:", results)

predictions = model.predict(x_test[:3])   # probabilities from the last layer
print("predictions shape:", predictions.shape)

evaluate() reports loss and metrics on held-out test data
predict() runs inference and returns the model's raw output (for example, class probabilities)

Self-Study

Keras Functional API

The functional API is more flexible than Sequential: it supports non-linear topology, shared layers, and multiple inputs or outputs
A deep learning model is a directed acyclic graph (DAG) of layers; the functional API builds that graph explicitly

Input

784-dimensional vector

Dense

64 units, relu

Dense

64 units, relu

Dense

10 units, softmax

python

inputs = keras.Input(shape=(784,))
x = layers.Dense(64, activation="relu")(inputs)
x = layers.Dense(64, activation="relu")(x)
outputs = layers.Dense(10)(x)

model = keras.Model(inputs=inputs, outputs=outputs, name="mnist_model")

Calling a layer on a tensor draws an arrow in the graph: each call passes data forward and returns the layer's output
keras.utils.plot_model(model, "my_model.png", show_shapes=True) visualizes the graph

Key Takeaways

Sequential models suit a simple, single-input/single-output stack of layers; use the functional API for branching, sharing, or multi-input/output architectures
Layer and model weights are created lazily, sized by the first input they see; specify Input(shape=...) up front to build immediately and enable summary()
compile() wires together an optimizer, a loss function, and metrics to monitor; match the loss to your label format (sparse integer vs one-hot vs binary)
fit() trains over batches and epochs, optionally validating each epoch; history records the training curve
evaluate() measures final performance on held-out data; predict() returns raw model output for new inputs
The functional API models a network as an explicit graph of layer calls, enabling non-linear topologies the Sequential API cannot express

Open this lesson

Mahmoud AbasKeras Sequential model

model = keras.Sequential([ keras.Input(shape=(3, 1)), layers.Dense(2, activation="relu", name="layer1"), layers.Dense(3, activation="relu", name="layer2"), layers.Dense(4, name="layer3"), ])

# Pass a list of layers to the constructor model = keras.Sequential([ layers.Dense(2, activation="relu"), layers.Dense(3, activation="relu"), layers.Dense(4), ]) # Or build incrementally model = keras.Sequential() model.add(layers.Dense(2, activation="relu")) model.add(layers.Dense(3, activation="relu")) model.add(layers.Dense(4)) model.pop() # removes the last layer - behaves like a list of layers

Layer (type)

Output Shape

Param #

dense (Dense)

(1, 2)

dense_1 (Dense)

(1, 3)

dense_2 (Dense)

(1, 4)

model = keras.Sequential() model.add(keras.Input(shape=(784,))) model.add(layers.Dense(64, activation="relu")) model.add(layers.Dense(64, activation="relu")) model.add(layers.Dense(10, activation="softmax")) (x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data() x_train = x_train.reshape(60000, 784).astype("float32") / 255 x_test = x_test.reshape(10000, 784).astype("float32") / 255 # Reserve 10,000 samples for validation x_val, y_val = x_train[-10000:], y_train[-10000:] x_train, y_train = x_train[:-10000], y_train[:-10000]

model.compile( optimizer=keras.optimizers.RMSprop(learning_rate=1e-3), loss=keras.losses.SparseCategoricalCrossentropy(), metrics=[keras.metrics.SparseCategoricalAccuracy()], ) # Shortcut with string identifiers for default settings model.compile(optimizer="rmsprop", loss="sparse_categorical_crossentropy", metrics=["sparse_categorical_accuracy"])

Loss / metric

When to use

Label format

SparseCategoricalCrossentropy / SparseCategoricalAccuracy

Integer labels

0, 1, 2, ...

categorical_crossentropy / categorical_accuracy

Multi-class, one-hot labels

[0, 0, 1], [1, 0, 0]

binary_crossentropy / BinaryAccuracy

Binary classification

[0, 1, 1, 0]

Parameter

Description

x

Input data (NumPy array, Tensor, or Dataset)

y

Target labels, same format as the model's output

batch_size

Number of samples per gradient update (default 32)

epochs

Number of passes through the entire dataset

validation_data

Tuple (val_x, val_y) evaluated at the end of each epoch

shuffle

Whether to shuffle training data before each epoch (default True)

verbose

0 silent, 1 progress bar, 2 one line per epoch

results = model.evaluate(x_test, y_test, batch_size=128) print("test loss, test acc:", results) predictions = model.predict(x_test[:3]) # probabilities from the last layer print("predictions shape:", predictions.shape)

inputs = keras.Input(shape=(784,)) x = layers.Dense(64, activation="relu")(inputs) x = layers.Dense(64, activation="relu")(x) outputs = layers.Dense(10)(x) model = keras.Model(inputs=inputs, outputs=outputs, name="mnist_model")