Mahmoud Abas, Full-Stack Software Engineer & AI

Machine Learning & AI - Introduction to TensorFlow 2.x

In the print dialog, choose "Save as PDF" as the destination.

Introduction to TensorFlow 2.x

Eager execution, tensors, and automatic differentiation

Why TensorFlow 2.x?

Open-source deep learning framework from Google
Eager execution by default: operations run immediately, easy to debug
tf.keras is now the recommended high-level API for building models
Runs on CPU, GPU, TPU, and mobile; supports distributed training
Production-ready: deploy to cloud, mobile, and web

Setup: Import and Check Version

python

import tensorflow as tf
print("TensorFlow Version:", tf.__version__)
print("Is GPU available?:", tf.config.list_physical_devices('GPU'))

Always verify the version and hardware before building models

Core Data Structures Overview

tf.Tensor: immutable n-dimensional array, the fundamental data structure
tf.Variable: mutable tensor, used to store trainable parameters
tf.TensorArray: dynamically sized array of tensors (used in RNNs)
tf.RaggedTensor: tensors with variable row lengths
tf.SparseTensor: efficient storage for mostly-zero data

tf.Tensor: Ranks and Shapes

Rank-0

scalar, no axes

Rank-1

vector, one axis

Rank-2

matrix, two axes

python

rank_0 = tf.constant(4)
rank_1 = tf.constant([2.0, 3.0, 4.0])
rank_2 = tf.constant([[1, 2], [3, 4], [5, 6]], dtype=tf.float16)

Tensors can have arbitrarily many axes (dimensions)

Tensor Attributes: shape and dtype

.shape gives size along each axis; .dtype gives the element data type
Convert to NumPy with tensor.numpy() or np.array(tensor)

python

tensor = tf.constant([[1, 2], [3, 4]], dtype=tf.float32)
tensor.shape          # (2, 2)
tensor.dtype          # float32
tensor[0, 0]          # element access
tensor[:, 0]          # slicing

tf.Variable: Mutable State

tf.Variable: Tensors are immutable; model weights need to change, that is what Variable is for

python

var = tf.Variable([[5.0, 6.0], [7.0, 8.0]])
var.assign([[1, 2], [3, 4]])       # replace values
var.assign_add([[1, 1], [1, 1]])   # increment in place

RaggedTensor and SparseTensor

`tf.RaggedTensor`

Handles variable-length rows, e.g. sentences of different lengths

python

ragged = tf.ragged.constant([[1, 2], [3, 4, 5], [6]])

`tf.SparseTensor`

Efficiently stores data that is mostly zeros

python

sparse = tf.SparseTensor(indices=[[0,3],[2,4]], values=[10,20], dense_shape=[3,10])
tf.sparse.to_dense(sparse)

Math and Linear Algebra Operations

Basic arithmetic: tf.add, tf.subtract, tf.multiply, tf.divide
Linear algebra: tf.linalg.matmul, .det, .inv, .eig, .svd

python

A = tf.constant([[2.0, 3.0], [1.0, 4.0]])
B = tf.constant([[1.0, 2.0], [3.0, 4.0]])
tf.linalg.matmul(A, B)

These operations power every neural network under the hood

Random Number Generation

Used for weight initialization, data augmentation, and stochastic models

python

tf.random.normal([3, 3], mean=0.0, stddev=1.0)
tf.random.uniform([2, 2], minval=0, maxval=10)
tf.random.set_seed(42)     # reproducibility

tf.reduce_mean

tf.reduce_mean: Computes the average across specified axes, or all elements if no axis is given

python

x = tf.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.float32)
tf.reduce_mean(x)            # 3.5 (all elements)
tf.reduce_mean(x, axis=0)    # [2.5 3.5 4.5] (column means)
tf.reduce_mean(x, axis=1)    # [2. 5.] (row means)

Eager Execution vs. tf.function

Eager execution: every operation runs immediately, great for debugging
@tf.function: traces Python code into a static computation graph, much faster on repeated calls

Python function

@tf.function

trace

Computation graph

Faster repeated execution

python

@tf.function
def computation_graph(x1, x2):
    return (x1 + x2) - (x1 * x2)

Trade-off: graph mode is faster but has stricter control-flow rules

Automatic Differentiation with GradientTape

TensorFlow uses reverse-mode autodiff to compute gradients automatically
Record operations inside tf.GradientTape(), then call .gradient()

y = x³

At x = 2.0, what is dy/dx?

python

x = tf.Variable(2.0)
with tf.GradientTape() as tape:
    y = x ** 3
dy_dx = tape.gradient(y, x)   # 12.0, since d/dx(x^3) = 3x^2

Works the same way for multiple variables (weights and biases together)

Key Takeaways

TensorFlow 2.x runs eagerly by default: intuitive, Pythonic, easy to debug
tf.Tensor is immutable; tf.Variable is the mutable container for trainable parameters
@tf.function compiles Python into a graph for significant performance gains
tf.GradientTape powers automatic differentiation, the core of model training
These low-level building blocks underlie everything tf.keras does at a higher level

Open this lesson

Mahmoud AbasIntroduction to TensorFlow 2.x

x = tf.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.float32) tf.reduce_mean(x) # 3.5 (all elements) tf.reduce_mean(x, axis=0) # [2.5 3.5 4.5] (column means) tf.reduce_mean(x, axis=1) # [2. 5.] (row means)