Introduction to TensorFlow 2.x

TensorFlow 2.x is an open-source deep learning framework developed by Google that makes it easy to build and train machine learning models. It is widely used for tasks such as image classification, natural language processing (NLP), time series forecasting, and reinforcement learning. With eager execution enabled by default, TensorFlow 2.x provides a more intuitive and Pythonic approach compared to its predecessor, TensorFlow 1.x.

Why TensorFlow 2.x?

TensorFlow 2.x introduces several improvements, making deep learning simpler and more efficient:

• Eager Execution: Executes operations immediately, allowing for easy debugging.
• Keras as the Primary API: tf.keras is now the recommended high-level API for model building.
• End-to-End Machine Learning Workflow: Supports data preprocessing, model training, evaluation, and deployment.
• Works on Any Hardware: Compatible with CPU, GPU, TPU, and even mobile devices.
• Distributed Training: Train across multiple GPUs/TPUs effortlessly.
• Production-Ready: Deploy models to cloud services, mobile apps, and web apps.

# Use this to load your data from Google Drive
# from google.colab import drive
# drive.mount('/content/drive')

Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).

🚀 1. Import TensorFlow and Check Version

Let's begin by importing TensorFlow and verifying that we have the desired version. ✅
We'll also do a quick check for GPU availability. 🔍💻

import tensorflow as tf
print("TensorFlow Version:", tf.__version__)
 
import numpy as np
# Check for GPU
print("Is GPU available?:", tf.config.list_physical_devices('GPU'))

TensorFlow Version: 2.18.0
Is GPU available?: []

🔹 2. TensorFlow Core API

The TensorFlow Core API provides low-level operations and data structures. ⚙️
Working with the Core API helps you understand how tensors, variables, and operations interact under the hood. 🧠

🔢 2.1 Data Structures in TensorFlow

• tf.Tensor 🏗️: Immutable objects that represent n-dimensional arrays.
• tf.Variable 🔄: Mutable versions of tensors (useful for storing parameters).
• tf.TensorArray 📦: A dynamically sized array of tensors, often used in complex RNNs.
• tf.RaggedTensor 📏: Tensors with variable row lengths.
• tf.SparseTensor 🧩: Tensors for representing sparse data.

✨ 2.1.1 tf.Tensor

A tf.Tensor is the fundamental data structure in TensorFlow. 🔢
It represents an immutable, multi-dimensional array. 📊

Below is an example of creating a constant tensor and exploring its attributes. 🛠️

⚡ Immutability:
All tensors are immutable, just like Python numbers and strings. 🔒
You cannot update the contents of a tensor; you can only create a new one. 🔄

🎯 Basics

First, let's create some basic tensors! 🔢✨

Here is a scalar or rank-0 tensor. 🎯

A scalar contains a single value and has no axes. ⚡🔢

# This will be an int32 tensor by default; see "dtypes" below.
rank_0_tensor = tf.constant(4)
print(rank_0_tensor)

tf.Tensor(4, shape=(), dtype=int32)

A vector or rank-1 tensor is like a list of values. 📋

A vector has one axis: 📏

# Let's make this a float tensor.
rank_1_tensor = tf.constant([2.0, 3.0, 4.0])
print(rank_1_tensor)

tf.Tensor([2. 3. 4.], shape=(3,), dtype=float32)

A matrix or rank-2 tensor has two axes: 🏗️📊

# If you want to be specific, you can set the dtype (see below) at creation time
rank_2_tensor = tf.constant([[1, 2],
                             [3, 4],
                             [5, 6]], dtype=tf.float16)
print(rank_2_tensor)

tf.Tensor(
[[1. 2.]
 [3. 4.]
 [5. 6.]], shape=(3, 2), dtype=float16)

Tensors may have more axes; here is a tensor with three axes: 📐🔢

# There can be an arbitrary number of
# axes (sometimes called "dimensions")
rank_3_tensor = tf.constant([
  [[0, 1, 2, 3, 4],
   [5, 6, 7, 8, 9]],
  [[10, 11, 12, 13, 14],
   [15, 16, 17, 18, 19]],
  [[20, 21, 22, 23, 24],
   [25, 26, 27, 28, 29]],])
 
print(rank_3_tensor)

tf.Tensor(
[[[ 0  1  2  3  4]
  [ 5  6  7  8  9]]
 
 [[10 11 12 13 14]
  [15 16 17 18 19]]
 
 [[20 21 22 23 24]
  [25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)

There are many ways to visualize a tensor with more than two axes. 🎨🖼️

You can convert a tensor to a NumPy array using either: 🔄📊

• np.array(tensor) 🐍
• tensor.numpy() ⚡

np.array(rank_2_tensor)

array([[1., 2.],
       [3., 4.],
       [5., 6.]], dtype=float16)

rank_2_tensor.numpy()

array([[1., 2.],
       [3., 4.],
       [5., 6.]], dtype=float16)

The most important attributes of a tf.Tensor are its shape and dtype: 🏗️📊

• Tensor.shape 📏: Tells you the size of the tensor along each of its axes.
• Tensor.dtype 🔡: Tells you the data type of all the elements in the tensor.

# Creating a constant tensor
tensor = tf.constant([[1, 2], [3, 4]], dtype=tf.float32)
print("Tensor:", tensor)
print("Shape:", tensor.shape)
print("Data Type:", tensor.dtype)
 
# Accessing elements in a tensor
element_0_0 = tensor[0, 0]
print("Element at [0,0]:", element_0_0.numpy())
 
# Performing slicing
slice_of_tensor = tensor[:, 0]
print("Sliced Tensor (all rows, col 0):", slice_of_tensor.numpy())

Tensor: tf.Tensor(
[[1. 2.]
 [3. 4.]], shape=(2, 2), dtype=float32)
Shape: (2, 2)
Data Type: <dtype: 'float32'>
Element at [0,0]: 1.0
Sliced Tensor (all rows, col 0): [1. 3.]

🔹 2.1.2 tf.Variable

tf.Variable is a modifiable tensor typically used to store and update model parameters during training. 🔄📊

Normal tf.Tensor objects are immutable. 🚫
To store model weights (or other mutable state) in TensorFlow, use a tf.Variable. ✅

# Creating a tf.Variable
var = tf.Variable([[5.0, 6.0], [7.0, 8.0]])
print("Initial Value:\n", var.numpy())
 
# Assigning new values
var.assign([[1, 2], [3, 4]])
print("Re-assigned Value:\n", var.numpy())
 
# Incrementing values
var.assign_add([[1, 1], [1, 1]])
print("Incremented Value:\n", var.numpy())

Initial Value:
 [[5. 6.]
 [7. 8.]]
Re-assigned Value:
 [[1. 2.]
 [3. 4.]]
Incremented Value:
 [[2. 3.]
 [4. 5.]]

🔹 2.1.3 tf.RaggedTensor

If you need to handle variable-length data (e.g., sentences of different lengths),
tf.RaggedTensor can be used to store these values efficiently. 📏🗂️

ragged = tf.ragged.constant([[1, 2], [3, 4, 5], [6]])
print("Ragged Tensor:", ragged)
print("Ragged Tensor rows:", [row.numpy() for row in ragged])

Ragged Tensor: <tf.RaggedTensor [[1, 2], [3, 4, 5], [6]]>
Ragged Tensor rows: [array([1, 2], dtype=int32), array([3, 4, 5], dtype=int32), array([6], dtype=int32)]

🔹 2.1.4 tf.SparseTensor

tf.SparseTensor is used to efficiently represent data with a large proportion of zero entries. 🧩⚡

indices = [[0, 3], [2, 4]]
values = [10, 20]
dense_shape = [3, 10]
 
sparse = tf.SparseTensor(indices=indices, values=values, dense_shape=dense_shape)
print("Sparse Tensor:", sparse)
 
dense_from_sparse = tf.sparse.to_dense(sparse)
print("\n Converted to Dense:\n", dense_from_sparse.numpy())

Sparse Tensor: SparseTensor(indices=tf.Tensor(
[[0 3]
 [2 4]], shape=(2, 2), dtype=int64), values=tf.Tensor([10 20], shape=(2,), dtype=int32), dense_shape=tf.Tensor([ 3 10], shape=(2,), dtype=int64))
 
 Converted to Dense:
 [[ 0  0  0 10  0  0  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0 20  0  0  0  0  0]]

🔢 3. Numerical APIs: Math, Linear Algebra, and Random

TensorFlow provides a suite of mathematical and linear algebra operations, as well as random number generation utilities. 🎯📊

These are crucial for model building, as they include everything from simple math operations ➕ to advanced linear algebra routines (like matrix decompositions). 🔢📏

✨ 3.1 Basic Math Operations

# Basic arithmetic
x = tf.constant([2.0, 4.0, 6.0])
y = tf.constant([1.0, 2.0, 3.0])
 
add_result = tf.add(x, y)
sub_result = tf.subtract(x, y)
mul_result = tf.multiply(x, y)
div_result = tf.divide(x, y)
 
print("x + y =", add_result.numpy())
print("x - y =", sub_result.numpy())
print("x * y =", mul_result.numpy())
print("x / y =", div_result.numpy())

x + y = [3. 6. 9.]
x - y = [1. 2. 3.]
x * y = [ 2.  8. 18.]
x / y = [2. 2. 2.]

📐 3.2 Linear Algebra Operations

TensorFlow provides efficient linear algebra operations, essential for deep learning and scientific computing. 🔢🧮

These include:

• Matrix multiplications (tf.matmul) ➗
• Determinants and inverses (tf.linalg.det, tf.linalg.inv) 📏
• Eigenvalues and eigenvectors (tf.linalg.eig) 🔄
• Singular Value Decomposition (SVD) (tf.linalg.svd) 📊

These operations power everything from neural networks to statistical modeling! 🚀💡

# Matrix multiplication
A = tf.constant([[2.0, 3.0], [1.0, 4.0]])
B = tf.constant([[1.0, 2.0], [3.0, 4.0]])
matmul_result = tf.linalg.matmul(A, B)
print("Matrix Multiplication (A @ B):\n", matmul_result.numpy())

Matrix Multiplication (A @ B):
 [[11. 16.]
 [13. 18.]]

🎲 3.3 Random Number Generation

TensorFlow provides random number generation (RNG) utilities for initializing weights, augmenting data, and creating stochastic models. 🔢🎯

Some key functions include:

• tf.random.normal 📊 – Generates numbers from a normal distribution.
• tf.random.uniform 📏 – Generates numbers from a uniform distribution.
• tf.random.shuffle 🔀 – Shuffles the order of elements in a tensor.
• tf.random.set_seed 🔒 – Sets a seed for reproducibility.

Random number generation is essential for machine learning experiments and ensuring reproducibility! 🚀

# Normal distribution
normal_dist = tf.random.normal([3, 3], mean=0.0, stddev=1.0)
print("Random Normal:", normal_dist.numpy())
 
# Uniform distribution
uniform_dist = tf.random.uniform([2, 2], minval=0, maxval=10)
print("Random Uniform:", uniform_dist.numpy())
 
# Setting a seed for reproducibility
tf.random.set_seed(42)
seeded_normal = tf.random.normal([2, 2])
print("Seeded Random Normal:", seeded_normal.numpy())

📉 3.4 tf.reduce_mean

tf.reduce_mean computes the average (arithmetic mean) of elements across specified dimensions (axes) of a tensor. 📊🔢

• If no axis is provided, it calculates the mean of all elements in the tensor. 🔄⚡

# Suppose we have a 2D tensor
x = tf.constant([[1, 2, 3],
                 [4, 5, 6]], dtype=tf.float32)
 
# 1) Mean of all elements
mean_all = tf.reduce_mean(x)
print("mean_all:", mean_all.numpy())
# Output: 3.5  (average of [1,2,3,4,5,6])
 
# 2) Mean along axis=0 (columns)
mean_cols = tf.reduce_mean(x, axis=0)
print("mean_cols:", mean_cols.numpy())
# Output: [2.5 3.5 4.5]  (col-wise means)
 
# 3) Mean along axis=1 (rows)
mean_rows = tf.reduce_mean(x, axis=1)
print("mean_rows:", mean_rows.numpy())
# Output: [2. 5.]  (row-wise means)

mean_all: 3.5
mean_cols: [2.5 3.5 4.5]
mean_rows: [2. 5.]

🔗 4. Computation Graphs in TensorFlow 2.x

TensorFlow 2.x executes eagerly by default. ⚡
This means that operations run immediately in Python rather than first building a static graph.

However, for optimized performance, you can create computation graphs via tf.function(). 📈
This transforms your Python functions into static graphs for faster execution. 🚀

⚡ 4.1 Eager Execution

• By default, each operation executes immediately. 🏃‍♂️
• Easy for debugging and interactive exploration. 🛠️

🔄 4.2 tf.function()

• Decorate a Python function with @tf.function to trace it and create a graph. 🔍
• Significantly improves performance, especially for repeated function calls. ⚡📊

🔥 Without @tf.function

• TensorFlow runs eagerly, executing one operation at a time (like normal Python). 🐍
• Great for debugging but may be slower for repetitive tasks. 🛠️🐢

🚀 With @tf.function

• TensorFlow traces your Python function and builds a reusable computational graph (aka “graph execution”). 📈
• Runs faster due to optimizations but introduces some constraints (e.g., control flow must be TensorFlow-compatible). ⚠️

🏗️ Example: Creating a Computational Graph

Below is an example of a computational graph in TensorFlow:

@tf.function  # Convert to a computation graph
def computation_graph(x1, x2):
    add_op = x1 + x2
    mul_op = x1 * x2
    sub_op = add_op - mul_op
    print('Tracing.\n')
    return sub_op
 
# Inputs
x1 = tf.constant(7, dtype=tf.float32)
x2 = tf.constant(3, dtype=tf.float32)
 
computation_graph(x1, x2)

Tracing.
 
<tf.Tensor: shape=(), dtype=int32, numpy=-11>

On subsequent calls, TensorFlow only executes the optimized graph,
skipping any non-TensorFlow steps for faster execution. ⚡📈🚀

⚠️ Note: The computation_graph does not print tracing because print() is a Python function,
not a TensorFlow function. 🐍❌🔄

x1 = tf.constant(2, dtype=tf.float32)
x2 = tf.constant(1, dtype=tf.float32)
computation_graph(x1, x2)

Tracing.
 
<tf.Tensor: shape=(), dtype=int32, numpy=1>

🎯 5. Automatic Differentiation using tf.GradientTape()

TensorFlow uses the reverse-mode autodiff algorithm to compute gradients. 🔄📈
You can record operations on tensors inside a tf.GradientTape() context,
and then request gradients with respect to any trainable variables or tensors. 🛠️

🔢 5.1 Single Variable Example

x = tf.Variable(2.0)
with tf.GradientTape() as tape:
    y = x ** 3  # y = x^3
 
dy_dx = tape.gradient(y, x)
print("If y = x^3 and x = 2.0, dy/dx should be 3*x^2 = 12.0.")
print("Computed Gradient:", dy_dx.numpy())

If y = x^3 and x = 2.0, dy/dx should be 3*x^2 = 12.0.
Computed Gradient: 12.0

🔢 5.2 Multiple Variables Example

In neural networks, you'd typically track multiple trainable variables. 🤖🧠
Here's a simplified demonstration with two variables. 🎯📊

w = tf.Variable(1.0)
b = tf.Variable(2.0)
x_vals = tf.constant([1.0, 2.0, 3.0])
 
with tf.GradientTape() as tape:
    # A simple linear function: y = w * x + b
    y_vals = w * x_vals + b
    loss = tf.reduce_mean((y_vals - tf.constant([2.0, 4.0, 6.0]))**2)
 
gradients = tape.gradient(loss, [w, b])
print("Loss:", loss.numpy())
print("dLoss/dw:", gradients[0].numpy())
print("dLoss/db:", gradients[1].numpy())

Loss: 0.6666667
dLoss/dw: -1.3333333
dLoss/db: 0.0

🎯 Conclusion

In this notebook, you learned about:

1. ✅ Importing TensorFlow and checking versions/devices.
2. 🏗️ The TensorFlow Core API, including fundamental data structures (tf.Tensor, tf.Variable, etc.).
3. 🔢 Numerical APIs (math, linear algebra, random operations).
4. ⚡ Building Computation Graphs in TF 2.x using tf.function.
5. 🔄 Automatic Differentiation with tf.GradientTape().

This foundation will help you build more complex TensorFlow models
and understand performance tuning, debugging, and deployment strategies. 🚀📊