Lab 1: Introduction to Python for Data Science
10 min read
Lesson slides
1 / 15
Topics Covered
๐น 1. Jupyter Notebooks
- Introduction to Jupyter Notebook
- Features and benefits
- Running Python code in Jupyter
๐น 2. Google Colab Notebooks
- Overview of Google Colab
- Advantages over local Jupyter Notebook
- Running Python code on the cloud
๐น 3. NumPy Library (Numerical Python)
- Creating and manipulating arrays
- Vectorized operations for efficiency
- Indexing, slicing, and reshaping arrays
๐น 4. Pandas Library (Data Manipulation)
- Understanding Series and DataFrames
- Importing and exporting datasets
- Data cleaning and transformation
- Applying functions and aggregations
๐น 5. Matplotlib Library (Data Visualization)
- Creating basic plots (line, bar, scatter)
- Customizing graphs (labels, legends, colors)
- Saving figures for reports
Numpy
import numpy as npCreating Arrays
my_list = [0,1,2,3,4]arr = np.array(my_list)arrarray([0, 1, 2, 3, 4])np.arange(0,10)array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])np.arange(0,10,3)array([0, 3, 6, 9])np.zeros((5,5))array([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]])np.ones((2,4))array([[1., 1., 1., 1.],
[1., 1., 1., 1.]])np.random.randint(0,10)7np.random.randint(0,100,(3,3))array([[27, 73, 80],
[53, 26, 91],
[91, 54, 68]])np.linspace(0,10,6)array([ 0., 2., 4., 6., 8., 10.])np.linspace(0,10,101)array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ,
1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2. , 2.1,
2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 3. , 3.1, 3.2,
3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 4. , 4.1, 4.2, 4.3,
4.4, 4.5, 4.6, 4.7, 4.8, 4.9, 5. , 5.1, 5.2, 5.3, 5.4,
5.5, 5.6, 5.7, 5.8, 5.9, 6. , 6.1, 6.2, 6.3, 6.4, 6.5,
6.6, 6.7, 6.8, 6.9, 7. , 7.1, 7.2, 7.3, 7.4, 7.5, 7.6,
7.7, 7.8, 7.9, 8. , 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7,
8.8, 8.9, 9. , 9.1, 9.2, 9.3, 9.4, 9.5, 9.6, 9.7, 9.8,
9.9, 10. ])Operations
np.random.seed(101)
arr2 = np.random.randint(0,100,10)arr2array([95, 11, 81, 70, 63, 87, 75, 9, 77, 40])print( np.random.randint(0,100,10))[ 4 63 40 60 92 64 5 12 93 40]arr2.max()95arr2.min()9arr2.mean()60.8arr2.argmin()7arr2.argmax()0arr2.reshape(2,5)array([[95, 11, 81, 70, 63],
[87, 75, 9, 77, 40]])arr3=np.random.randint(0,100,9)arr3.reshape(3,3)array([[49, 83, 8],
[29, 59, 34],
[44, 72, 19]])Indexing
mat = np.arange(0,100).reshape(10,10)matarray([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
[40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
[60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
[70, 71, 72, 73, 74, 75, 76, 77, 78, 79],
[80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
[90, 91, 92, 93, 94, 95, 96, 97, 98, 99]])row = 0
col = 1
mat[row,col]1# With Slices
mat[:,0]array([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90])mat[0,:]array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])mat[0:3,0:3]array([[ 0, 1, 2],
[10, 11, 12],
[20, 21, 22]])mat[0:3,-3:-1]array([[ 7, 8],
[17, 18],
[27, 28]])Pandas
Pandas library
Library for computation with tabular data
import pandas as pdstep_data = [3620, 7891, 9761, 3907, 4338, 5373]
print(step_data)
step_counts = pd.Series(step_data, name='steps')
print(step_counts)[3620, 7891, 9761, 3907, 4338, 5373]
0 3620
1 7891
2 9761
3 3907
4 4338
5 5373
Name: steps, dtype: int64step_counts.index = pd.date_range('20240329',periods=6)
print(step_counts)2024-03-29 3620
2024-03-30 7891
2024-03-31 9761
2024-04-01 3907
2024-04-02 4338
2024-04-03 5373
Freq: D, Name: steps, dtype: int64# Just like a dictionary
print(step_counts['2024-04-01'])3907# Or by index position--like an array
print(step_counts[3])3907# View the data type
print(step_counts.dtypes)int64cycling_data = [10.7, 0, None, 2.4, 15.3,10.9, 0, None]
# Create a tuple of data
joined_data = list(zip(step_data, cycling_data))
print(joined_data)
# The dataframe
activity_df = pd.DataFrame(joined_data)
print(activity_df)[(3620, 10.7), (7891, 0), (9761, None), (3907, 2.4), (4338, 15.3), (5373, 10.9)]
0 1
0 3620 10.7
1 7891 0.0
2 9761 NaN
3 3907 2.4
4 4338 15.3
5 5373 10.9activity_df = pd.DataFrame(joined_data,
index=pd.date_range('20240329',
periods=6),
columns=['Walking','Cycling'])
print(activity_df) Walking Cycling
2024-03-29 3620 10.7
2024-03-30 7891 0.0
2024-03-31 9761 NaN
2024-04-01 3907 2.4
2024-04-02 4338 15.3
2024-04-03 5373 10.9print(activity_df['Walking'])2024-03-29 3620
2024-03-30 7891
2024-03-31 9761
2024-04-01 3907
2024-04-02 4338
2024-04-03 5373
Freq: D, Name: Walking, dtype: int64print(activity_df.loc['2024-04-01'])Walking 3907.0
Cycling 2.4
Name: 2024-04-01 00:00:00, dtype: float64print(activity_df.loc['2024-04-01':'2024-04-03']) Walking Cycling
2024-04-01 3907 2.4
2024-04-02 4338 15.3
2024-04-03 5373 10.9print(activity_df.loc['2024-04-01':'2024-04-03',['Walking']]) Walking
2024-04-01 3907
2024-04-02 4338
2024-04-03 5373print(activity_df.iloc[0,:])
print(activity_df.iloc[:2,0])Walking 3620.0
Cycling 10.7
Name: 2024-03-29 00:00:00, dtype: float64
2024-03-29 3620
2024-03-30 7891
Freq: D, Name: Walking, dtype: int64data = pd.read_csv('.\Iris_Data.csv')data.head() sepal_length sepal_width petal_length petal_width species
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosaprint(data.iloc[:5]) sepal_length sepal_width petal_length petal_width species
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa#access dataset commands or from file mount drive
#from google.colab import drive
#drive.mount('/content/drive')#data = pd.read_csv('/content/drive/MyDrive/Iris_Data.csv')data.head() sepal_length sepal_width petal_length petal_width species
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosa#num of rows
data.shape[1]5data.shape(150, 5)print(data['sepal_width'])
#print(data.sepal_width)0 3.5
1 3.0
2 3.2
3 3.1
4 3.6
...
145 3.0
146 2.5
147 3.0
148 3.4
149 3.0
Name: sepal_width, Length: 150, dtype: float64Loc and iloc are two functions in Pandas that are used to slice a data set in a Pandas DataFrame. The function .loc is typically used for label indexing and can access multiple columns, while .iloc is used for integer indexing.
The loc() function is label based data selecting method which means that we have to pass the name of the row or column which we want to select. This method includes the last element of the range passed in it, unlike iloc(). loc() can accept the boolean data unlike iloc(). Many operations can be performed using the loc() method https://www.geeksforgeeks.org/difference-between-loc-and-iloc-in-pandas-dataframe/
print(data.loc[2],'\n')
print(data.loc[2,['sepal_length']])sepal_length 4.7
sepal_width 3.2
petal_length 1.3
petal_width 0.2
species Iris-setosa
Name: 2, dtype: object
sepal_length 4.7
Name: 2, dtype: objectprint(data.iloc[:5,:]) sepal_length sepal_width petal_length petal_width species
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
4 5.0 3.6 1.4 0.2 Iris-setosaprint(data.iloc[0,:])
print(data.iloc[:,0])sepal_length 5.1
sepal_width 3.5
petal_length 1.4
petal_width 0.2
species Iris-setosa
Name: 0, dtype: object
0 5.1
1 4.9
2 4.7
3 4.6
4 5.0
...
145 6.7
146 6.3
147 6.5
148 6.2
149 5.9
Name: sepal_length, Length: 150, dtype: float64data['species'] = data.species.str.replace('Iris-', '')
print(data.loc[(data.sepal_width>3.5)]) sepal_length sepal_width petal_length petal_width species
4 5.0 3.6 1.4 0.2 setosa
5 5.4 3.9 1.7 0.4 setosa
10 5.4 3.7 1.5 0.2 setosa
14 5.8 4.0 1.2 0.2 setosa
15 5.7 4.4 1.5 0.4 setosa
16 5.4 3.9 1.3 0.4 setosa
18 5.7 3.8 1.7 0.3 setosa
19 5.1 3.8 1.5 0.3 setosa
21 5.1 3.7 1.5 0.4 setosa
22 4.6 3.6 1.0 0.2 setosa
32 5.2 4.1 1.5 0.1 setosa
33 5.5 4.2 1.4 0.2 setosa
44 5.1 3.8 1.9 0.4 setosa
46 5.1 3.8 1.6 0.2 setosa
48 5.3 3.7 1.5 0.2 setosa
109 7.2 3.6 6.1 2.5 virginica
117 7.7 3.8 6.7 2.2 virginica
131 7.9 3.8 6.4 2.0 virginica# Create a new column that is a product
# of both measurements
data['sepal_area'] = data.sepal_length *data.sepal_width
# Print a few rows and columns
print(data.iloc[:5, -3:]) petal_width species sepal_area
0 0.2 setosa 17.85
1 0.2 setosa 14.70
2 0.2 setosa 15.04
3 0.2 setosa 14.26
4 0.2 setosa 18.00# Use the size method with a
# DataFrame to get count For a Series, use the .value_counts method
group_sizes = (data.groupby('species').size())
print(group_sizes)species
setosa 50
versicolor 50
virginica 50
dtype: int64In Machine Learning (and in mathematics) there are often three values that interests us:
- Mean - The average value
- Median - The mid point value
- Mode - The most common value
https://www.w3schools.com/python/python_ml_mean_median_mode.asp
print(data.sepal_width.mean())3.0540000000000003print(data.petal_length.median())4.35print(data.petal_length.mode())0 1.5
Name: petal_length, dtype: float64Standard deviation is a number that describes how spread out the values are. A low standard deviation means that most of the numbers are close to the mean (average) value.
A high standard deviation means that the values are spread out over a wider range Standard Deviation is calculated as the square root of the variance
A variance is the average of the squared differences from the mean. To figure out the variance, calculate the difference between each point within the data set and the mean. Once you figure that out, square and average the results.
Standard error of the mean (SEM) measures how far the sample mean (average) of the data is likely to be from the true population mean. The SEM is always smaller than the SD. SEM is calculated simply by taking the standard deviation and dividing it by the square root of the sample size
print(data.petal_length.std())
print(data.petal_length.var())
print(data.petal_length.sem())1.7644204199522626
3.113179418344519
0.1440643240210085Matplotlib
import matplotlib.pyplot as pltax = plt.axes()
ax.hist(data.petal_length, bins=25)
ax.set(xlabel='Petal Length (cm)',
ylabel='Frequency',
title='Distribution of Petal Lengths')[Text(0.5, 0, 'Petal Length (cm)'),
Text(0, 0.5, 'Frequency'),
Text(0.5, 1.0, 'Distribution of Petal Lengths')]plt.plot(data.sepal_length,
data.sepal_width,
marker='o')
# plt.scatter(data.sepal_length,
# data.sepal_width,
# marker='o')
#plt.xlim(3,9)
#plt.ylim(1,20)
plt.title("Distribution of Petal Lengths")
plt.xlabel("Petal Length (cm)")
plt.ylabel("sepal_width")Text(0, 0.5, 'sepal_width')plt.hist(data.sepal_length, bins=25)
plt.title("Histogram")
plt.xlabel("Petal Length (cm)")
plt.ylabel("Frequency")Text(0, 0.5, 'Frequency')