Air Pollution Prediction Using Machine Learning

Air pollution prediction is a crucial field where machine learning techniques play a vital role. Its objective is to understand and tackle the harmful effects of air pollutants on human health and the environment. By utilizing a range of machine learning algorithms, including regression, decision trees, and neural networks, we can analyze historical data on air quality alongside meteorological and geographical factors. This analysis enables us to develop models that can forecast pollution levels and detect patterns. By taking proactive measures such as adjusting controls on emissions, implementing strategies to reduce pollution, and providing timely alerts to the public, we can work towards creating cleaner and healthier environments. Embracing the potential of machine learning in air pollution prediction empowers us to make informed decisions and protect the well-being of our communities.

Now we will try to implement it into code.

Code:

Importing Libraries

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings("ignore")

from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn import metrics
from sklearn.metrics import mean_absolute_error,mean_squared_error,r2_score
from sklearn.metrics import accuracy_score,confusion_matrix

Reading the Dataset

Now we will read the dataset.

df=pd.read_csv('../input/india-air-quality-data/data.csv',encoding='unicode_escape')

Exploratory Data Analysis

Exploratory Data Analysis (EDA) is akin to embarking on an adventure to unravel the secrets of a treasure map. It involves a meticulous examination of our data, searching for intriguing insights, and uncovering hidden gems. Through various techniques and methods, we explore patterns, anomalies, and peculiarities that catch our attention. We also generate summaries and statistics that paint a vivid picture of the data, enriching our understanding of its nuances. EDA is a crucial undertaking as it unveils concealed information and allows us to grasp the profound knowledge the data holds.

Output:

df.shape
# As we can see that there are 4,35,742 rows and 13 columns in the dataset

Output:

df.info()
# Checking the overall information on the dataset.

Output:

df.isnull().sum()
# There are a lot of missing values present in the dataset

Output:

df.describe()
# Checking the descriptive stats of the numeric values present in the data like mean, standard deviation, min values, and max values present in the data

Output:

df.nunique()
# These are all the unique values present in the dataframe

Output:

df.columns
# These are all the columns present in the dataset.

Output:

Data Visualization

Data visualization is like turning numbers and information into pictures or graphs that are easy to understand. It's a way of presenting data in a visual format so that patterns, trends, and relationships can be easily seen. Instead of staring at a bunch of numbers, data visualization helps us see the bigger picture and make sense of the information.

Output:

df['state'].value_counts()
# Viewing the count of values present in the state column

Output:

plt.figure(figsize=(15, 6))
plt.xticks(rotation=90)
df.type.hist()
plt.xlabel('Type')
plt.ylabel('Frequencies')
plt.plot()
# The visualization shows us the count of Types present in the dataset.

Output:

df['type'].value_counts()
# Viewing the count of values present in the type column

Output:

plt.figure(figsize=(15, 6))
plt.xticks(rotation=90)
df.type.hist()
plt.xlabel('Type')
plt.ylabel('Frequencies')
plt.plot()
# The visualization shows us the count of Types present in the dataset.

Output:

df['agency'].value_counts()
# Viewing the counts of values present in the agency column

Output:

plt.figure(figsize=(15, 6))
plt.xticks(rotation=90)
df.agency.hist()
plt.xlabel('Agency')
plt.ylabel('Frequencies')
plt.plot()
# The visualization shows us the count of Agency present in the dataset.

Output:

plt.figure(figsize=(30, 10))
plt.xticks(rotation=90)
sns.barplot(x='state',y='so2',data=df);
# This visualization shows the name of the state having higher so2 levels in the air, which is Uttaranchal, followed by Uttarakhand

Output:

plt.rcParams['figure.figsize']=(30,10)
df[['so2','state']].groupby(["state"]).mean().sort_values(by='so2').plot.bar(color='purple')
plt.show()
# We can also use the groupby function to sort values in an ascending order based on the x-axis, y-axis and its keys
# Below, we get a clear picture of the states in increasing order based on their so2 levels.

Output:

plt.figure(figsize=(30, 10))
plt.xticks(rotation=90)
sns.barplot(x='state',y='no2',data=df);
# West Bengal has a higher no2 level compared to other states 

Output:

df[['no2','state']].groupby(["state"]).mean().sort_values(by='no2').plot.bar(color='purple')
plt.show()
# We can also use the groupby function to sort values in an ascending order based on the x-axis, y-axis and its keys
# Below, we get a clear picture of the states in increasing order based on their no2 levels.

Output:

plt.figure(figsize=(30, 10))
plt.xticks(rotation=90)
sns.barplot(x='state',y='rspm',data=df);
# Delhi has a higher rspm level compared to other states 

Output:

plt.figure(figsize=(30, 10))
plt.xticks(rotation=90)
sns.barplot(x='state',y='spm',data=df);
# Delhi has a higher spm level compared to other states 

Output:

plt.figure(figsize=(30, 10))
plt.xticks(rotation=90)
sns.barplot(x='state',y='pm2_5',data=df);
# Delhi has a higher pm2_5 level compared to other states 

Output:

Checking all null values and treating those null values

nullvalues = df.isnull().sum().sort_values(ascending=False)
# Checking all null values
nullvalues
# higher null values present in pm2_5 followed by spm

Output:

null_values_percentage = (df.isnull().sum()/df.isnull().count()*100).sort_values(ascending=False)
#count(returns Non-NAN value)

missing_data_with_percentage = pd.concat([nullvalues, null_values_percentage], axis=1, keys=['Total', 'Percent'])
# Concatenating total null values and their percentage of missing values for further imputation or column deletion

missing_data_with_percentage
# As you can see below, these are the percentages of null values present in the dataset

Output:

df.drop(['agency'],axis=1,inplace=True)
df.drop(['stn_code'],axis=1,inplace=True)
df.drop(['date'],axis=1,inplace=True)
df.drop(['sampling_date'],axis=1,inplace=True)
df.drop(['location_monitoring_station'],axis=1,inplace=True)
# Dropping unnecessary columns

df.isnull().sum()
# Now checking the null values

Output:

df['location']=df['location'].fillna(df['location'].mode()[0])
df['type']=df['type'].fillna(df['type'].mode()[0])
# Null value Imputation for categorical data

df.fillna(0, inplace=True)
# Null values are replaced with zeros for the numerical data

df.isnull().sum()
# Now we have successfully imputed null values which were present in the dataset

Output:

df
# The following features are important for our machine learning models.

Output:

Functions

We will be defining functions as they will be usable for later purposes also.

def cal_SOi(so2):
    """  calculating the individual pollutant index for so2(sulphur dioxide)
"""
    si=0
    if (so2<=40):
     si= so2*(50/40)
    elif (so2>40 and so2<=80):
     si= 50+(so2-40)*(50/40)
    elif (so2>80 and so2<=380):
     si= 100+(so2-80)*(100/300)
    elif (so2>380 and so2<=800):
     si= 200+(so2-380)*(100/420)
    elif (so2>800 and so2<=1600):
     si= 300+(so2-800)*(100/800)
    elif (so2>1600):
     si= 400+(so2-1600)*(100/800)
    return si
df['SOi']=df['so2'].apply(cal_SOi)
data= df[['so2','SOi']]
data.head()

Output:

def cal_Noi(no2):
    """calculating the individual pollutant index for no2(nitrogen dioxide)"""
    ni=0
    if(no2<=40):
     ni= no2*50/40
    elif(no2>40 and no2<=80):
     ni= 50+(no2-40)*(50/40)
    elif(no2>80 and no2<=180):
     ni= 100+(no2-80)*(100/100)
    elif(no2>180 and no2<=280):
     ni= 200+(no2-180)*(100/100)
    elif(no2>280 and no2<=400):
     ni= 300+(no2-280)*(100/120)
    else:
     ni= 400+(no2-400)*(100/120)
    return ni
df['Noi']=df['no2'].apply(cal_Noi)
data= df[['no2','Noi']]
data.head()

Output:

def cal_RSPMI(rspm):
    """calculating the individual pollutant index for rspm(respirable suspended particualte matter concentration)
"""
    rpi=0
    if(rpi<=30):
     rpi=rpi*50/30
    elif(rpi>30 and rpi<=60):
     rpi=50+(rpi-30)*50/30
    elif(rpi>60 and rpi<=90):
     rpi=100+(rpi-60)*100/30
    elif(rpi>90 and rpi<=120):
     rpi=200+(rpi-90)*100/30
    elif(rpi>120 and rpi<=250):
     rpi=300+(rpi-120)*(100/130)
    else:
     rpi=400+(rpi-250)*(100/130)
    return rpi
df['Rpi']=df['rspm'].apply(cal_RSPMI)
data= df[['rspm','Rpi']]
data.head()

Output:

def cal_SPMi(spm):
    """calculating the individual pollutant index for spm(suspended particulate matter)"""
    spi=0
    if(spm<=50):
     spi=spm*50/50
    elif(spm>50 and spm<=100):
     spi=50+(spm-50)*(50/50)
    elif(spm>100 and spm<=250):
     spi= 100+(spm-100)*(100/150)
    elif(spm>250 and spm<=350):
     spi=200+(spm-250)*(100/100)
    elif(spm>350 and spm<=430):
     spi=300+(spm-350)*(100/80)
    else:
     spi=400+(spm-430)*(100/430)
    return spi
   
df['SPMi']=df['spm'].apply(cal_SPMi)
data= df[['spm','SPMi']]
data.head()

Output:

def cal_aqi(si,ni,rspmi,spmi):
    """Caluclating the Air Quality Index."""
    aqi=0
    if(si>ni and si>rspmi and si>spmi):
     aqi=si
    if(ni>si and ni>rspmi and ni>spmi):
     aqi=ni
    if(rspmi>si and rspmi>ni and rspmi>spmi):
     aqi=rspmi
    if(spmi>si and spmi>ni and spmi>rspmi):
     aqi=spmi
    return aqi

df['AQI']=df.apply(lambda x:cal_aqi(x['SOi'],x['Noi'],x['Rpi'],x['SPMi']),axis=1)
data= df[['state','SOi','Noi','Rpi','SPMi','AQI']]
data.head()

Output:

def AQI_Range(x):
    """Using threshold values to classify a particular values as good, moderate, poor, unhealthy, very unhealthy and Hazardous"""
    if x<=50:
        return "Good"
    elif x>50 and x<=100:
        return "Moderate"
    elif x>100 and x<=200:
        return "Poor"
    elif x>200 and x<=300:
        return "Unhealthy"
    elif x>300 and x<=400:
        return "Very unhealthy."
    elif x>400:
        return "Hazardous"

df['AQI_Range'] = df['AQI'] .apply(AQI_Range)
df.head()

Output:

df['AQI_Range'].value_counts()
# These are the counts of values present in the AQI_Range column.

Output:

Splitting Dataset into Training and Testing Set

X2 = df[['SOi','Noi','Rpi','SPMi']]
Y2 = df['AQI_Range']
# Splitting the data into independent and dependent columns for classification 

X_train2, X_test2, Y_train2, Y_test2 = train_test_split(X2, Y2, test_size=0.33, random_state=70)
# Splitting the data into training and testing data 

Models

Here we will employ various machine learning models to predict air pollution.

from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier

1. Logistic Regression

#fit the model on train data 
log_reg = LogisticRegression().fit(X_train2, Y_train2)

#predict on train 
train_preds2 = log_reg.predict(X_train2)
#accuracy on train
print("Model accuracy on train is: ", accuracy_score(Y_train2, train_preds2))

#predict on test
test_preds2 = log_reg.predict(X_test2)
#accuracy on test
print("Model accuracy on test is: ", accuracy_score(Y_test2, test_preds2))
print('-'*50)

# Kappa Score.
print('KappaScore is: ', metrics.cohen_kappa_score(Y_test2,test_preds2))

Output:

2. Decision Tree Classifier

#fit the model on train data 
DT2 = DecisionTreeClassifier().fit(X_train2,Y_train2)

#predict on train 
train_preds3 = DT2.predict(X_train2)
#accuracy on train
print("Model accuracy on train is: ", accuracy_score(Y_train2, train_preds3))

#predict on test
test_preds3 = DT2.predict(X_test2)
#accuracy on test
print("Model accuracy on test is: ", accuracy_score(Y_test2, test_preds3))
print('-'*50)

# Kappa Score
print('KappaScore is: ', metrics.cohen_kappa_score(Y_test2,test_preds3))

Output:

3. Random Forest Classifier

#fit the model on train data 
RF=RandomForestClassifier().fit(X_train2,Y_train2)
#predict on train 
train_preds4 = RF.predict(X_train2)
#accuracy on train
print("Model accuracy on train is: ", accuracy_score(Y_train2, train_preds4))

#predict on test
test_preds4 = RF.predict(X_test2)
#accuracy on test
print("Model accuracy on test is: ", accuracy_score(Y_test2, test_preds4))
print('-'*50)

# Kappa Score
print('KappaScore is: ', metrics.cohen_kappa_score(Y_test2,test_preds4))

Output:

4. KNN

#fit the model on train data 
KNN = KNeighborsClassifier().fit(X_train2,Y_train2)
#predict on train 
train_preds5 = KNN.predict(X_train2)
#accuracy on train
print("Model accuracy on train is: ", accuracy_score(Y_train2, train_preds5))

#predict on test
test_preds5 = KNN.predict(X_test2)
#accuracy on test
print("Model accuracy on test is: ", accuracy_score(Y_test2, test_preds5))
print('-'*50)

# Kappa Score
print('KappaScore is: ', metrics.cohen_kappa_score(Y_test2,test_preds5))

Output:

Model Comparison

Model comparison helps to choose the best options among the models.

models = {"Logistic Regression": LogisticRegression(),
          "DecisionTreeClassifier": DecisionTreeClassifier(),
         "KNN": KNeighborsClassifier(),
         "Random Forest": RandomForestClassifier()}

def fit_and_score(models, X_train, X_test, y_train, y_test):
    """
    Fits and evaluates given machine learning models.
    models : a dict of different Scikit-Learn machine learning models
    X_train : training data (no labels)
    X_test : testing data (no labels)
    y_train : training labels
    y_test : test labels
    """
    #set the random seed
    np.random.seed(42)
    #dictionary to keep model scores
    model_scores = {}
    #loop thru models
    for name, model in models.items():
        #fit model
        model.fit(X_train, y_train)
        #evaluate model and append score
        model_scores[name]=model.score(X_test, y_test)
    return model_scores

model_scores = fit_and_score(models=models, X_train=X_train2, X_test=X_test2, y_train=Y_train2, y_test=Y_test2)
model_scores

Output:

model_compare = pd.DataFrame(model_scores, index=["accuracy"])
model_compare.T.plot.bar(legend=False)

Output:

Random Forest and Decision Tree have the accuracy for predicting air pollution as multiple factors are involved, and they are quite effective when there is a wide variety of factors involved.

Conclusion

Anticipating and forecasting air pollution using machine learning techniques represents a burgeoning area of study that provides a priceless understanding of the intricate workings of pollution and its repercussions on human well-being and the environment. By merging the realms of data science and environmental research, we can forge impactful strategies to alleviate pollution, ameliorate air quality, and establish healthier habitats for forthcoming generations. As machine learning and data collection progress, we hold the key to charting a course toward an unpolluted and sustainable future.

Next TopicCustomer Churn Prediction Using Machine Learning

← prev next →