1. Import Libraries and Dataset

1.1 Package imports

import pandas as pd 
import numpy as np                    
import seaborn as sns                 
import matplotlib.pyplot as plt       
%matplotlib inline 
import warnings
from scipy.stats import chi2_contingency
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.tree import DecisionTreeClassifier,export_graphviz
from sklearn.metrics import confusion_matrix,accuracy_score
from IPython.display import Image 
from pydot import graph_from_dot_data
from six import StringIO  
from sklearn.tree import export_graphviz
from imblearn.over_sampling import SMOTE
from imblearn.metrics import specificity_score
from collections import Counter
from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay, accuracy_score, precision_score, recall_score, f1_score
from sklearn.linear_model import LogisticRegression 

1.2 Read data from data set

pd.set_option('future.no_silent_downcasting', True)
data = pd.read_csv("./diabetes_risk_prediction_dataset.csv")

1.3 Confirm the imports

data.shape

(520, 17)

1.4 Identify the features

data.columns

Index(['Age', 'Gender', 'Polyuria', 'Polydipsia', 'sudden weight loss',
       'weakness', 'Polyphagia', 'Genital thrush', 'visual blurring',
       'Itching', 'Irritability', 'delayed healing', 'partial paresis',
       'muscle stiffness', 'Alopecia', 'Obesity', 'class'],
      dtype='object')

2. Data visualization and exploration

2.1 Sanity check

data.head(2)

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Itching	Irritability	delayed healing	partial paresis	muscle stiffness	Alopecia	Obesity	class
0	40	Male	No	Yes	No	Yes	No	No	No	Yes	No	Yes	No	Yes	Yes	Yes	Positive
1	58	Male	No	No	No	Yes	No	No	Yes	No	No	No	Yes	No	Yes	No	Positive

2.2 Check for unique data values

# Get the count of unique values in each column
unique_counts = data.nunique()

# Get the actual unique values for each column
unique_values = data.apply(lambda x: x.unique())

# Print the results
print("Unique value counts per feature:")
print(unique_counts)

print("\nUnique values per feature:")
for column, values in unique_values.items():
    print(f"{column}: {values}")

Unique value counts per feature:
Age                   51
Gender                 2
Polyuria               2
Polydipsia             2
sudden weight loss     2
weakness               2
Polyphagia             2
Genital thrush         2
visual blurring        2
Itching                2
Irritability           2
delayed healing        2
partial paresis        2
muscle stiffness       2
Alopecia               2
Obesity                2
class                  2
dtype: int64

Unique values per feature:
Age: [40 58 41 45 60 55 57 66 67 70 44 38 35 61 54 43 62 39 48 32 42 52 53 37
 49 63 30 50 46 36 51 59 65 25 47 28 68 56 31 85 90 72 69 79 34 16 33 64
 27 29 26]
Gender: ['Male' 'Female']
Polyuria: ['No' 'Yes']
Polydipsia: ['Yes' 'No']
sudden weight loss: ['No' 'Yes']
weakness: ['Yes' 'No']
Polyphagia: ['No' 'Yes']
Genital thrush: ['No' 'Yes']
visual blurring: ['No' 'Yes']
Itching: ['Yes' 'No']
Irritability: ['No' 'Yes']
delayed healing: ['Yes' 'No']
partial paresis: ['No' 'Yes']
muscle stiffness: ['Yes' 'No']
Alopecia: ['Yes' 'No']
Obesity: ['Yes' 'No']
class: ['Positive' 'Negative']

2.3 Data visualization to get insights about data

2.3.1 Histogram for numeric feature (Age) to visualize its distribution

plt.figure() 
sns.displot(data['Age']); 

<Figure size 640x480 with 0 Axes>

Observation: - Age variable is normally distributed

2.3.2 Bar cart for categorical feature (gender) to visualize frequency of counts of gender.

data['Gender'].value_counts().plot.barh(figsize=(10,5),  title= 'Gender Frequency')

<Axes: title={'center': 'Gender Frequency'}, ylabel='Gender'>

2.3.3 Class Distribution of target varaible.

class_counts = data['class'].value_counts()
# Plot the class distribution
plt.figure(figsize=(8, 6))
class_counts.plot(kind='bar')
plt.title('Class Distribution of Target Variable', fontsize=16)
plt.xlabel('Class')
plt.ylabel('Count')
plt.xticks(rotation=0)
plt.show()

Observation: - 38% of records belong to the minority class(Negative) i.e dataset imbalance is negligible.

2.4 Corelational Analysis

data.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 520 entries, 0 to 519
Data columns (total 17 columns):
 #   Column              Non-Null Count  Dtype 
---  ------              --------------  ----- 
 0   Age                 520 non-null    int64 
 1   Gender              520 non-null    object
 2   Polyuria            520 non-null    object
 3   Polydipsia          520 non-null    object
 4   sudden weight loss  520 non-null    object
 5   weakness            520 non-null    object
 6   Polyphagia          520 non-null    object
 7   Genital thrush      520 non-null    object
 8   visual blurring     520 non-null    object
 9   Itching             520 non-null    object
 10  Irritability        520 non-null    object
 11  delayed healing     520 non-null    object
 12  partial paresis     520 non-null    object
 13  muscle stiffness    520 non-null    object
 14  Alopecia            520 non-null    object
 15  Obesity             520 non-null    object
 16  class               520 non-null    object
dtypes: int64(1), object(16)
memory usage: 69.2+ KB

2.4.1 Data Conversion

Transformed Binary attributes mentioned below from dtype object to int64 i.e 0/1. Symmetric Binary: Gender({Male: 1, Female:0}), Asymmetric Binary: Symptoms({Yes:1, No:0})

Target variable encoding: Class({Positive:1, Negative:0})

For decision tree and most other classification models, processing numerical data is more efficient than processing categorical data like Yes/No or Positive/Negative.

for feature in data.select_dtypes("object").columns.to_list():
    col = pd.Categorical(data[feature])
    data[feature] = col.codes
data

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Itching	Irritability	delayed healing	partial paresis	muscle stiffness	Alopecia	Obesity	class
0	40	1	0	1	0	1	0	0	0	1	0	1	0	1	1	1	1
1	58	1	0	0	0	1	0	0	1	0	0	0	1	0	1	0	1
2	41	1	1	0	0	1	1	0	0	1	0	1	0	1	1	0	1
3	45	1	0	0	1	1	1	1	0	1	0	1	0	0	0	0	1
4	60	1	1	1	1	1	1	0	1	1	1	1	1	1	1	1	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
515	39	0	1	1	1	0	1	0	0	1	0	1	1	0	0	0	1
516	48	0	1	1	1	1	1	0	0	1	1	1	1	0	0	0	1
517	58	0	1	1	1	1	1	0	1	0	0	0	1	1	0	1	1
518	32	0	0	0	0	1	0	0	1	1	0	1	0	0	1	0	0
519	42	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

520 rows × 17 columns

2.4.2 Corealtional Analysis

h_labels = [x for x in 
            list(data.select_dtypes(include=['number', 'bool']).columns.values)]

fig, ax = plt.subplots(figsize=(20,12))
_ = sns.heatmap(data.corr(), annot=True, xticklabels=h_labels, yticklabels=h_labels, cmap=sns.cubehelix_palette(as_cmap=True), ax=ax)

correlations = data.corr()['class'].sort_values(ascending=False)

# Print the correlations
print("Correlation of each feature with the target column:")
print(correlations)

Correlation of each feature with the target column:
class                 1.000000
Polyuria              0.665922
Polydipsia            0.648734
sudden weight loss    0.436568
partial paresis       0.432288
Polyphagia            0.342504
Irritability          0.299467
visual blurring       0.251300
weakness              0.243275
muscle stiffness      0.122474
Genital thrush        0.110288
Age                   0.108679
Obesity               0.072173
delayed healing       0.046980
Itching              -0.013384
Alopecia             -0.267512
Gender               -0.449233
Name: class, dtype: float64

Corelational analysis will effect feature selection, more a feature is positively or negatively co-related to the target variable, it is more helpful in predicting the target variable. The variables should be co-related to the target variable and uncorellated among themselves, so if there is high co-relation between any 2 variables, the model only needs one as second will not add any additional value. For given field i.e. healthcare, requiring high confidence, a threshold of absolute value greater than 0 will be considered, rest will be eliminated to reduce curse of dimentionality.

We will dismiss Obesity, Delayed healing, Itching as their absolute value is less than 0.0.

Features that are weakly orleast related to target class can lead to:

• Increase model complexity as dimentionality of data remains more.
• Reduces model performance.
• The model might work well in training data but might work poorly on test data because of overfitting caused by the unrelated features.

drop_list = ['Obesity','delayed healing','Itching']
data = data.drop(drop_list, axis='columns')

3. Data Prprocessing and Cleaning

3.1 Identify missing or null values.

data.isnull().sum()

Age                   0
Gender                0
Polyuria              0
Polydipsia            0
sudden weight loss    0
weakness              0
Polyphagia            0
Genital thrush        0
visual blurring       0
Irritability          0
partial paresis       0
muscle stiffness      0
Alopecia              0
class                 0
dtype: int64

data.isna().sum()

Age                   0
Gender                0
Polyuria              0
Polydipsia            0
sudden weight loss    0
weakness              0
Polyphagia            0
Genital thrush        0
visual blurring       0
Irritability          0
partial paresis       0
muscle stiffness      0
Alopecia              0
class                 0
dtype: int64

data

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Irritability	partial paresis	muscle stiffness	Alopecia	class
0	40	1	0	1	0	1	0	0	0	0	0	1	1	1
1	58	1	0	0	0	1	0	0	1	0	1	0	1	1
2	41	1	1	0	0	1	1	0	0	0	0	1	1	1
3	45	1	0	0	1	1	1	1	0	0	0	0	0	1
4	60	1	1	1	1	1	1	0	1	1	1	1	1	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
515	39	0	1	1	1	0	1	0	0	0	1	0	0	1
516	48	0	1	1	1	1	1	0	0	1	1	0	0	1
517	58	0	1	1	1	1	1	0	1	0	1	1	0	1
518	32	0	0	0	0	1	0	0	1	0	0	0	1	0
519	42	1	0	0	0	0	0	0	0	0	0	0	0	0

520 rows × 14 columns

There are no missing or null values in the dataset, therefore, there is no need to apply an additional step to handle missing values.

3.2 Identify duplicates, and remove the duplicate values.

Duplicate records do not provide new informationt to the model and model fails to generlize patterns which fails when unseen data is presented. By removing duplicates, model can focus on unique and meaningfull patterns.

duplicates = data.duplicated
data[duplicates]

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Irritability	partial paresis	muscle stiffness	Alopecia	class
47	60	0	1	1	1	1	1	0	1	0	1	1	0	1
69	50	0	1	1	1	1	1	0	1	0	1	0	0	1
84	35	0	1	1	1	1	1	0	1	0	1	1	0	1
92	40	0	1	1	1	1	0	0	1	0	1	1	0	1
159	38	0	1	1	1	1	1	0	1	1	1	1	0	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
496	53	1	0	0	0	1	0	0	1	0	0	1	1	0
497	47	1	0	0	0	0	0	0	0	1	1	0	0	0
498	68	0	1	1	0	1	1	0	1	0	1	0	0	1
499	64	1	0	0	0	1	1	0	1	1	0	1	1	0
504	38	1	0	0	0	0	0	0	0	0	0	0	0	0

275 rows × 14 columns

data.drop_duplicates(inplace=True)
data

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Irritability	partial paresis	muscle stiffness	Alopecia	class
0	40	1	0	1	0	1	0	0	0	0	0	1	1	1
1	58	1	0	0	0	1	0	0	1	0	1	0	1	1
2	41	1	1	0	0	1	1	0	0	0	0	1	1	1
3	45	1	0	0	1	1	1	1	0	0	0	0	0	1
4	60	1	1	1	1	1	1	0	1	1	1	1	1	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
515	39	0	1	1	1	0	1	0	0	0	1	0	0	1
516	48	0	1	1	1	1	1	0	0	1	1	0	0	1
517	58	0	1	1	1	1	1	0	1	0	1	1	0	1
518	32	0	0	0	0	1	0	0	1	0	0	0	1	0
519	42	1	0	0	0	0	0	0	0	0	0	0	0	0

245 rows × 14 columns

3.3 Identify outliers for Age and handle it.

def find_outliers(data,column):
    Q1 = data[column].quantile(0.25)
    Q3 = data[column].quantile(0.75)
    IQR = Q3 - Q1
    lower_bound = Q1 - 1.5 * IQR
    upper_bound = Q3 + 1.5 * IQR
    outliers = data[(data[column] < lower_bound) | (data[column] > upper_bound)]
       
    return outliers[column]
# Identify outliers for each column
outliers = find_outliers(data, 'Age')
print(outliers)

plt.figure(figsize=(10, 5 ))
plt.boxplot(data['Age'], vert=False)
plt.title('Box Plot of Age')
plt.xlabel('Age')

plt.tight_layout()
plt.show()

102    90
Name: Age, dtype: int64

def impute_outliers_with_median(data, column):
    Q1 = data[column].quantile(0.25)
    Q3 = data[column].quantile(0.75)
    IQR = Q3 - Q1
    lower_bound = Q1 - 1.5 * IQR
    upper_bound = Q3 + 1.5 * IQR
    
    # Calculate median
    median = data[column].median()
    
    # Replace outliers with median
    data[column] = data[column].apply(lambda x: median if (x < lower_bound or x > upper_bound) else x)

# Apply to each column
impute_outliers_with_median(data, "Age")

plt.figure(figsize=(10, 5 ))
plt.boxplot(data['Age'], vert=False)
plt.title('Box Plot of Age')
plt.xlabel('Age')

plt.tight_layout()
plt.show()

Steps performed for data pre-processing

• Identified missing and null values, no further action taken as there are no missing or null values
• Identified the duplicates objects and data set and removed them.
• Plotted histogram for Age to check its distribution and check if it is skewed.
• Identified the outlier in Age using using Box plot, calculating IQR.
• Replaced the outlier with median.

Median is used to replace the outlier in a continous valued attribute as it is NOT highly influenced by values at end of the distribution, whereas mean is highly influenced by outliers, so it is ideal to use Median over mean.

4. Exploratory Data Analysis

4.1 Univariate Analysis

4.1.1 Explore Target variable

Having a count of number of instances available in dataset for each class will give insight about balancing of the training dataset.

data['class'].value_counts()

class
1    170
0     75
Name: count, dtype: int64

(75 / (170+75))*100

30.612244897959183

30.61% of instances belong to the minority class (Non-diabetic), data is moderately imbalanced.

• This will be handled using SMOTE Algorithm while model building.

data['class'].value_counts().plot.barh()

<Axes: ylabel='class'>

4.1.2 Explore gender (Post duplicates removal)

data['Gender'].value_counts().plot.barh(figsize=(10,5),  title= 'Gender Frequency')

<Axes: title={'center': 'Gender Frequency'}, ylabel='Gender'>

4.2 Bivariate Analysis

4.2.1 Comparing gender with target class.

gender_class = pd.crosstab(data['Gender'],data['class'])
gender_class.div(gender_class.sum(1).astype(float), axis=0).plot(kind="bar", stacked=True, figsize=(4,4))

<Axes: xlabel='Gender'>

Observation: - More number of females are diabetic compared to male.

4.2.2 Comparing Age with target class.

Some Feature engineering.

Feature Creation: Binning Age in young(15-35), midle age(36-60) and elderly(60+). Discretization.

In context of Decision tree model, a continous attribute like Age leads to more splits. By binning it, splits are reduced and tree structure is simplified, which is easier to interpret.

bins = [15, 35, 60, 100]  # Age ranges: 15-35, 36-60, 61-100
labels = ['Young', 'Middle-aged', 'Elderly']
data['age_group'] = pd.cut(data['Age'], bins=bins, labels=labels, right=False)
data

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Irritability	partial paresis	muscle stiffness	Alopecia	class	age_group
0	40.0	1	0	1	0	1	0	0	0	0	0	1	1	1	Middle-aged
1	58.0	1	0	0	0	1	0	0	1	0	1	0	1	1	Middle-aged
2	41.0	1	1	0	0	1	1	0	0	0	0	1	1	1	Middle-aged
3	45.0	1	0	0	1	1	1	1	0	0	0	0	0	1	Middle-aged
4	60.0	1	1	1	1	1	1	0	1	1	1	1	1	1	Elderly
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
515	39.0	0	1	1	1	0	1	0	0	0	1	0	0	1	Middle-aged
516	48.0	0	1	1	1	1	1	0	0	1	1	0	0	1	Middle-aged
517	58.0	0	1	1	1	1	1	0	1	0	1	1	0	1	Middle-aged
518	32.0	0	0	0	0	1	0	0	1	0	0	0	1	0	Young
519	42.0	1	0	0	0	0	0	0	0	0	0	0	0	0	Middle-aged

245 rows × 15 columns

ageGroup_class = pd.crosstab(data['age_group'],data['class'])
ageGroup_class.div(ageGroup_class.sum(1).astype(float), axis=0).plot(kind="bar", stacked=True, figsize=(4,4))

<Axes: xlabel='age_group'>

Observation: - Almost 50% of youth are diabetic. - Majorly middle aged people are diabetic.

4.2.3 Compare every occuring symptom with target.

target_column = 'class'  # Replace with the actual target column name
symptoms = [col for col in data.columns if (col != target_column and col!= 'Age' and col != 'Gender' and col != 'age_group')]  # Binary symptom columns

# Perform bivariate analysis for each symptom
for symptom in symptoms:
    # Calculate the proportion of target classes for each value of the symptom
    symptom_analysis = pd.crosstab(data[symptom], data[target_column], normalize='index')
    
    # Plot the data
    symptom_analysis.plot(
        kind='bar',
        stacked=True,
        figsize=(6, 4),
    )

    plt.title(f"Bivariate Analysis: {symptom} vs {target_column}")
    plt.xlabel(f"{symptom} (0=No, 1=Yes)")
    plt.ylabel("Proportion")
    plt.legend(title=target_column, loc='upper right')
    plt.tight_layout()
    plt.show()

Observation: - People having Polyuria or Polydipsia are majorly diabetic.

4.3 Feature Weight

Calculating feature weight using Chi-square test for Gender.

# Create a contingency table
contingency_table = pd.crosstab(data['Gender'], data['class'])

# Perform Chi-Square Test
chi2, p, dof, expected = chi2_contingency(contingency_table)

# Print results
print("Contingency Table:")
print(contingency_table)
print("\nChi-Square Test Results:")
print(f"Chi-Square Statistic: {chi2:.2f}")
print(f"Degrees of Freedom: {dof}")
print(f"P-Value: {p:.4f}")

# Interpret the results
if p < 0.05:
    print("\nThere is a significant relationship between gender and the target variable.")
else:
    print("\nThere is no significant relationship between gender and the target variable.")

Contingency Table:
class    0   1
Gender        
0       11  77
1       64  93

Chi-Square Test Results:
Chi-Square Statistic: 19.90
Degrees of Freedom: 1
P-Value: 0.0000

There is a significant relationship between gender and the target variable.

5. Model Building

5.1 Some feature Engineering

Binarzation of the newly created feature (age_group)

data.info()

<class 'pandas.core.frame.DataFrame'>
Index: 245 entries, 0 to 519
Data columns (total 15 columns):
 #   Column              Non-Null Count  Dtype   
---  ------              --------------  -----   
 0   Age                 245 non-null    float64 
 1   Gender              245 non-null    int8    
 2   Polyuria            245 non-null    int8    
 3   Polydipsia          245 non-null    int8    
 4   sudden weight loss  245 non-null    int8    
 5   weakness            245 non-null    int8    
 6   Polyphagia          245 non-null    int8    
 7   Genital thrush      245 non-null    int8    
 8   visual blurring     245 non-null    int8    
 9   Irritability        245 non-null    int8    
 10  partial paresis     245 non-null    int8    
 11  muscle stiffness    245 non-null    int8    
 12  Alopecia            245 non-null    int8    
 13  class               245 non-null    int8    
 14  age_group           245 non-null    category
dtypes: category(1), float64(1), int8(13)
memory usage: 7.3 KB

col = pd.Categorical(data['age_group'])
data['age_group'] = col.codes

data

	Age	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Irritability	partial paresis	muscle stiffness	Alopecia	class	age_group
0	40.0	1	0	1	0	1	0	0	0	0	0	1	1	1	1
1	58.0	1	0	0	0	1	0	0	1	0	1	0	1	1	1
2	41.0	1	1	0	0	1	1	0	0	0	0	1	1	1	1
3	45.0	1	0	0	1	1	1	1	0	0	0	0	0	1	1
4	60.0	1	1	1	1	1	1	0	1	1	1	1	1	1	2
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
515	39.0	0	1	1	1	0	1	0	0	0	1	0	0	1	1
516	48.0	0	1	1	1	1	1	0	0	1	1	0	0	1	1
517	58.0	0	1	1	1	1	1	0	1	0	1	1	0	1	1
518	32.0	0	0	0	0	1	0	0	1	0	0	0	1	0	0
519	42.0	1	0	0	0	0	0	0	0	0	0	0	0	0	1

245 rows × 15 columns

5.2 Splitting features and class

X = data.drop(labels = ['class', 'Age'],axis="columns") 
X

	Gender	Polyuria	Polydipsia	sudden weight loss	weakness	Polyphagia	Genital thrush	visual blurring	Irritability	partial paresis	muscle stiffness	Alopecia	age_group
0	1	0	1	0	1	0	0	0	0	0	1	1	1
1	1	0	0	0	1	0	0	1	0	1	0	1	1
2	1	1	0	0	1	1	0	0	0	0	1	1	1
3	1	0	0	1	1	1	1	0	0	0	0	0	1
4	1	1	1	1	1	1	0	1	1	1	1	1	2
...	...	...	...	...	...	...	...	...	...	...	...	...	...
515	0	1	1	1	0	1	0	0	0	1	0	0	1
516	0	1	1	1	1	1	0	0	1	1	0	0	1
517	0	1	1	1	1	1	0	1	0	1	1	0	1
518	0	0	0	0	1	0	0	1	0	0	0	1	0
519	1	0	0	0	0	0	0	0	0	0	0	0	1

245 rows × 13 columns

y = data['class']

X.columns

Index(['Gender', 'Polyuria', 'Polydipsia', 'sudden weight loss', 'weakness',
       'Polyphagia', 'Genital thrush', 'visual blurring', 'Irritability',
       'partial paresis', 'muscle stiffness', 'Alopecia', 'age_group'],
      dtype='object')

5.3 Splitting dataset into train and test set

X_train, X_test, y_train, y_test = train_test_split(X,y, test_size =0.2, random_state=28)

X_train_7030, X_test_7030, y_train_7030, y_test_7030 = train_test_split(X,y, test_size =0.3, random_state=28)

Smaller amount of train data (70%) may cause underfitting of the model.

Applying SMOTE Algorithm to handle imbalanced data.

print('Before (80-20)', Counter(y_train))
print('Before (70-30)', Counter(y_train_7030))

# oversampling the train dataset using SMOTE
smt = SMOTE()
X_train_sm, y_train_sm = smt.fit_resample(X_train, y_train)
X_train_7030_sm, y_train_7030_sm = smt.fit_resample(X_train_7030, y_train_7030) 

print('After (80-20)', Counter(y_train_sm))
print('After (70-30)', Counter(y_train_7030_sm))

Before (80-20) Counter({1: 139, 0: 57})
Before (70-30) Counter({1: 122, 0: 49})
After (80-20) Counter({1: 139, 0: 139})
After (70-30) Counter({1: 122, 0: 122})

X_train = X_train_sm
y_train = y_train_sm
X_train_7030 = X_train_7030_sm
y_train_7030 = y_train_7030_sm

dt = DecisionTreeClassifier(random_state = 28)
dt.fit(X_train, y_train)

DecisionTreeClassifier(random_state=28)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

dt_7030 = DecisionTreeClassifier(random_state=30)
dt_7030.fit(X_train_7030, y_train_7030)

DecisionTreeClassifier(random_state=30)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

feature_names = X.columns
feature_names

Index(['Gender', 'Polyuria', 'Polydipsia', 'sudden weight loss', 'weakness',
       'Polyphagia', 'Genital thrush', 'visual blurring', 'Irritability',
       'partial paresis', 'muscle stiffness', 'Alopecia', 'age_group'],
      dtype='object')

target_names = ['Yes','NO']
target_names

['Yes', 'NO']

y_pred = dt.predict(X_test)
y_pred_7030 = dt_7030.predict(X_test_7030)

5.4 Visualizing feature importance for the trained model (80-20 split).

importances=pd.Series(dt.feature_importances_, index=X.columns).sort_values()
importances.plot(kind='barh', figsize=(12,8))

<Axes: >

Polyuria and Polydipsia being important features as observed in bivariate analysis.

actuals = np.array(y_test)
predictions = np.array(y_pred)
confusion_matrix(actuals, predictions)

array([[15,  3],
       [ 3, 28]])

actuals_7030 = np.array(y_test_7030)
predictions_7030 = np.array(y_pred_7030)
confusion_matrix(actuals_7030, predictions_7030)

array([[20,  6],
       [ 3, 45]])

print("Accuracy for Decision Tree with 70-30 Split: ", accuracy_score(y_test_7030, y_pred_7030)*100)
print("Accuracy for Decision Tree with 80-20 Split: ", accuracy_score(y_test, y_pred)*100)

Accuracy for Decision Tree with 70-30 Split:  87.83783783783784
Accuracy for Decision Tree with 80-20 Split:  87.75510204081633

5.5 Cross Validation for hyperparameter tuning.

Validating Hyperparameters for 80-20 split data

param_grid = {
    'max_depth': [3, 5, 10, None],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4],
    'criterion': ['gini', 'entropy']
}
grid_search = GridSearchCV(estimator=dt, param_grid=param_grid, cv=5, scoring='accuracy', verbose=1)
grid_search.fit(X_train, y_train)

# Retrieve and evaluate the best model
best_model = grid_search.best_estimator_
y_pred_best = best_model.predict(X_test)

print("Best Parameters:", grid_search.best_params_)
print("Accuracy with Tuned Model:", accuracy_score(y_test, y_pred_best)*100)

Fitting 5 folds for each of 72 candidates, totalling 360 fits
Best Parameters: {'criterion': 'entropy', 'max_depth': 10, 'min_samples_leaf': 2, 'min_samples_split': 2}
Accuracy with Tuned Model: 87.75510204081633

Validating Hyperparameters for 70-30 split data

param_grid = {
    'max_depth': [3, 5, 10, None],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4],
    'criterion': ['gini', 'entropy']
}
grid_search = GridSearchCV(estimator=dt_7030, param_grid=param_grid, cv=5, scoring='accuracy', verbose=1)
grid_search.fit(X_train_7030, y_train_7030)

# Retrieve and evaluate the best model
best_model = grid_search.best_estimator_
y_pred_7030_best = best_model.predict(X_test_7030)

print("Best Parameters:", grid_search.best_params_)
print("Accuracy with Tuned Model:", accuracy_score(y_test_7030, y_pred_7030_best)*100)

Fitting 5 folds for each of 72 candidates, totalling 360 fits
Best Parameters: {'criterion': 'gini', 'max_depth': 5, 'min_samples_leaf': 2, 'min_samples_split': 2}
Accuracy with Tuned Model: 90.54054054054053

Understanding Overfitting of the model.

When the model with 80-20 split is fit with 10 folds instead of 5 folds, the accuracy of the model changes.

param_grid = {
    'max_depth': [3, 5, 10, None],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4],
    'criterion': ['gini', 'entropy']
}
grid_search = GridSearchCV(estimator=dt, param_grid=param_grid, cv=10, scoring='accuracy', verbose=1)
grid_search.fit(X_train, y_train)

# Retrieve and evaluate the best model
best_model = grid_search.best_estimator_
y_pred_best = best_model.predict(X_test)

print("Best Parameters:", grid_search.best_params_)
print("Accuracy with Tuned Model:", accuracy_score(y_test, y_pred_best)*100)

Fitting 10 folds for each of 72 candidates, totalling 720 fits
Best Parameters: {'criterion': 'entropy', 'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 2}
Accuracy with Tuned Model: 87.75510204081633

6. Performance Evaluation

6.1 Performance Metrics

CM = confusion_matrix(actuals, predictions)
CM_7030 = confusion_matrix(actuals_7030, predictions_7030)


disp = ConfusionMatrixDisplay(confusion_matrix=CM,display_labels=['Positive','Negative'])
disp_7030 = ConfusionMatrixDisplay(confusion_matrix=CM_7030, display_labels=['Positive','Negative'])
disp.plot()
disp_7030.plot()
plt.show()

We will consider model with 80-20 dataset split for comparsion with Logistic Regression Mode. Evaluation metrics used are

• Accuracy
• Precision
• Recall
• F1 Score
• Specificity

6.2 Building Logistic Regression model.

model_LR = LogisticRegression(random_state=28) 
model_LR.fit(X_train, y_train)

LogisticRegression(random_state=28)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

model_LR.classes_

array([0, 1], dtype=int8)

model_LR.coef_

array([[-1.46825469,  2.25303346,  2.63408544,  1.30897724,  0.2754068 ,
         0.70164682,  1.60815608,  0.17973608,  0.95024488,  1.26590417,
        -0.41331926, -0.7221683 , -0.1584415 ]])

model_LR.intercept_

array([-1.20548766])

predicted_LR = model_LR.predict(X_test)
predicted_LR

array([1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1,
       1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1,
       1, 1, 1, 0, 0], dtype=int8)

actual_LR = y_test

accuracy_LR = accuracy_score(actual_LR,predicted_LR)
precision_LR = precision_score(actual_LR, predicted_LR)
recall_LR = recall_score(actual_LR, predicted_LR)
specificity_LR = specificity_score(actual_LR, predicted_LR)
f1_LR = f1_score(actual_LR, predicted_LR)


accuracy = accuracy_score(actuals,predictions)
precision = precision_score(actuals, predictions)
recall = recall_score(actuals, predictions)
specificity = specificity_score(actuals, predictions)
f1 = f1_score(actuals, predictions)

print("Accuracy of Logistic Regression model is:", accuracy_LR*100)
print("Precision of Logistic Regression model is:", precision_LR*100 )
print("Recall of Logistic Regression model is:", recall_LR*100)
print("Specificity of Logistic Regression model is:", specificity_LR*100)
print("F1 Score of Logistic Regression model is:", f1_LR*100)
print("\n_________________________________________________\n")
print("Accuracy of Decision Tree model is:", accuracy*100)
print("Precision of Decision Tree model is:", precision*100)
print("Recall of Decision Tree model is:", recall*100)
print("Specificity of Decision Tree model is:", specificity*100)
print("F1 Score of Decision Tree model is:", f1*100)

Accuracy of Logistic Regression model is: 85.71428571428571
Precision of Logistic Regression model is: 87.5
Recall of Logistic Regression model is: 90.32258064516128
Specificity of Logistic Regression model is: 77.77777777777779
F1 Score of Logistic Regression model is: 88.88888888888889

_________________________________________________

Accuracy of Decision Tree model is: 87.75510204081633
Precision of Decision Tree model is: 90.32258064516128
Recall of Decision Tree model is: 90.32258064516128
Specificity of Decision Tree model is: 83.33333333333334
F1 Score of Decision Tree model is: 90.32258064516128

Conclusion

From the given metrics above, it is clear that Decision Tree performs better, the accuracy and precision of Decision tree model is significantly higher than Logistic Regression. In medical diagnosis of diabetes specificity should be as high as possible to reduce domain specific cost of errors. Spcificity of Decision tree model is higher than that of Logistic Regression model.

Since the data set was balanced using SMOTE algorigthm, Accuracy is a reliable metrics which for Decision tree model is higher Logistic Regression mode.