Cleaning Student Performance Data with Pandas – My First Practical Data Cleaning Case Study 📊🧹

Hey there 👋
I’m Avadhoot Kamble, and this blog is a breakdown of one of my first real-world data cleaning projects using Pandas in Python.

I recently downloaded a dataset from Kaggle on student performance and decided to clean it thoroughly using only code — no Excel this time! In this post, I’ll walk you through everything I did, step-by-step, including the challenges I faced and how I solved them.

Whether you're a beginner learning Pandas or someone curious about how data is cleaned before analysis or modeling, this post will give you a solid, realistic picture.

📁 Dataset Overview

The dataset I worked on contains student information — not just their test scores, but also family, education, and lifestyle-related features. Here’s what each column means:

One weird thing I spotted was an Unnamed: 0 column — probably an index from an earlier save. I dropped it immediately:

data = data.drop(columns=['Unnamed: 0'])

🛠️ Step 2: Handling Null Values

Several columns had missing values. Here’s how I tackled them:

🔹 EthnicGroup & ParentEduc

I filled their missing values using the most frequent value (mode):

data['EthnicGroup'] = data['EthnicGroup'].fillna(data['EthnicGroup'].mode()[0])
data['ParentEduc'] = data['ParentEduc'].fillna(data['ParentEduc'].mode()[0])

🔹 TestPrep, ParentMaritalStatus, PracticeSport

I looped through these related columns and applied mode-filling as well:

col_to_fill = ['TestPrep', 'ParentMaritalStatus', 'PracticeSport']
for col in col_to_fill:
    data[col] = data[col].fillna(data[col].mode()[0])

🎯 Step 3: Fixing IsFirstChild (Binary Encoding)

This column had "yes"/"no" values. I filled nulls with mode and mapped it to 1s and 0s for binary classification:

data['IsFirstChild'] = data['IsFirstChild'].fillna(data['IsFirstChild'].mode()[0])
data['IsFirstChild'] = data['IsFirstChild'].map({'yes': 1, 'no': 0})

📊 Step 4: Numerical Fixes – NrSiblings, WklyStudyHours

For NrSiblings, I filled nulls with the median and converted it to integer:

data['NrSiblings'] = data['NrSiblings'].fillna(data['NrSiblings'].median())
data['NrSiblings'] = data['NrSiblings'].astype('int64')

For WklyStudyHours, I:

• Removed spaces

• Lowercased all entries

• Mapped ranges (<5, 5-10, >10) to numerical midpoints: 2.5, 7.5, 12.5

• Filled any missing values with the median

data['WklyStudyHours'] = data['WklyStudyHours'].str.replace(" ", "").str.lower()
data['WklyStudyHours'] = data['WklyStudyHours'].map({'<5': 2.5, '5-10': 7.5, '>10': 12.5})
data['WklyStudyHours'] = data['WklyStudyHours'].fillna(data['WklyStudyHours'].median())

Column “WklyStudyHours” before mapping:

Column “WklyStudyHours” after mapping:

✂️ Step 5: Categorical Cleaning – Lowercasing & Trimming

I applied .str.strip().str.lower() on a few selected columns to clean casing and whitespace:

pythonCopyEditcol_to_clean = ['TestPrep', 'Gender', 'PracticeSport']
for col in col_to_clean:
    data[col] = data[col].str.strip().str.lower()

---

🔎 Step 6: Checking for Outliers in Score Columns

Before finalizing the dataset, I wanted to make sure there were no unusual or invalid scores in the three academic columns: MathScore, ReadingScore, and WritingScore.

I used the .unique() function to inspect the values in each column:

data['ReadingScore'].unique() #checking for outliers
data['MathScore'].unique()
data['WritingScore'].unique()

🧹 Step 7: Removing Duplicates

Finally, I checked for duplicate rows and dropped them if any were found:

pythonCopyEditdata.duplicated().sum()
data = data.drop_duplicates()

---

💾 Final Step: Exporting Cleaned Dataset

Once everything looked clean and consistent, I saved the dataset:

pythonCopyEditdata.to_csv("Cleaned student exam data.csv", index=False)

✅ Done! I now had a cleaned, structured dataset ready for EDA or modeling.

💡 What I Learned from This Project

• Real data isn’t clean — even well-structured datasets need attention.

• Pandas is powerful — chaining small functions (.map(), .fillna(), .str.strip()) creates big improvements.

• Data cleaning is decision-making — context matters when choosing between median, mode, or dropping.

🚀 What’s Next?

This was my first real experience cleaning a dataset using only code. Next, I’ll be working on:

• 🔎 Exploratory Data Analysis with Pandas and Seaborn

• 📈 Visualizations comparing scores by gender, lunch type, parental education

• 🤖 My first machine learning project: predicting test outcomes

📂 Project Files on GitHub

You can find the full project here, including:

• The Jupyter Notebook (StudentsExamScores.ipynb)

• The Raw and Cleaned CSVs

🔗 View the Project on GitHub

🙌 Let’s Connect

If you're on a similar path or want to share tips, feedback, or connect — I’d love to hear from you!

• 💻 GitHub – Projects

• 🌐 LinkedIn – Say hi

• 📝 More Blogs on Hashnode

Thanks for reading 🙏

---

#Pandas #Python #DataCleaning #DataScience #KaggleDataset #LearningInPublic #BeginnerProjects #JupyterNotebook #PortfolioProject