Real World Data Cleaning in Python Pandas (Step By Step)

🧠 Don’t miss out! Get FREE access to my Skool community — packed with resources, tools, and support to help you with Data, Machine Learning, and AI Automations! 📈 https://www.skool.com/data-and-ai-aut... In this video, I show you how to clean up data within Python Pandas within Jupyter notebook. This Python tutorial is great for those trying to get into Data Analytics or Data Science. Cricket Data: https://www.espncricinfo.com/records/... Code: https://ryanandmattdatascience.com/py... 🚀 Hire me for Data Work: https://ryanandmattdatascience.com/da... 👨‍💻 Mentorships: https://ryanandmattdatascience.com/me... 📧 Email: [email protected] 🌐 Website & Blog: https://ryanandmattdatascience.com/ 🖥️ Discord:   / discord   📚 *Practice SQL & Python Interview Questions: https://stratascratch.com/?via=ryan 📖 *SQL and Python Courses: https://datacamp.pxf.io/XYD7Qg 🍿 WATCH NEXT Python Pandas Playlist:    • Python Pandas for Beginners   Python Groupby:    • The Complete Guide to Python Pandas Groupby   Python Pandas Interview Questions:    • 23 Python Pandas Coding Interview Question...   Python Lambda Functions:    • Python Pandas Lambda Function Tutorial Wit...   In this comprehensive tutorial, I walk you through essential data cleaning techniques using pandas with real cricket data from ESPN. You'll learn how to transform messy, raw data into a clean, analysis-ready dataset by tackling common data challenges that you'll encounter in any data science project. We start by extracting data directly from a website using Excel's web query feature, then dive deep into pandas to rename columns, handle null values, remove duplicates, and split complex string data. I show you how to manipulate the player span column to create separate start and end date fields, remove unwanted characters like asterisks and plus signs, and properly convert data types from objects to integers and floats. Throughout the video, we encounter real debugging scenarios—like dealing with unexpected NaN values and invalid data formats—and I demonstrate exactly how to troubleshoot and fix these issues. You'll see how to use string methods like str.split(), handle missing data with fillna(), drop unnecessary columns and rows, and create new calculated fields like career length. By the end, you'll confidently perform data cleaning operations including dropping duplicates, converting data types, creating new series, manipulating strings, debugging common pandas errors, and using group by operations to answer analytical questions. All code and data files are available in the description so you can follow along and practice these essential data cleaning skills yourself. TIMESTAMPS 00:00 Introduction & Data Overview 01:40 Importing Data from ESPN to Excel 05:17 Loading CSV into Pandas 06:02 Renaming Columns 09:00 Checking for Null Values 10:17 Handling Missing Data with fillna 11:57 Finding & Removing Duplicates 15:17 Splitting the Span Column 17:32 Creating Rookie Year & Final Year Columns 18:15 Dropping Unnecessary Columns 19:05 Manipulating Player Names & Country Data 22:40 Checking & Converting Data Types 25:17 Removing Special Characters (Stars) 27:32 Converting Data Types (int & float) 29:30 Debugging Data Type Conversion Issues 32:00 Dropping Problematic Rows 33:44 Creating Career Length Column 35:17 Question 1: Average Career Length 35:55 Question 2: Batting Strike Rate Analysis 36:42 Question 3: Players Before 1960 37:33 Question 4: Group By Country Analysis 38:35 Question 5: Averages by Country OTHER SOCIALS: Ryan’s LinkedIn:   / ryan-p-nolan   Matt’s LinkedIn:   / matt-payne-ceo   Twitter/X: https://x.com/RyanMattDS Who is Ryan Ryan is a Data Scientist at a fintech company, where he focuses on fraud prevention in underwriting and risk. Before that, he worked as a Data Analyst at a tax software company. He holds a degree in Electrical Engineering from UCF. Who is Matt Matt is the founder of Width.ai, an AI and Machine Learning agency. Before starting his own company, he was a Machine Learning Engineer at Capital One. *This is an affiliate program. We receive a small portion of the final sale at no extra cost to you.