Python for Data Science Intermediate: Cleaning Data

Name: Python for Data Science Intermediate: Cleaning Data
Author: Nizar Dajani

Learn how to clean data with Python

Preview the first lesson free — get full access to all 3 lessons.

Course: On-Demand

Hello, my name is Nizar Dajani, and in these lessons, you will learn about different data cleaning practices, and in this lesson, you will learn about data inspection. Data cleaning, also known as data cleansing or data scrubbing, is an important step in the data preparation process that involves identifying and correcting errors, inconsistencies, and inaccuracies in datasets to ensure that the data is accurate, reliable, and suitable for analysis. Python, being a popular programming language for data analysis, provides many built-in functions for inspecting data, which can be used to better understand the structure, the type, and values of data. Here are some commonly used data inspection functions in Python. The first example is the type function. This is used to identify the data type of an object in Python. For example, if I call the type function and pass it the value 10 as an argument, I should get back a result that this is an integer type, and if I pass it a value of 3.14, I should get a float type, and if I pass the type function the word hello, I should get back string as the type. Now, when I pass it a list of numbers as we see here, then I should get back a list as my type. And finally, if I pass it a dictionary of one key value pair, then the type function is intelligent enough to know that this data is of the dictionary type. So if I run this code, I get back the data types of each value that was passed as an argument to the type function. The next example shows the len function. It's common practice encoding to know the length of an item. This could be the number of items in your list, the count of letters in a string, or the number of key value pairs in a data dictionary. So here I have a list called list of invited guests, and it has these names. Then I have a phrase with a string value, we are living. Then I have a dictionary called data dictionary, and it contains these key value pairs. And now I want to inspect the length of each variable. So I call the len function and pass it the names of these variables. And when I run this code, I get back the length of each variable, the list of invited guests contains five names, the phrase variable contains 13 characters, and that includes spaces. And the data dictionary contains three key value pairs. The next code example brings focus to the print function. The print function in Python is a built-in function that outputs data to the console. It is often used for debugging purposes, such as inspecting the values of variables, displaying messages, and tracing the execution of a program. I can use the print function to display the value of one or more expressions. And by default, the function separates the expressions with a single space and appends a new line at the end. So if I run this code cell, I get 10, which is the value of the x variable, followed by a space, followed by 20, which is the value of the y variable. Now I can control the separator and the end of line character by specifying the set and end arguments as I have in the following code cell. This will separate my values with a comma and end the output with an exclamation mark. So let me run this code. And there you go. We can also inspect our data when working with a pandas series and pandas data frames. The next code cell has a series of hard-coded values, and I can call the describe method in order to inspect my data. So let me run this code, and I get back helpful information like the count, the mean value, standard deviation, the mean and max, as well as the 25th percentile, 50th percentile, and 75th percentile. Another pandas method we can call to inspect our data is the info method. In my next code cell, I call this method on my data, and when I run it, I get back my range index, which is five, ranging from zero through four. I also get my pandas series name, which currently is not set, so it's none. And I also can see if I have any null values, which in this case I don't, and I also get the data types, which is integer. Then my last three code cell examples will use a pandas data frame to inspect our data. So in my next example, I have a data frame loaded with a data dictionary, which contains two keys, and each key has a list of three values. And just like we can use the print function to inspect our data, I can also use the data frames variable name, which in this case is just TF to inspect our data. And if I run this code cell, I get my data frame. It's a nice representation of my data, and I can inspect it visually. Now sometimes I only want to inspect the first row of my data frame to see what it looks like. For that, I can go to my next code cell, and here I call the head method of my data frame. By default, the head method returns the first five records of data, plus the column headers. But since I only want the first record, then I'll pass the number one as the argument, the head method, as you can see in this example. I'll now run my code, and I do just get the first row in my data frame. The next cell contains a code to show the last record, my data frame. So if head shows the first few records, then tail is the method to inspect our last record. And by default, it will return the last five records, but I only want to inspect the very last record. So I'll pass that value one as an argument, then I'll run my code, and that's what I get back my last record in my data frame. So in this lesson, you learned about data inspection, and in the next lesson, you will learn about inconsistent data. Thanks for watching.

Intermediate Provider : Nizar Dajani 3 Lessons · 20m in English

Course Description

In these lessons, you will learn how to clean data. Python comes with several built-in functions and methods, as well as external libraries that we can import to allow us to clean data effectively and efficiently. These lessons will cover some of these functions and methods, including approaches to inspecting data and working with inconsistent data.

Learning how to clean the data is an essential skill in the data preparation process, which is why this course also covers topics like data cleansing and data scrubbing. You will understand how to identify and correct errors, inconsistencies, and inaccuracies in datasets to ensure that the data is accurate, reliable, and suitable for analysis. As a popular programming language for data analysis, Python provides many built-in functions for inspecting data, which can be used to better understand the data’s structure, type, and values. We will leverage some of Python’s functions to accomplish this, as well as import the Pandas library.

What You'll Learn

Describe the different approaches to data cleansing, including data scrubbing
Recognize the different cleaning functions and methods Python provides
Identify how to inspect data to understand its structure, type, and values
Explain how to work with inconsistent data
Use Python's built-in functions and the Pandas library to clean data effectively and efficiently
Identify and correct errors, inconsistencies, and inaccuracies in datasets

Key Takeaways

Data cleaning is an essential skill in the data preparation process, ensuring data is accurate, reliable, and suitable for analysis.
Python comes with several built-in functions and methods, plus external libraries such as Pandas, that can be imported to clean data effectively and efficiently.
Python's built-in functions for inspecting data help you better understand the data's structure, type, and values.
Working with inconsistent data involves identifying and correcting errors, inconsistencies, and inaccuracies in datasets.
The course covers data cleansing and data scrubbing as part of the broader data preparation process.

Frequently Asked Questions

What will I learn in this course?

You will learn how to clean data using Python's built-in functions and methods as well as the Pandas library, including how to inspect data, work with inconsistent data, and identify and correct errors, inconsistencies, and inaccuracies in datasets.

What topics does this course cover?

The course covers data inspection, inconsistent data, and functions and methods, along with data cleansing and data scrubbing as part of the data preparation process.

Why is learning to clean data important?

Learning how to clean data is an essential skill in the data preparation process, ensuring that data is accurate, reliable, and suitable for analysis.

What Python tools does this course use for data cleaning?

It uses Python's built-in functions and methods for inspecting and cleaning data, and also imports the Pandas library to help clean data effectively and efficiently.

What skills will I gain from this course?

You will gain skills in data abstraction, data class, data cleansing, data manipulation, data science, and working with dirty data.