Pandas is a popular data manipulation library in Python, widely used for data analysis and data science tasks. Pandas Dataframe is a two-dimensional labeled data structure with columns of potentially different types, similar to a spreadsheet or SQL table. One of the common tasks in data manipulation is adding new rows or columns to an existing dataframe. It might seem like a trivial task, but choosing the right method to add rows or columns can significantly impact the performance and efficiency of your code.
In this blog, we will explore the different ways to add rows and columns to a Pandas Dataframe. We will look into different methods available in Pandas, such as .loc, .iloc, .append, .concat, and many more. While working on a data project using Python programming, there are several scenarios when you’ll need to add new rows and columns to your Dataframe. In this article, we will show you how to do it. As data scientists or data analysts, you must get a good understanding of how to add Dataframe rows and columns.
In this post, we will work with the following Pandas data frame.
import pandas as pd
df = pd.DataFrame({
"Mathematics": [95, 99],
"Science": [98, 94]
}, index=["Aiyana", "Anisha"])
df
How to Add a Row in a Dataframe
There are multiple ways of adding rows to Dataframe. You can use Dataframe.loc or Dataframe.append method to add a row at the end of Dataframe. If you want to insert a row at any specific position, then you can use Dataframe.insert() method. Let’s see all these methods one by one with an example.
Method 1: Add a new row at the End using loc method
In this method, we will take the help of the Dataframe.loc method which is used to access a group of rows and columns by label(s). The loc method of Pandas DataFrame allows users to select subsets of data from a DataFrame based on specific row and column labels. The loc method stands for “location” and is used to filter data by specifying the row and column indices.
Add a new row using .loc[label]: If you want to add a single row to your Dataframe, you can use the .loc[] indexing method. The .loc[] method is used to access Dataframe elements by label, and it supports adding new rows as well as creating copies of existing ones. The following code represents how to add a Dataframe row at the end.
# Use loc method to add a new row with label
#
df.loc["Saanvi"] = [96, 90]
Add a new row using .loc[len(df)]: In the above code, the rows got added using label as index. You could also use loc method to add a new row to dataframe which does not have labels defined as indices. In the code given below, we first created a sample DataFrame with two rows and three columns. We then defined a new row to be added to the DataFrame as a dictionary with the same keys (Name, Mathematics, Science) as the DataFrame columns. We, then, used the .loc method to add the new row to the DataFrame at the index position of the last row (which is obtained using len(df)). Here is the Python code:
import pandas as pd
# Create a sample DataFrame
data = {'Name': ['Aiyana', 'Anisha'],
'Mathematics': [95, 98],
'Science': [99, 94]}
df = pd.DataFrame(data)
# Display the original DataFrame
print("Original DataFrame:")
print(df)
# Define the new row to be added
new_row = {'Name': 'Saanvi', 'Mathematics': 96, 'Science': 90}
# Use the loc method to add the new row to the DataFrame
df.loc[len(df)] = new_row
# Display the modified DataFrame
print("\n\nModified DataFrame:")
print(df)
This is what will get printed.
The loc method works by specifying the row index and the column labels where we want to add the new data. In the above two cases, we specified the row index as label and len(df) respectively, which is the next available index after the last row of the DataFrame.
Method 2: Add a new row at the End using append method
In this method, we will take the help of Dataframe.append() method. Dataframe.append() is used to append rows of other Dataframes to the end of this Dataframe, returning a new object. Rows are added at the bottom, so the index labels are increasing, and duplicate index values are not preserved. The code below represents the same:
# Append one or more rows of another dataframe
#
df1 = pd.DataFrame({
"Mathematics": [92],
"Science": [95]
}, index=["Snehal"])
#
# Append a dataframe
#
df = df.append(df1)
Method 3: Add new rows at the End using concat method
The concat method can be used to add rows to a Pandas Dataframe. The concat method takes an iterable of Series or Dataframe objects and concatenates them into a single Dataframe. The concat method can be used to combine two or more Dataframes into a single Dataframe, or to combine a Series and a Dataframe into a single Dataframe. The following code represents way to add one or more rows to the end of Dataframe.
import pandas as pd
#
# Create a dataframe
#
dict = {"Mathematics":[95, 90, 99],
"Science": [99, 95, 92]}
df1 = pd.DataFrame(dict, index=["Aiyana", "Anisha", "Saanvi"])
#
# Create another dataframe
#
df2 = pd.DataFrame({"Mathematics": [96],
"Science": [99]},
index=["Snehal"])
#
# Concat dataframes
#
pd.concat([df1, df2])
When to use which method to add row in dataframe: .loc, .append or .concat
The choice between using .loc, .append, or .concat to add one or more rows to a Pandas dataframe depends on the specific use case and desired outcome.
- Use the .loc method when you want to add a row to a specific index position in the dataframe. This method allows you to specify the index location where you want to add the new row and provide the values for the row as a dictionary.
- Use the .append method when you want to add one or more rows to the end of a dataframe. This method allows you to concatenate a new row to the existing dataframe.
- Use the .concat method when you want to combine two or more dataframes, either vertically or horizontally. This method can be used to add rows or columns to a dataframe, depending on the axis parameter.
How to Add a column in a Dataframe
There are multiple ways of adding columns to Dataframe. You can use Dataframe.loc or bracket method to add a new column at the end of Dataframe. Let’s see all these methods one by one with an example.
Method 1: Add a new Column at the End using loc method
If you want to add a single column to your Dataframe, you can use the .loc[] indexing method. The .loc[] method is used to access Dataframe elements by label, and it supports adding new columns as well as creating copies of existing ones. The following code represents how to add a Dataframe column at the end.
# Adding a new column using loc method
#
df.loc[:, ["English"]] = [85, 92, 79, 87]
Method 2: Add a new Column at the End using brackets
In this method, we will take the help of using brackets on data frame object to insert a new column. The column is inserted at the end of all the columns. The following code represents how to add columns using brackets.
# Adding a new column using brackets
#
df["Hindi"] = [81, 79, 72, 76]
Method 3: Add a new Column at the End using insert method
In this method, we will take the help of Dataframe.insert() method. Dataframe.insert() is used to insert a column in Dataframe at a specified location. The column is inserted at the given position among all columns. The following code represents how to add a Dataframe column using Dataframe.insert() method:
# Adding a column at a specified position using insert method
#
df.insert(1, "Social Science", [86, 78, 82, 80])
When to use which method to add columns: .loc, .insert, label
The choice between using .loc, .insert, or adding at the end of columns to add one or more columns to a Pandas dataframe depends on the specific use case and desired outcome. Here are some guidelines to help you decide which method to use:
- Use the .loc method when you want to add a new column with specific values at a specific index position in the dataframe. This method allows you to specify the index location where you want to add the new column and provide the values for the column as a list or an array.
- Use the .insert method when you want to add a new column at a specific index position and shift the existing columns to the right. This method allows you to specify the index location where you want to add the new column, provide a name for the column, and provide the values for the column as a list or an array.
- Use the “label” (adding at the end of columns) method when you want to add a new column at the end of the dataframe. This method allows you to append a new column to the existing dataframe.
Conclusion
Adding rows and columns to Dataframe is a very easy process. In this article, we have shown you different ways in which you can add new rows and columns to your Dataframe. We have also provided examples for each method so that you can understand it better. So, go ahead and try these methods out while working on your data projects using Python programming.
Check out my latest book titled as First Principles Thinking: Building winning products using first principles thinking
- Generative Modeling in Machine Learning: Examples - March 19, 2023
- Data Analytics Training Program (Beginners) - March 18, 2023
- Histogram Plots using Matplotlib & Pandas: Python - March 18, 2023
Leave a Reply