Python Pandas - Aggregations

Aggregating data is a key step in data analysis, especially when dealing with large datasets. In Pandas, you can perform aggregations using the DataFrame.agg() method, This method is flexible, enabling various operations that summarize and analyze your data. Aggregation operations in Pandas can be applied to either the index axis (default) or the column axis.

In this tutorial we will discuss about how to use the DataFrame.agg() method to perform various aggregation techniques, including how to apply multiple aggregation functions, customize aggregations for specific columns, and work with both rows and columns.

Understanding the DataFrame.agg() Method

TheDataFrame.agg() method (an alias for aggregate) is a powerful tool that allows you to apply one or more aggregation functions to a DataFrame, either across rows or columns, providing a summary of the data.

Syntax

Following is the syntax −

DataFrame.agg(func=None, axis=0, *args, **kwargs)

Where,

• func: This parameter specifies the aggregation function(s) to be applied. It accepts a single function or function name (e.g., np.sum, 'mean'), a list of functions or function names, or a dictionary mapping axis labels to functions.

• axis: Specifies the axis along which to apply the aggregation. 0 or 'index' applies the function(s) to each column (default), while 1 or 'columns' applies the function(s) to each row.

• *args: Positional arguments to pass to the aggregation function(s).

• **kwargs: Keyword arguments to pass to the aggregation function(s).

The result ofagg() method depends on the input, it returns ascalar orSeries if a single function is used, or aDataFrame if multiple functions are applied.

Applying Aggregations on DataFrame Rows

You can aggregate multiple functions over the rows (index axis) using the agg function. This method applies the specified aggregation functions to each column in the DataFrame.

Example

Let us create a DataFrame and apply aggregation functionssum andmin on it. In this example, the sum and min functions are applied to each column, providing a summary of the data.

import pandas as pdimport numpy as npdf = pd.DataFrame([[1, 2, 3, 1],                   [4, 5, 6, np.nan],                   [7, 8, 9, 2],                   [np.nan, 2, np.nan, 3]],   index = pd.date_range('1/1/2024', periods=4),   columns = ['A', 'B', 'C', 'D'])print("Input DataFrame:\n",df)result = df.agg(['sum', 'min'])print("\nResults:\n",result)

Itsoutput is as follows −

Input DataFrame:               A  B    C    D2024-01-01  1.0  2  3.0  1.02024-01-02  4.0  5  6.0  NaN2024-01-03  7.0  8  9.0  2.02024-01-04  NaN  2  NaN  3.0Results:         A   B     C    Dsum  12.0  17  18.0  6.0min   1.0   2   3.0  1.0

Applying Different Functions Per Column

You can also apply different aggregation functions to different columns by passing a dictionary to the agg function. Each key in the dictionary corresponds to a column, and the value is a list of aggregation functions to apply.

import pandas as pdimport numpy as npdf = pd.DataFrame([[1, 2, 3, 1],                   [4, 5, 6, np.nan],                   [7, 8, 9, 2],                   [np.nan, 2, np.nan, 3]],   index = pd.date_range('1/1/2024', periods=4),   columns = ['A', 'B', 'C', 'D'])print("Input DataFrame:\n",df)result = df.agg({'A': ['sum', 'min'], 'B': ['min', 'max']})print("\nResults:\n",result)

On executing the above code, it produces followingoutput:

Input DataFrame:               A  B    C    D2024-01-01  1.0  2  3.0  1.02024-01-02  4.0  5  6.0  NaN2024-01-03  7.0  8  9.0  2.02024-01-04  NaN  2  NaN  3.0Results:         A    Bsum  12.0  NaNmin   1.0  2.0max   NaN  8.0

Apply Aggregation on a Single Column

You can apply aggregation functions to individual columns, such as calculating a rolling sum.

import pandas as pdimport numpy as npdf = pd.DataFrame(np.random.randn(10, 4),   index = pd.date_range('1/1/2000', periods=10),   columns = ['A', 'B', 'C', 'D'])print(df)r = df.rolling(window=3,min_periods=1)print(r['A'].aggregate(np.sum))

Itsoutput is as follows −

                 A           B           C           D2000-01-01   1.088512   -0.650942   -2.547450   -0.5668582000-01-02   1.879182   -1.038796   -3.215581   -0.2995752000-01-03   1.303660   -2.003821   -3.155154   -2.4793552000-01-04   1.884801   -0.141119   -0.862400   -0.4833312000-01-05   1.194699    0.010551    0.297378   -1.2166952000-01-06   1.925393    1.968551   -0.968183    1.2840442000-01-07   0.565208    0.032738   -2.125934    0.4827972000-01-08   0.564129   -0.759118   -2.454374   -0.3254542000-01-09   2.048458   -1.820537   -0.535232   -1.2123812000-01-10   2.065750    0.383357    1.541496   -3.2014692000-01-01   1.0885122000-01-02   1.8791822000-01-03   1.3036602000-01-04   1.8848012000-01-05   1.1946992000-01-06   1.9253932000-01-07   0.5652082000-01-08   0.5641292000-01-09   2.0484582000-01-10   2.065750Freq: D, Name: A, dtype: float64

Customizing the Result

Pandas allows you to aggregate different functions across the columns and rename the resulting DataFrame's index. This can be done by passing tuples to theagg() function.

Example

The following example applies the aggregation with custom index labels.

import pandas as pdimport numpy as npdf = pd.DataFrame([[1, 2, 3, 1],                   [4, 5, 6, np.nan],                   [7, 8, 9, 2],                   [np.nan, 2, np.nan, 3]],   index = pd.date_range('1/1/2024', periods=4),   columns = ['A', 'B', 'C', 'D'])print("Input DataFrame:\n",df)result = df.agg(x=('A', 'max'), y=('B', 'min'), z=('C', 'mean'))print("\nResults:\n",result)

Itsoutput is as follows −

Input DataFrame:               A  B    C    D2024-01-01  1.0  2  3.0  1.02024-01-02  4.0  5  6.0  NaN2024-01-03  7.0  8  9.0  2.02024-01-04  NaN  2  NaN  3.0Results:      A    B    Cx  7.0  NaN  NaNy  NaN  2.0  NaNz  NaN  NaN  6.0

Applying Aggregating Over Columns

In addition to aggregating over rows, you can aggregate over the columns by setting the axis parameter to columns (axis=1). This is useful when you want to apply an aggregation function across the rows.

Example

This example applies themean() function across the columns for each row.

import pandas as pdimport numpy as npdf = pd.DataFrame([[1, 2, 3, 1],                   [4, 5, 6, np.nan],                   [7, 8, 9, 2],                   [np.nan, 2, np.nan, 3]],   index = pd.date_range('1/1/2024', periods=4),   columns = ['A', 'B', 'C', 'D'])print("Input DataFrame:\n",df)result = df.agg("mean", axis="columns")print("\nResults:\n",result)

Itsoutput is as follows −

Input DataFrame:               A  B    C    D2024-01-01  1.0  2  3.0  1.02024-01-02  4.0  5  6.0  NaN2024-01-03  7.0  8  9.0  2.02024-01-04  NaN  2  NaN  3.0Results: 2024-01-01    1.752024-01-02    5.002024-01-03    6.502024-01-04    2.50Freq: D, dtype: float64