Python Pandas - Stacking and Unstacking

---

---

Stacking and unstacking in Pandas are the useful techniques for reshaping DataFrames to extract more information in different ways. It works efficiently with multi-level indices also. Whether it's compressing columns into row levels or expanding rows into columns, these operations are crucial for handling complex datasets.

The Pandas library provides two main methods for these operations −

• stack(): Used for Stacking, converts columns into row indices, creating a long-format DataFrame.

• unstack(): For Unstacking, which moves an index level back to columns, converting long-format data into a wide format.

In this tutorial, we will learn about stacking and unstacking techniques in Pandas along with practical examples, including handling missing data.

Stacking in Pandas

Stacking in Pandas is a process of compressing a DataFrame columns into rows. TheDataFrame.stack() method in Pandas is used for stacking the levels from column to index. This method pivots a level of column labels (possibly hierarchical) into row labels, and returns a new DataFrame or Series with a multi-level index.

Example

Following example uses thedf.stack() method for pivoting the columns into the row index.

import pandas as pdimport numpy as np# Create MultiIndextuples = [["x", "x", "y", "y", "", "f", "z", "z"],["1", "2", "1", "2", "1", "2", "1", "2"]]index = pd.MultiIndex.from_arrays(tuples, names=["first", "second"])# Create a DataFramedf = pd.DataFrame(np.random.randn(8, 2), index=index, columns=["A", "B"])# Display the input DataFrameprint('Input DataFrame:\n', df)# Stack columnsstacked = df.stack()print('Output Reshaped DataFrame:\n', stacked)

Following is the output of the above code −

Input DataFrame:

		A	B
first	second
x	1	0.596485	-1.356041
	2	-1.091407	0.246216
y	1	0.499328	-1.346817
	2	-0.893557	0.014678
1	-0.059916	0.106597
f	2	-0.315096	-0.950424
z	1	1.050350	-1.744569
	2	-0.255863	0.539803

Output Reshaped DataFrame:

first	second
x	1	A	0.596485
		B	-1.356041
	2	A	-1.091407
		B	0.246216
y	1	A	0.499328
		B	-1.346817
	2	A	-0.893557
		B	0.014678
	1	A	-0.059916
		B	0.106597
f	2	A	-0.315096
		B	-0.950424
z	1	A	1.050350
		B	-1.744569
	2	A	-0.255863
		B	0.539803

dtype: float64

Here, thestack() method pivots the columns A and B into the index, compressing the DataFrame into a long format.

Unstacking in Pandas

Unstacking reverses the stacking operation by moving the row index level back to the columns. The PandasDataFrame.unstack() method is used to pivot a level of the row index to become a column, which is useful for converting a long-format DataFrame into a wide format.

Example

The following example demonstrates the working of thedf.unstack() method for unstacking a DataFrame.

import pandas as pdimport numpy as np# Create MultiIndextuples = [["x", "x", "y", "y", "", "f", "z", "z"],["1", "2", "1", "2", "1", "2", "1", "2"]]index = pd.MultiIndex.from_arrays(tuples, names=["first", "second"])# Create a DataFramedf = pd.DataFrame(np.random.randn(8, 2), index=index, columns=["A", "B"])# Display the input DataFrameprint('Input DataFrame:\n', df)# Unstack the DataFrameunstacked = df.unstack()print('Output Reshaped DataFrame:\n', unstacked)

Following is the output of the above code −

Input DataFrame:

		A	B
first	second
x	1	-0.407537	-0.957010
	2	0.045479	0.789849
y	1	0.751488	-0.474536
	2	-1.043122	-0.015152
1	-0.133349	1.094900
f	2	1.681111	2.480652
z	1	0.283679	0.769553
	2	-2.034907	0.301275

Output Reshaped DataFrame: A B second 1 2 1 2first -0.133349 NaN 1.094900 NaNf NaN 1.681111 NaN 2.480652x -0.407537 0.045479 -0.957010 0.789849y 0.751488 -1.043122 -0.474536 -0.015152z 0.283679 -2.034907 0.769553 0.301275

Handling Missing Data during Unstacking

Unstacking can produce missing values when the reshaped DataFrame has unequal label sets in subgroups. Pandas handles these missing values with NaN by default, but you can specify a custom fill value.

Example

This example demonstrates how to handle missing values when unstacking a DataFrame.

import pandas as pdimport numpy as np# Create Dataindex = pd.MultiIndex.from_product([["bar", "baz", "foo", "qux"], ["one", "two"]], names=["first", "second"])columns = pd.MultiIndex.from_tuples([("A", "cat"), ("B", "dog"), ("B", "cat"), ("A", "dog")], names=["exp", "animal"])df = pd.DataFrame(np.random.randn(8, 4), index=index, columns=columns)# Create a DataFramedf3 = df.iloc[[0, 1, 4, 7], [1, 2]]print(df3)# Unstack the DataFameunstacked = df3.unstack()# Display the Unstacked DataFrameprint("Unstacked DataFrame without Filling:\n",unstacked)unstacked_filled = df3.unstack(fill_value=1)print("Unstacked DataFrame with Filling:\n",unstacked_filled)

Following is the output of the above code −

exp                  B          animal             dog       catfirst second                    bar   one    -0.556587 -0.157084      two     0.109060  0.856019foo   one    -1.034260  1.548955qux   two    -0.644370 -1.871248Unstacked DataFrame without Filling:exp            B                             animal       dog                cat          second       one      two       one       twofirst                                        bar    -0.556587  0.10906 -0.157084  0.856019foo    -1.034260      NaN  1.548955       NaNqux          NaN -0.64437       NaN -1.871248Unstacked DataFrame with Filling:exp            B                             animal       dog                cat          second       one      two       one       twofirst                                        bar    -0.556587  0.10906 -0.157084  0.856019foo    -1.034260  1.00000  1.548955  1.000000qux     1.000000 -0.64437  1.000000 -1.871248