Python Pandas to_pickle() Method

---

---

Theto_pickle() method in Python's Pandas library allows you to serialize the Pandas objects, such as DataFrame, or Series, into a file or file-like object in the pickle format. This allows you to save your data and load it later for future use. This method can also handle compression, support different file formats, and allow customization using various parameters.

Pickle is a Python-specific file format used for serializing and deserializing Python objects. Serialization, also known aspickling, refers to the process of converting a Python Pandas object (like DataFrame or Series) into a byte stream, which can be stored or transmitted.

Syntax

Following is the syntax of the Python Pandas to_pickle() method −

DataFrame.to_pickle(path, *, compression='infer', protocol=5, storage_options=None)

When using theto_pickle() method on aSeries object, you should call it asSeries.to_pickle().

Parameters

The Python Pandas to_pickle() method accepts the below parameters −

• path; This parameter accepts a string, path object, or file-like object, representing the location where the pickled object will be stored.

• compression: Specifies the compression method to use. If set to 'infer', the method will automatically detect the compression type based on the file extension (e.g., .gz, .bz2, .zip). You can also pass a dictionary to customize compression methods such as gzip, zip, bz2, zstd, etc. If set toNone, no compression will be applied.

• protocol: This parameter takes an integer indicating which pickle protocol to use for serializing the object. The default is theHIGHEST_PROTOCOL. The possible values are 0, 1, 2, 3, 4, and 5. If a negative value is provided, it is equivalent to using the default protocol.

• storage_options: Additional options for connecting to certain storage back-ends (e.g., AWS S3, Google Cloud Storage).

Return Value

The Pandasto_pickle() method returnsNone, but saves the DataFrame or Series as a serialized object at the specified path.

Example: Saving a DataFrame to a Pickle File

Here is a basic example demonstrating saving a Pandas DataFrame object into a pickle file using theDataFrame.to_pickle() method.

import pandas as pd# Create a DataFramedf = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)})print("Original DataFrame:")print(df)# Save the DataFrame as a pickle filedf.to_pickle("df_pickle_file.pkl")print("\nDataFrame is successfully saved as a pickle file.")

When we run above program, it produces following result −

Original DataFrame:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame is successfully saved as a pickle file.

If you check the folder where the pickle file was saved, you will find the generated file.

Example: Saving Pandas Series to a Pickle File

This example saves a Pandas Series object into a pickle file using theSeries.to_pickle() method.

import pandas as pd# Creating a Pandas Seriess = pd.Series([1, 2, 3, 4], index=["cat", "dog", "fish", "mouse"])# Display the Input Seriesprint("Original Series:")print(s)# Save the Series as a pickle files.to_pickle("series_to_pickle_file.pkl")print("\nPandas Series is successfully saved as a pickle file.")

While executing the above code we get the following output −

Original Series:cat      1dog      2fish     3mouse    4dtype: int64Pandas Series is successfully saved as a pickle file.

Example: Saving pickle file with Compression

The following example demonstrates how to use theto_pickle() method to compress a Pandas DataFrame usinggzip compression.

import pandas as pd# Create a DataFramedf = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)})print("Original DataFrame:")print(df)# Save the DataFrame to a pickle file with gzip compressiondf.to_pickle("dataframe_compressed.pkl", compression="gzip")print("\nDataFrame is successfully saved as a pickle file with gzip compression.")

Following is an output of the above code −

Original DataFrame:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame is successfully saved as a pickle file with gzip compression.

Example: Saving Pickle with a Custom Compression Method

Theto_pickle() method can also accepts a dictionary for customizing the compression method. Here, we apply a custom compression method (zip) with specific compression level.

import pandas as pd# Create a DataFramedf = pd.DataFrame({"Col_1": [1, 2, 3, 4, 5],"Col_2": ["a", "b", "c", "d", "e"]})print("Original DataFrame:")print(df)# Save the DataFrame to a pickle file with custom zip compressiondf.to_pickle("dataframe_custom_compressed.pkl", compression={'method': 'zip', 'compresslevel': 2})print("\nDataFrame is successfully saved as a pickle file with custom zip compression.")

Following is an output of the above code −

Original DataFrame:

	Col_1	Col_2
0	1	a
1	2	b
2	3	c
3	4	d
4	5	e

DataFrame is successfully saved as a pickle file with custom zip compression.