Python Pandas to_orc() Method

---

---

The ORC (Optimized Row Columnar) format is a binary columnar serialization designed for efficient reading and writing of DataFrames, similar to theParquet Format. It provides compact and high-speed columnar data storage, facilitating efficient data exchange across different data analysis tools.

The PandasDataFrame.to_orc() method allows you to save DataFrames in theORC format, enabling efficient data storage and sharing across different tools. However, there are some limitations −

• Timezone handling: Timezones in datetime columns are not preserved when saving DataFrames in ORC format.

• Platform support: Windows operating system is not supported forto_orc() andread_orc() as of now.

Note: Theto_orc() method requires the pyarrow library (version >= 7.0.0). It is highly recommended to install thepyarrow library usingConda package manager to prevent compatibility issues. You can install it using the following command −

conda install pyarrow

Syntax

Following is the syntax of the Python Pandas to_orc() method −

DataFrame.to_orc(path=None, *, engine='pyarrow', index=None, engine_kwargs=None)

Parameters

The Python PandasDataFrame.to_orc() method accepts the below parameters −

• path: Specifies the file path or file-like object where the ORC file will be saved. IfNone, a bytes object is returned.

• engine: It specifies ORC library to use. Currently, only 'pyarrow' is supported.

• index: Whether to include the DataFrame's index in the output ORC file. If set to False exclude the index. If True, Includes the index in the output file. By default it is set to None, indexes other than RangeIndex will be included as columns.

• engine_kwargs: Additional keyword arguments passed topyarrow.orc.write_table().

Return Value

The PandasDataFrame.to_orc() method returnsbytes object, ifpath parameter is set toNone. Otherwise, returnsNone but saves the DataFrame as an ORC file at the specified path.

Errors

• The to_orc() method raises aNotImplementedError if the DataFrame columns contains unsupported dtypes, such as category, unsigned integers, interval, period, or sparse data.

• ValueError is raised if thepyarrow engine is not used.

Example: Saving a DataFrame as an ORC File

Here is a basic example demonstrating saving a Pandas DataFrame object into anORC file format using theDataFrame.to_orc() method.

import pandas as pd# Create a DataFramedf = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)})print("Original DataFrame:")print(df)# Save the DataFrame as an ORC filedf.to_orc("df_orc_file.orc")print("\nDataFrame is successfully saved as an ORC file.")

When we run above program, it produces following result −

Original DataFrame:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame is successfully saved as an ORC file.

If you visit the path where the ORC files are saved, you can observe the generated orc file.

Example: Save Pandas DataFrame to In-Memory orc

This example saves a Pandas DataFrame object into a in-memory ORC file using theDataFrame.to_orc() method.

import pandas as pdimport io# Create a DataFramedf = pd.DataFrame(data={"Col1": [1, 2], "Col2": [3.0, 4.0]})print("Original DataFrame:")print(df)# Save the DataFrame to an in-memory bufferbuffer = io.BytesIO()df.to_orc(buffer)# Read the ORC file from the bufferoutput = pd.read_orc(buffer)print("\nDataFrame saved as an in-memory ORC file:")print(output)

While executing the above code we get the following output −

Original DataFrame:

	Col_1	Col_2
0	1	3.0
1	2	4.0

Saved DataFrame as an in-memory orc file:

	Col_1	Col_2
0	1	3.0
1	2	4.0

Example: Saving ORC file Without Index

The ORC format does not support serializing custom indexes directly. To avoid the errors, you can reset the DataFrame index before saving it as an ORC file.

import pandas as pd# Create a DataFrame with an indexdf = pd.DataFrame({"Col1": [1, 2, 3], "Col2": ["a", "b", "c"]}, index=["r1", "r2", "r3"])print("Original DataFrame:")print(df)# Save the DataFrame without its indexdf.reset_index().to_orc("data_no_index.orc")# Read the fileoutput = pd.read_orc("data_no_index.orc")print("\nSaved DataFrame without index:")print(output)

Following is an output of the above code −

Original DataFrame:

	Col_1	Col_2
r1	1	a
r2	2	b
r3	3	c

Saved DataFrame without index:index

		Col_1	Col_2
0	r1	1	a
1	r2	2	b
2	r3	3	c