Python Pandas read_orc() Method

---

---

Theread_orc() method in Pandas library allows you to load data stored in theORC format into a Pandas DataFrame. ORC (Optimized Row Columnar) is a binary columnar storage format designed for efficient data analysis, making reading and writing DataFrames highly efficient. It offers support for smallest, fastest columnar data storage and sharing across data analysis languages and this format is similar to theParquet Format.

Theread_orc() method supports reading ORC files from various storage back-ends, including local files, URLs, and cloud storage services.

Here are some key points to consider when using the read_orc() method −

• Installation Requirements: This format requires thepyarrow library for both reading and writing ORC files in Pandas.

• Timezone handling: Timezones in datetime columns are not preserved when saving DataFrames in ORC format.

• Platform support: This method is not supported on Windows operating system as of now.

Note: Before using theread_orc() method, you need to install the 'pyarrow' library and it is highly recommended to install it using theconda installer to avoid compatibility issues −

conda install pyarrow

In this tutorial, we will explore theread_orc() method from the Pandas library and how to use it for reading data stored in the ORC format.

Syntax

Following is the syntax of the Python Pandas read_orc() method −

pandas.read_orc(path, columns=None, dtype_backend=<no_default>, filesystem=None, **kwargs)

Parameters

The Python Pandasread_orc() method accepts the below parameters −

• path: This method accepts a string, path object, or file-like object that specifies the ORC file path. It can be a local file path or a remote URL supporting schemes like http, ftp, s3, and file. For local files, the expected format isfile://localhost/path/to/table.orc.

• columns: Specifies which columns to load from the ORC file. If not specified, all columns are loaded.

• dtype_backend: Defines the back-end data type applied to the resultant DataFrame ('numpy_nullable' or 'pyarrow').

• filesystem: Apyarrow orfsspec filesystem object for reading the parquet file. By defaultNone.

• **kwargs: Additional arguments passed toPyArrow engine.

Return Value

The Pandasread_orc() method returns a Pandas DataFrame containing the data from the ORC file.

Example: Reading an ORC File

Here is a basic example demonstrating loading a Pandas DataFrame object from an ORC file using the Pandasread_orc() method.

import pandas as pd# Create a DataFramedf = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)})print("Original DataFrame:")print(df)# Save the DataFrame as an ORC filedf.to_orc("df_orc_file.orc")# Load the DataFrame from the ORC fileresult = pd.read_orc("df_orc_file.orc")print("\nDataFrame loaded from ORC File:")print(result)

When we run above program, it produces following result −

Original DataFrame:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame Loaded from ORC File:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

Example: Reading Selected Columns from an ORC File

This example reads the Pandas DataFrame with the selected columns from an ORC file using the Pandasread_orc() method.

import pandas as pd# Create a DataFramedf = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)})print("Original DataFrame:")print(df)# Save the DataFrame as an ORC filedf.to_orc("df_orc_file.orc")# Read only specific columnsdf = pd.read_orc('df_orc_file.orc', columns=['Col_1'])print("\nDataFrame from ORC file with Selected Column:")print(df)

While executing the above code we get the following output −

Original DataFrame:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame from ORC file with Selected Column:

	Col_1
0	0
1	1
2	2
3	3
4	4

Example: Reading an In-Memory ORC File

This example demonstrates how to save a DataFrame as an ORC file in memory using an in-memory buffer and read it back into a DataFrame.

import pandas as pdimport io# Create a DataFramedf = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)})print("Original DataFrame:")print(df)# Save the DataFrame to an in-memory bufferbuffer = io.BytesIO()df.to_orc(buffer)# Read the ORC file from the bufferoutput = pd.read_orc(buffer)print("\nDataFrame saved as an in-memory ORC file:")print(output)

Following is an output of the above code −

Original DataFrame:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame saved as an in-memory ORC file:

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9