Python Pandas read_fwf() Method

Theread_fwf() method in Python's Pandas library is used to read data from files where the columns have fixed widths, meaning each column has a set size. This method is helpful for processing data files where columns are aligned with specific widths or fixed delimiters. Additionally, it allows you to optionally iterate or break up large files into smaller chunks, making it easier to process them.

Fixed-width file formats are commonly used in older systems or specific applications, where each piece of data starts and ends at specific positions. This method allows for reading such data efficiently into DataFrames for analysis. Theread_fwf() method is similar toread_csv(), but instead of reading delimited data, it works with files where each column has a fixed width.

Syntax

The syntax of the read_fwf() method is as follows −

pandas.read_fwf(filepath_or_buffer, *, colspecs='infer', widths=None, infer_nrows=100, dtype_backend=<no_default>, iterator=False, chunksize=None, **kwds)

Parameters

The Python Pandas read_fwf() method accepts the below parameters −

• filepath_or_buffer: Specifies the file path or file-like object. It can be a string, path object, or URL (e.g., http, ftp, s3, or file).

• colspecs: This is a list of tuples (pairs) that specify the start and end positions of each column. For example, (0, 5) means the column starts at position 0 and ends at position 5 (excluding 5). If you don't specify this, the method tries to figure out the column positions from the first 100 rows by default.

• widths: Specifies field widths as a list of integers. Use this instead ofcolspecs for contiguous intervals.

• infer_nrows: Number of rows considered by the parser for detectingcolspecs when set to 'infer'.

• dtype_backend: Backend for resulting DataFrame data types. Default isnumpy_nullable.

• iterator: If True, returns an iterator for reading the file in chunks.

• chunksize: Number of lines to read per chunk for iteration.

• **kwargs: Additional optional keyword arguments passed toTextFileReader.

Return Value

The Pandasread_fwf() method returns a DataFrame or a TextFileReader ifiterator=True.

Example: Reading a Fixed-Width File

Here is a basic example demonstrating reading a fixed-width file using the pandasread_fwf() method with the default settings to infer column specifications automatically from the file.

import pandas as pd# Sample fixed-width data saved to a filedata = """Name   Age   City   SalaryTom   28   Toronto   20000Lee   32   HongKong   3000Steven   43   Bay Area   8300Ram   38   Hyderabad   3900"""# Writing data to a filewith open("sample_fwf.txt", "w") as file:    file.write(data)# Reading the fixed-width filedf = pd.read_fwf("sample_fwf.txt")print("DataFrame from fixed-width file:")print(df)

When we run above program, it produces following result −

DataFrame from fixed-width file:

Example: Reading Fixed-Width File with Column Specifications

The following example demonstrates using theread_fwf() method with thecolspecs parameter to define custom start and end intervals for columns.

import pandas as pd# Sample fixed-width data saved to a filedata = """Name   Age   City   SalaryTom   28   Toronto   20000Lee   32   HongKong   3000Steven   43   Bay Area   8300Ram   38   Hyderabad   3900"""# Writing data to a filewith open("sample_fwf.txt", "w") as file:    file.write(data)# Specifying column intervalscolspecs = [(0, 6), (6, 11), (11, 20), (20, 26)]# Reading the fixed-width filedf = pd.read_fwf("sample_fwf.txt", colspecs=colspecs)print("DataFrame from fixed-width file with specified column intervals:")print(df)

Following is an output of the above code −

DataFrame from fixed-width file with specified column intervals:

Example: Specifying Column Widths While Reading Fixed-Width File

You can also manually specify column widths using thewidths parameter of theread_fwf() method instead of using thecolspecs parameter. The following example demonstrates the same.

import pandas as pd# Sample fixed-width data saved to a filedata = """Name   Age   City   SalaryTom   28   Toronto   20000Lee   32   HongKong   3000Steven   43   Bay Area   8300Ram   38   Hyderabad   3900"""# Writing data to a filewith open("sample_fwf.txt", "w") as file:    file.write(data)# Reading the file with specified widthsdf = pd.read_fwf("sample_fwf.txt", widths=[6, 5, 9, 3])print("DataFrame from fixed-width file with specified widths:")print(df)

While executing the above code we get the following output −

DataFrame from fixed-width file with specified widths:

Example: Skipping Rows While Reading Fixed-Width Files

Theread_fwf() method allows you to skip the rows while reading using theskiprows parameter. This example demonstrates skipping the first two rows from the fixed-width text file.

import pandas as pd# Sample fixed-width data saved to a filedata = """Name   Age   City   SalaryTom   28   Toronto   20000Lee   32   HongKong   3000Steven   43   Bay Area   8300Ram   38   Hyderabad   3900"""# Writing data to a filewith open("sample_fwf.txt", "w") as file:    file.write(data)# Reading the file, skipping the first three rowsdf = pd.read_fwf("sample_fwf.txt", skiprows=3)print("DataFrame from fixed-width file by skipping the first three rows:")print(df)

Following is an output of the above code −

DataFrame from fixed-width file by skipping the first two rows:

Example: Iterating through Fixed-Width File Chunk by Chunk

By specifying achunksize parameter of theread_fwf() method, you can get an iterable object of typeTextFileReader for iterating the data of a fixed-width file chunk by chunk.

import pandas as pd# Sample fixed-width data saved to a filedata = """Name   Age   City   SalaryTom   28   Toronto   20000Lee   32   HongKong   3000Steven   43   Bay Area   8300Ram   38   Hyderabad   3900"""# Writing data to a filewith open("sample_fwf.txt", "w") as file:    file.write(data)    # Reading the file in chunkschunk_size = 1chunks = pd.read_fwf("sample_fwf.txt", chunksize=chunk_size)print("Iterating through Fixed-Width File Chunk by Chunk:")for chunk in chunks:    print(chunk)

When we run above program, it produces following result −

Iterating through Fixed-Width File Chunk by Chunk: