Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

BUG: Decimal and float-to-int conversion issues with pyarrow ≥18.0.0 in parquet and Arrow dtype tests #61464

Open
Labels
BugNeeds TriageIssue that has not been reviewed by a pandas team member
@bhavya2109sharma

Description

@bhavya2109sharma

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on thelatest version of pandas.

  • I have confirmed this bug exists on themain branch of pandas.

Reproducible Example

Issue1importpyarrowaspaarray=pa.array([1.5,2.5],type=pa.float64())array.to_pandas(types_mapper={pa.float64():pa.int64()}.get)ArrowInvalid:Floatvalue1.5wastruncatedconvertingtoint64Issue2importpandasaspdimportpyarrowaspafromdecimalimportDecimaldf=pd.DataFrame({"a": [Decimal("123.00")]},dtype="string[pyarrow]")df.to_parquet("decimal.pq",schema=pa.schema([("a",pa.decimal128(5))]))result=pd.read_parquet("decimal.pq")expected=pd.DataFrame({"a": ["123"]},dtype="string[python]")pd.testing.assert_frame_equal(result,expected)AssertionError:AttributesofDataFrame.iloc[:,0] (columnname="a")aredifferentAttribute"dtype"aredifferent[left]:object[right]:string[python]

Issue Description

Two issues have been observed when using pandas 2.2.3 with pyarrow >= 18.0.0:

  • Test cases Failing : pandas/tests/extension/test_arrow.py::test_from_arrow_respecting_given_dtype_unsafe and pandas/tests/io/test_parquet.py::TestParquetPyArrow::test_roundtrip_decimal

  • Stricter float-to-int casting causes ArrowInvalid in tests like test_from_arrow_respecting_given_dtype_unsafe.

  • Decimal roundtrip mismatch: test_roundtrip_decimal fails due to dtype mismatches (object vs. string[python]) when reading back a decimal column written with a specified pyarrow schema.

These issues were not present with pyarrow==17.x.

Expected Behavior

  • Float to int casting should either handle truncation more gracefully (as in older versions) or tests should be updated to skip/adjust.

  • Decimal roundtrips to parquet should maintain the same pandas dtype or document clearly if type coercion is expected.

Installed Versions

python : 3.11.11
pandas : 2.2.3
pyarrow : 19.0.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugNeeds TriageIssue that has not been reviewed by a pandas team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions


      [8]ページ先頭

      ©2009-2025 Movatter.jp