Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork18.5k
Closed
Description
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on thelatest version of pandas.
I have confirmed this bug exists on themain branch of pandas.
Reproducible Example
importpandasaspddf=pd.DataFrame( {"dates": [pd.to_datetime("2025-05-21 18:44:22"),pd.to_datetime("2025-05-21 19:12:42"), ],"tags": [12,45, ] },)df["dates"]=df["dates"].astype("datetime64[ms]")print(df.dtypes)print(df)df.to_hdf("dates.h5",key="dates")df2=pd.read_hdf("dates.h5",key="dates")print(df2)df2["corrected"]=df2["dates"].astype("i8").astype("datetime64[ms]")print(df2)
Issue Description
Dataframes containing dtype of "datetime64[ms]" seem to be correctly written in hdf format, but the readback is misinterpreted as “datetime64[ns]”.
The output of the code above is:
tags int64dtype: object dates tags0 2025-05-21 18:44:22 121 2025-05-21 19:12:42 45 dates tags0 1970-01-01 00:29:07.853062 121 1970-01-01 00:29:07.854762 45 dates tags corrected0 1970-01-01 00:29:07.853062 12 2025-05-21 18:44:221 1970-01-01 00:29:07.854762 45 2025-05-21 19:12:42```### Expected BehaviorCorrect dates when read back.### Installed Versions```INSTALLED VERSIONS------------------commit : d9cdd2ee5a58015ef6f4d15c7226110c9aab8140python : 3.11.9.final.0python-bits : 64OS : WindowsOS-release : 10Version : 10.0.22631machine : AMD64processor : Intel64 Family 6 Model 183 Stepping 1, GenuineIntelbyteorder : littleLC_ALL : NoneLANG : en_US.UTF-8LOCALE : de_DE.cp1252pandas : 2.2.2numpy : 1.26.4pytz : 2024.1dateutil : 2.9.0.post0setuptools : 80.8.0pip : 25.1.1Cython : 3.1.1pytest : 8.3.5hypothesis : 6.131.20sphinx : 8.2.3blosc : Nonefeather : Nonexlsxwriter : 3.2.3lxml.etree : 5.4.0html5lib : 1.1pymysql : Nonepsycopg2 : Nonejinja2 : 3.1.6IPython : 8.36.0pandas_datareader : Noneadbc-driver-postgresql: Noneadbc-driver-sqlite : Nonebs4 : 4.13.4bottleneck : 1.5.0dataframe-api-compat : Nonefastparquet : Nonefsspec : 2025.5.0gcsfs : Nonematplotlib : 3.8.4numba : 0.61.2numexpr : 2.10.2odfpy : Noneopenpyxl : 3.1.2pandas_gbq : Nonepyarrow : 17.0.0pyreadstat : Nonepython-calamine : Nonepyxlsb : Nones3fs : Nonescipy : 1.15.3sqlalchemy : 2.0.30tables : 3.10.2tabulate : Nonexarray : 2025.4.0xlrd : 2.0.1zstandard : 0.23.0tzdata : 2025.2qtpy : 2.4.3pyqt5 : None```