@dbaxa Python3.13 testing can be included. As far as relaxing the numpy version we reverted it as it had compatibility issues with pandas -https://pandas.pydata.org/docs/whatsnew/v2.2.2.html#pandas-2-2-2-is-now-compatible-with-numpy-2-0

@jprakash-db can you provide more details about the compatibility issues? In our use case we didn't see any problems with using python 3.13 pandas 2.2.3 & numpy 2.2.3 when using a modified version ofdatabricks-sql-python locally.

As in

One major caveat is that arrays created with numpy 2.0's newStringDtype will convert toobject dtyped arrays upon :class:Series/:class:DataFrame creation. Full support for numpy 2.0's StringDtype is expected to land in pandas 3.0.

doesn't seem like an issue to me becauseStringDtype was introduced in numpy 2.0.0 and people apparently would need to explicitly use the new type (pandas-dev/pandas#58104 (comment) &aws/sagemaker-python-sdk#4882 (comment)).

Asdatabricks-sql-python doesn't directly importnumpy we shouldn't be specifying it as a dependency here.

In this code base we only rely onpandas functionality, we don't care how pandas implements that functionality under the hood.

Our contract is with the public API ofpandas and we trust thepandas devs to ensure their library works correctly. Ifpandas doesn't work with a modernnumpy it is for them to pinnumpy as it is their dependency.

pandas claims to supportnumpy>=1.26:
https://github.com/pandas-dev/pandas/blob/main/pyproject.toml#L30-L31

...so if that is not the case we should open an issue (or PR) in thepandas repo to apply the pin there.

Packaging bugs occur (all the time!) but if this pin here is to work around a mis-specified dependency thatpandas once had at one time, the cure is worse that the cold. Pinning to an oldnumpy means thatdatabricks-sql-connector can't be used in any modern analytics environment which isn't great for a package designed to support analytics.

Thanks for raising this :dhirschfeld, this makes sense to me. I am closing this in favour of#510