Tagging@TomNicholas@kylebarron in case they have some insight or opinion!

But, I'm concerned about having more async stuff in python, I have fought those async loops way more than I want to remember. For example, we would need to explore how well this interacts with multithreaded python code.

This is my exact worry as well.

A macro actually might be the best way to do this, because the inner async closure will be the same for both sync and async and the only thing that changes are the call signatures and the runtime wrapper. I'll look into that

the inner async closure will be the same for both sync and async and the only thing that changes are the call signatures and the runtime wrapper.

Exactly.

Maybe we can do a small proof of concept and see how it goes. The other concern with this is that we still need to create both python classes calling to the proper Rust object. I wonder if there are any tricks we play there without killing performance.

I don't think anybody will like "Classes with Async methods", or I don't at least (but it's great to keep it in the doc as an option).

FWIW this is my preferred option 😄. E.g. in obstore the same operations are synchronous withdef get_range or async withasync def get_range_async.

I don't think there should be two separate classes that represent the same underlying concept and data on the rust side. E.g. for obstore the act of creating an S3Store is totally indifferent to whether you plan to use it synchronously or asynchronously. And the underlying Rust data (Arc<dyn ObjectStore>) is the same whether or not you use async methods on it. So I think it would be more confusing to have two separate classes that do the same thing.

Even when you might want an async constructor, I prefer having a single class. E.g. inkylebarron/arro3#313 I'm updating myParquetFile API. But since the underlying data held byParquetFile is the same in a sync or async case, I prefer having alternate constructors:open andopen_async (or potentially both__enter__ and__aenter__ to use as a sync or async context manager).

I also preferfetch_config_async instead ofasync_fetch_config in your example so you get quicker narrowing of autocompletion in IDE type hinting.

Thanks@kylebarron . This gives me a new appreciation for that option.