- Notifications
You must be signed in to change notification settings - Fork22
fix: permission error withpb.load_dataset() for duckdb on Windows#245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
matt-humphrey commentedJul 17, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
My bad, I should have run all tests before pushing changes. From what I gather, when pb.load_dataset() is used with duckdb, it's returning a live connection to the Ibis table. Hence a bunch of these tests failed, because I explicitly closed the connection. The problem appears to be that Windows runs into permission errors due to maintaining that live connection beyond the function where that connection is defined. I'm not sure what an optimal solution is here - possibly converting the file into memory, and then closing the connection? [EDIT] Upon further investigation, part of the problem likely has to do with the test fixtures also returning live connections, for example: @pytest.fixturedeftbl_dates_times_text_duckdb():file_path=pathlib.Path.cwd()/"tests"/"tbl_files"/"tbl_dates_times_text.ddb"withtempfile.TemporaryDirectory()astmp:fpath:Path=Path(tmp)/"tbl_dates_times_text.ddb"shutil.copy(file_path,fpath)returnibis.connect(f"duckdb://{fpath!s}").table("tbl_dates_times_text") Part of the solution might be changing these fixtures also. |
gregorywaynepower commentedSep 22, 2025
@matt-humphrey Are you saying that these tests need to be edited as well? |
matt-humphrey commentedOct 7, 2025
@gregorywaynepower yes, it's been a while so I can revise and identify exactly which tests need to be edited, but running on Windows I was running into problems with most tests with duckdb. |
Summary
There was an error with loading temporary file for duckdb on Windows.
I changed the code to explicitly close the Ibis connection, and that seems to have resolved the problem.
Related GitHub Issues and PRs
pb.load_data_set()with duckdb on Windows #244