Commit8eebb9d

committed

fix: avoid blocking in download thread when using BQ Storage API

This prevents a deadlock between the main thead and download threadswhen the threadpool is shutdown prematurely.

1 parentfef8b88 commit8eebb9dCopy full SHA for 8eebb9d

File tree

-3

lines changed

-3

lines changed

Lines changed: 8 additions & 3 deletions

Original file line number	Diff line number	Diff line change
`@@ -796,10 +796,15 @@ def _download_table_bqstorage_stream(`
`796`	`796`	`rowstream=reader.rows(session)`
`797`	`797`
`798`	`798`	`forpageinrowstream.pages:`
`799`		`-ifdownload_state.done:`
`800`		`-return`
`801`	`799`	`item=page_to_item(page)`
`802`		`-worker_queue.put(item)`
	`800`	`+whileTrue:`
	`801`	`+ifdownload_state.done:`
	`802`	`+return`
	`803`	`+try:`
	`804`	`+worker_queue.put(item,timeout=_PROGRESS_INTERVAL)`
	`805`	`+break`
	`806`	`+exceptqueue.Full:# pragma: NO COVER`
	`807`	`+continue`
`803`	`808`
`804`	`809`
`805`	`810`	`def_nowait(futures):`

Comments

(0)