Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

fix: avoid blocking in download thread when using BQ Storage API#2034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@kien-truong
Copy link
Contributor

This prevents a deadlock between the main thead and download threads when the threadpool is shutdown prematurely.

Fixes#2032 🦕

@kien-truongkien-truong requested review froma team ascode ownersSeptember 28, 2024 03:14
@product-auto-labelproduct-auto-labelbot added size: sPull request size is small. api: bigqueryIssues related to the googleapis/python-bigquery API. labelsSep 28, 2024
@LinchinLinchin added the kokoro:force-runAdd this label to force Kokoro to re-run the tests. labelOct 3, 2024
@LinchinLinchin self-assigned thisOct 3, 2024
@yoshi-kokoroyoshi-kokoro removed the kokoro:force-runAdd this label to force Kokoro to re-run the tests. labelOct 3, 2024
@LinchinLinchin removed their assignmentOct 8, 2024
@kien-truongkien-truongforce-pushed thefix-bq-storage-client-deadlock branch from256b0f7 to3f8bd10CompareOctober 10, 2024 15:12
@kien-truong
Copy link
ContributorAuthor

Just dropping by to see if there's any concerns about this PR.

This prevents a deadlock between the main thead and download threadswhen the threadpool is shutdown prematurely.
@kien-truongkien-truongforce-pushed thefix-bq-storage-client-deadlock branch fromd3f2f1e to8eebb9dCompareNovember 3, 2024 03:37
@chalmerlowechalmerlowe added the kokoro:runAdd this label to force Kokoro to re-run the tests. labelNov 18, 2024
@yoshi-kokoroyoshi-kokoro removed the kokoro:runAdd this label to force Kokoro to re-run the tests. labelNov 18, 2024
@chalmerlowe
Copy link
Collaborator

I took a quick look.
Gonna run our CI/CD checks on this.
My first thought is that because we are adding a couple code pathscoverage is likely to fail.
Also curious why the pragma forqueue.full: we recently got bitten by having a code branch that was not tested and I am hesitant about adding more such paths to the code base without understanding for myself why it is necessary/desireable.

@kien-truong
Copy link
ContributorAuthor

Thequeue.Full code path is hard to test because it's non-deterministic: the queue can be full or not depending on the timing of consumers and producers.

So, I just put apragma there to ignore the coverage, similar to the existing code a few lines below.

exceptqueue.Empty:# pragma: NO COVER

tswast reacted with thumbs up emoji

@tswasttswast added the kokoro:runAdd this label to force Kokoro to re-run the tests. labelFeb 3, 2025
@kokoro-teamkokoro-team removed the kokoro:runAdd this label to force Kokoro to re-run the tests. labelFeb 3, 2025
@tswast
Copy link
Contributor

Looks like cover is passing.

@tswasttswast merged commit54c8d07 intogoogleapis:mainFeb 3, 2025
16 of 22 checks passed
@kien-truongkien-truong deleted the fix-bq-storage-client-deadlock branchFebruary 4, 2025 02:18
@tswast
Copy link
Contributor

tswast commentedFeb 4, 2025
edited
Loading

After sleeping on this, I do think it's important to try and cover the currently uncovered branch. I'll see what I can do in a follow-up PR today.

Edit: mailed#2127

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@tswasttswasttswast approved these changes

@leahecoleleahecoleAwaiting requested review from leahecoleleahecole was automatically assigned from googleapis/api-bigquery

Assignees

@chalmerlowechalmerlowe

Labels

api: bigqueryIssues related to the googleapis/python-bigquery API.size: sPull request size is small.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Deadlock due to race condition in download thread when using Bigquery Storage API

7 participants

@kien-truong@chalmerlowe@tswast@hongalex@Linchin@kokoro-team@yoshi-kokoro

[8]ページ先頭

©2009-2025 Movatter.jp