Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: emit a warning when using alist() method returns max#1875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
nejch merged 1 commit intomainfromjlvillal/list_warning
Apr 13, 2022

Conversation

@JohnVillalovos
Copy link
Member

@JohnVillalovosJohnVillalovos commentedFeb 3, 2022
edited
Loading

A common cause of issues filed and questions raised is that a user
will call alist() method and only get 20 items. As this is the
default maximum of items that will be returned from alist() method.

To help with this we now emit a warning when the result from a
list() method is greater-than or equal to 20 (or the specified
per_page value) and the user is not using eitherall=True,
all=False,as_list=False, orpage=X.

@codecov-commenter
Copy link

codecov-commenter commentedFeb 3, 2022
edited
Loading

Codecov Report

Merging#1875 (1339d64) intomain (5370979) willincrease coverage by0.00%.
The diff coverage is96.15%.

@@           Coverage Diff           @@##             main    #1875   +/-   ##=======================================  Coverage   92.56%   92.57%           =======================================  Files          78       78             Lines        4910     4930   +20     =======================================+ Hits         4545     4564   +19- Misses        365      366    +1
FlagCoverage Δ
cli_func_v481.52% <61.53%> (-0.07%)⬇️
py_func_v480.46% <96.15%> (+0.34%)⬆️
unit83.50% <92.30%> (+0.02%)⬆️

Flags with carried forward coverage won't be shown.Click here to find out more.

Impacted FilesCoverage Δ
gitlab/client.py90.76% <96.15%> (+0.19%)⬆️

@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 2 times, most recently fromd6ef9fe to9d1e6e8CompareFebruary 3, 2022 23:37
@JohnVillalovosJohnVillalovos marked this pull request as draftFebruary 3, 2022 23:38
@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 3 times, most recently from92da850 toc790f8cCompareFebruary 4, 2022 17:09
@JohnVillalovos
Copy link
MemberAuthor

JohnVillalovos commentedFeb 4, 2022
edited
Loading

Example program using this:

$ cat show-it.py#!/usr/bin/python3 -ttuimport gitlabgl = gitlab.Gitlab(url="https://gitlab.com")project = gl.projects.get("gitlab-org/gitlab")mrs = project.mergerequests.list(state="opened")

Run of program:

$ ./show-it.py..././show-it.py:7: UserWarning: Calling a `list()` method without specifying `all=True` or `as_list=False` will return a maximum of 20 items. Your query returned 20 of 1286 items. See https://python-gitlab.readthedocs.io/en/v3.1.1/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `page=1` or `get_all=False` to the `list()` call.  mrs = project.mergerequests.list(state="opened")

@JohnVillalovosJohnVillalovos marked this pull request as ready for reviewFebruary 4, 2022 17:14
@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 2 times, most recently from0223989 to63a6f1eCompareFebruary 4, 2022 22:52
@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 5 times, most recently fromec10a57 to9aa7a66CompareFebruary 5, 2022 17:52
@JohnVillalovosJohnVillalovos marked this pull request as draftFebruary 5, 2022 18:41
@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 2 times, most recently fromfb933c5 to1fc8aa8CompareFebruary 6, 2022 19:07
@JohnVillalovosJohnVillalovos marked this pull request as ready for reviewFebruary 7, 2022 16:52
@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 4 times, most recently from3bd6d8d to439e50bCompareFebruary 14, 2022 16:47
@JohnVillalovosJohnVillalovosforce-pushed thejlvillal/list_warning branch 2 times, most recently fromf99cf74 toac15b21CompareMarch 27, 2022 14:31
@nejch
Copy link
Member

Finally had a look around and here's some more valid use cases where we don't want to emit warnings on this - essentially whenever any kind of list filter is applied, I'd say. In those cases users mostly only care about the first item.

  1. Getting a user by username (can't use the GET endpoint):https://github.com/ansible-collections/community.general/blob/21ee4c84b7da0f5f540882b6d035c47da473db1f/plugins/modules/source_control/gitlab/gitlab_user.py#L521-L525
  2. Get group/project via loose name search:https://github.com/ansible-collections/community.general/blob/21ee4c84b7da0f5f540882b6d035c47da473db1f/plugins/modules/source_control/gitlab/gitlab_group_members.py#L180-L190
  3. Checking for existence of items:https://github.com/ansible-collections/community.general/blob/21ee4c84b7da0f5f540882b6d035c47da473db1f/plugins/modules/source_control/gitlab/gitlab_group.py#L282-L284

@JohnVillalovos
Copy link
MemberAuthor

JohnVillalovos commentedApr 11, 2022
edited
Loading

Answering without doing a lot of research on these:

Finally had a look around and here's some more valid use cases where we don't want to emit warnings on this - essentially whenever any kind of list filter is applied, I'd say. In those cases users mostly only care about the first item.

  1. Getting a user by username (can't use the GET endpoint):https://github.com/ansible-collections/community.general/blob/21ee4c84b7da0f5f540882b6d035c47da473db1f/plugins/modules/source_control/gitlab/gitlab_user.py#L521-L525

Warning will not be issued unless for some reason 20 items are returned by thelist() call. In that case it is likely highlighting a real error. I have submitted a PR that adds aall=True for this.

  1. Get group/project via loose name search:https://github.com/ansible-collections/community.general/blob/21ee4c84b7da0f5f540882b6d035c47da473db1f/plugins/modules/source_control/gitlab/gitlab_group_members.py#L180-L190

Warning will not be issued unless for some reason 20 items are returned by thelist() call. In that case it is likely highlighting a real error. I have submitted a PR that adds aall=True for this.

  1. Checking for existence of items:https://github.com/ansible-collections/community.general/blob/21ee4c84b7da0f5f540882b6d035c47da473db1f/plugins/modules/source_control/gitlab/gitlab_group.py#L282-L284

Yes a warning has a good chance of being produced for this one. I have submitted a PR that adds aall=False for this.

ansible-collections/community.general#4491

A common cause of issues filed and questions raised is that a userwill call a `list()` method and only get 20 items. As this is thedefault maximum of items that will be returned from a `list()` method.To help with this we now emit a warning when the result from a`list()` method is greater-than or equal to 20 (or the specified`per_page` value) and the user is not using either `all=True`,`all=False`, `as_list=False`, or `page=X`.
Copy link
Member

@nejchnejch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for this@JohnVillalovos! Took a while as I am really uneasy about eagerly sending warnings but this will definitely reduce the issues filed.

@nejchnejch merged commit4d6f125 intomainApr 13, 2022
@nejchnejch deleted the jlvillal/list_warning branchApril 13, 2022 05:50
@JohnVillalovos
Copy link
MemberAuthor

Thanks for this@JohnVillalovos! Took a while as I am really uneasy about eagerly sending warnings but this will definitely reduce the issues filed.

Thanks@nejch ! I'm interested to see feedback on it once it starts getting used by people. I know when I tried it on my code I found spots where I was like "whoops!".

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@nejchnejchnejch approved these changes

@max-wittigmax-wittigAwaiting requested review from max-wittig

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@JohnVillalovos@codecov-commenter@nejch

[8]ページ先頭

©2009-2025 Movatter.jp