Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat(client): introduceiterator=True and deprecateas_list=False inlist()#2032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
nejch merged 1 commit intomainfromjlvillal/i_hate_as_list
May 29, 2022
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletionsdocs/api-usage.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -93,13 +93,13 @@ Examples:
.. code-block:: python

# list all the projects
projects = gl.projects.list(as_list=False)
projects = gl.projects.list(iterator=True)
for project in projects:
print(project)

# get the group with id == 2
group = gl.groups.get(2)
for project in group.projects.list(as_list=False):
for project in group.projects.list(iterator=True):
print(project)

# create a new user
Expand All@@ -109,7 +109,7 @@ Examples:

.. warning::
Calling ``list()`` without any arguments will by default not return the complete list
of items. Use either the ``all=True`` or ``as_list=False`` parameters to get all the
of items. Use either the ``all=True`` or ``iterator=True`` parameters to get all the
items when using listing methods. See the :ref:`pagination` section for more
information.

Expand DownExpand Up@@ -156,9 +156,9 @@ conflict with python or python-gitlab when using them as kwargs:

.. code-block:: python

gl.user_activities.list(from='2019-01-01',as_list=False) ## invalid
gl.user_activities.list(from='2019-01-01',iterator=True) ## invalid

gl.user_activities.list(query_parameters={'from': '2019-01-01'},as_list=False) # OK
gl.user_activities.list(query_parameters={'from': '2019-01-01'},iterator=True) # OK

Gitlab Objects
==============
Expand DownExpand Up@@ -282,13 +282,13 @@ order options. At the time of writing, only ``order_by="id"`` works.
Reference:
https://docs.gitlab.com/ce/api/README.html#keyset-based-pagination

``list()`` methods can also return a generator object which will handlethe
next calls to the API when required. This is the recommended way to iterate
through a large number of items:
``list()`` methods can also return a generator object, by passingthe argument
``iterator=True``, which will handle thenext calls to the API when required. This
is the recommended way to iteratethrough a large number of items:

.. code-block:: python

items = gl.groups.list(as_list=False)
items = gl.groups.list(iterator=True)
for item in items:
print(item.attributes)

Expand All@@ -310,6 +310,10 @@ The generator exposes extra listing information as received from the server:
For more information see:
https://docs.gitlab.com/ee/user/gitlab_com/index.html#pagination-response-headers

.. note::
Prior to python-gitlab 3.6.0 the argument ``as_list`` was used instead of
``iterator``. ``as_list=False`` is the equivalent of ``iterator=True``.

Sudo
====

Expand Down
4 changes: 2 additions & 2 deletionsdocs/gl_objects/search.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -63,13 +63,13 @@ The ``search()`` methods implement the pagination support::

# get a generator that will automatically make required API calls for
# pagination
for item in gl.search(gitlab.const.SEARCH_SCOPE_ISSUES, search_str,as_list=False):
for item in gl.search(gitlab.const.SEARCH_SCOPE_ISSUES, search_str,iterator=True):
do_something(item)

The search API doesn't return objects, but dicts. If you need to act on
objects, you need to create them explicitly::

for item in gl.search(gitlab.const.SEARCH_SCOPE_ISSUES, search_str,as_list=False):
for item in gl.search(gitlab.const.SEARCH_SCOPE_ISSUES, search_str,iterator=True):
issue_project = gl.projects.get(item['project_id'], lazy=True)
issue = issue_project.issues.get(item['iid'])
issue.state = 'closed'
Expand Down
2 changes: 1 addition & 1 deletiondocs/gl_objects/users.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -413,4 +413,4 @@ Get the users activities::

activities = gl.user_activities.list(
query_parameters={'from': '2018-07-01'},
all=True,as_list=False)
all=True,iterator=True)
31 changes: 24 additions & 7 deletionsgitlab/client.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -807,7 +807,9 @@ def http_list(
self,
path: str,
query_data: Optional[Dict[str, Any]] = None,
as_list: Optional[bool] = None,
*,
as_list: Optional[bool] = None, # Deprecated in favor of `iterator`
iterator: Optional[bool] = None,
**kwargs: Any,
) -> Union["GitlabList", List[Dict[str, Any]]]:
"""Make a GET request to the Gitlab server for list-oriented queries.
Expand All@@ -816,12 +818,13 @@ def http_list(
path: Path or full URL to query ('/projects' or
'http://whatever/v4/api/projects')
query_data: Data to send as query parameters
iterator: Indicate if should return a generator (True)
**kwargs: Extra options to send to the server (e.g. sudo, page,
per_page)

Returns:
A list of the objects returned by the server. If `as_list` is
False and no pagination-related arguments (`page`, `per_page`,
A list of the objects returned by the server. If `iterator` is
True and no pagination-related arguments (`page`, `per_page`,
`all`) are defined then a GitlabList object (generator) is returned
instead. This object will make API calls when needed to fetch the
next items from the server.
Expand All@@ -832,15 +835,29 @@ def http_list(
"""
query_data = query_data or {}

# In case we want to change the default behavior at some point
as_list = True if as_list is None else as_list
# Don't allow both `as_list` and `iterator` to be set.
if as_list is not None and iterator is not None:
raise ValueError(
"Only one of `as_list` or `iterator` can be used. "
"Use `iterator` instead of `as_list`. `as_list` is deprecated."
)

if as_list is not None:
iterator = not as_list
utils.warn(
message=(
f"`as_list={as_list}` is deprecated and will be removed in a "
f"future version. Use `iterator={iterator}` instead."
),
category=DeprecationWarning,
)

get_all = kwargs.pop("all", None)
url = self._build_url(path)

page = kwargs.get("page")

ifas_list is False:
ifiterator:
# Generator requested
return GitlabList(self, url, query_data, **kwargs)

Expand DownExpand Up@@ -879,7 +896,7 @@ def should_emit_warning() -> bool:
utils.warn(
message=(
f"Calling a `list()` method without specifying `all=True` or "
f"`as_list=False` will return a maximum of {gl_list.per_page} items. "
f"`iterator=True` will return a maximum of {gl_list.per_page} items. "
f"Your query returned {len(items)} of {total_items} items. See "
f"{_PAGINATION_URL} for more details. If this was done intentionally, "
f"then this warning can be supressed by adding the argument "
Expand Down
6 changes: 2 additions & 4 deletionsgitlab/mixins.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -201,12 +201,12 @@ def list(self, **kwargs: Any) -> Union[base.RESTObjectList, List[base.RESTObject
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set toFalse and no pagination option is
iterator: If set toTrue and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Returns:
The list of objects, or a generator if `as_list` isFalse
The list of objects, or a generator if `iterator` isTrue

Raises:
GitlabAuthenticationError: If authentication is not correct
Expand DownExpand Up@@ -846,8 +846,6 @@ def participants(self, **kwargs: Any) -> Dict[str, Any]:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand Down
4 changes: 2 additions & 2 deletionsgitlab/v4/objects/ldap.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -26,12 +26,12 @@ def list(self, **kwargs: Any) -> Union[List[LDAPGroup], RESTObjectList]:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set toFalse and no pagination option is
iterator: If set toTrue and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Returns:
The list of objects, or a generator if `as_list` isFalse
The list of objects, or a generator if `iterator` isTrue

Raises:
GitlabAuthenticationError: If authentication is not correct
Expand Down
8 changes: 2 additions & 6 deletionsgitlab/v4/objects/merge_requests.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -199,8 +199,6 @@ def closes_issues(self, **kwargs: Any) -> RESTObjectList:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand All@@ -211,7 +209,7 @@ def closes_issues(self, **kwargs: Any) -> RESTObjectList:
List of issues
"""
path = f"{self.manager.path}/{self.encoded_id}/closes_issues"
data_list = self.manager.gitlab.http_list(path,as_list=False, **kwargs)
data_list = self.manager.gitlab.http_list(path,iterator=True, **kwargs)
if TYPE_CHECKING:
assert isinstance(data_list, gitlab.GitlabList)
manager = ProjectIssueManager(self.manager.gitlab, parent=self.manager._parent)
Expand All@@ -226,8 +224,6 @@ def commits(self, **kwargs: Any) -> RESTObjectList:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand All@@ -239,7 +235,7 @@ def commits(self, **kwargs: Any) -> RESTObjectList:
"""

path = f"{self.manager.path}/{self.encoded_id}/commits"
data_list = self.manager.gitlab.http_list(path,as_list=False, **kwargs)
data_list = self.manager.gitlab.http_list(path,iterator=True, **kwargs)
if TYPE_CHECKING:
assert isinstance(data_list, gitlab.GitlabList)
manager = ProjectCommitManager(self.manager.gitlab, parent=self.manager._parent)
Expand Down
16 changes: 4 additions & 12 deletionsgitlab/v4/objects/milestones.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,8 +33,6 @@ def issues(self, **kwargs: Any) -> RESTObjectList:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand All@@ -46,7 +44,7 @@ def issues(self, **kwargs: Any) -> RESTObjectList:
"""

path = f"{self.manager.path}/{self.encoded_id}/issues"
data_list = self.manager.gitlab.http_list(path,as_list=False, **kwargs)
data_list = self.manager.gitlab.http_list(path,iterator=True, **kwargs)
if TYPE_CHECKING:
assert isinstance(data_list, RESTObjectList)
manager = GroupIssueManager(self.manager.gitlab, parent=self.manager._parent)
Expand All@@ -62,8 +60,6 @@ def merge_requests(self, **kwargs: Any) -> RESTObjectList:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand All@@ -74,7 +70,7 @@ def merge_requests(self, **kwargs: Any) -> RESTObjectList:
The list of merge requests
"""
path = f"{self.manager.path}/{self.encoded_id}/merge_requests"
data_list = self.manager.gitlab.http_list(path,as_list=False, **kwargs)
data_list = self.manager.gitlab.http_list(path,iterator=True, **kwargs)
if TYPE_CHECKING:
assert isinstance(data_list, RESTObjectList)
manager = GroupIssueManager(self.manager.gitlab, parent=self.manager._parent)
Expand DownExpand Up@@ -114,8 +110,6 @@ def issues(self, **kwargs: Any) -> RESTObjectList:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand All@@ -127,7 +121,7 @@ def issues(self, **kwargs: Any) -> RESTObjectList:
"""

path = f"{self.manager.path}/{self.encoded_id}/issues"
data_list = self.manager.gitlab.http_list(path,as_list=False, **kwargs)
data_list = self.manager.gitlab.http_list(path,iterator=True, **kwargs)
if TYPE_CHECKING:
assert isinstance(data_list, RESTObjectList)
manager = ProjectIssueManager(self.manager.gitlab, parent=self.manager._parent)
Expand All@@ -143,8 +137,6 @@ def merge_requests(self, **kwargs: Any) -> RESTObjectList:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set to False and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Raises:
Expand All@@ -155,7 +147,7 @@ def merge_requests(self, **kwargs: Any) -> RESTObjectList:
The list of merge requests
"""
path = f"{self.manager.path}/{self.encoded_id}/merge_requests"
data_list = self.manager.gitlab.http_list(path,as_list=False, **kwargs)
data_list = self.manager.gitlab.http_list(path,iterator=True, **kwargs)
if TYPE_CHECKING:
assert isinstance(data_list, RESTObjectList)
manager = ProjectMergeRequestManager(
Expand Down
4 changes: 2 additions & 2 deletionsgitlab/v4/objects/repositories.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -60,7 +60,7 @@ def repository_tree(
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set toFalse and no pagination option is
iterator: If set toTrue and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Expand DownExpand Up@@ -172,7 +172,7 @@ def repository_contributors(
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set toFalse and no pagination option is
iterator: If set toTrue and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Expand Down
2 changes: 1 addition & 1 deletiongitlab/v4/objects/runners.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -81,7 +81,7 @@ def all(self, scope: Optional[str] = None, **kwargs: Any) -> List[Runner]:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set toFalse and no pagination option is
iterator: If set toTrue and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Expand Down
4 changes: 2 additions & 2 deletionsgitlab/v4/objects/users.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -542,12 +542,12 @@ def list(self, **kwargs: Any) -> Union[RESTObjectList, List[RESTObject]]:
all: If True, return all the items, without pagination
per_page: Number of items to retrieve per request
page: ID of the page to return (starts with page 1)
as_list: If set toFalse and no pagination option is
iterator: If set toTrue and no pagination option is
defined, return a generator instead of a list
**kwargs: Extra options to send to the server (e.g. sudo)

Returns:
The list of objects, or a generator if `as_list` isFalse
The list of objects, or a generator if `iterator` isTrue

Raises:
GitlabAuthenticationError: If authentication is not correct
Expand Down
17 changes: 14 additions & 3 deletionstests/functional/api/test_gitlab.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -220,9 +220,20 @@ def test_list_all_true_nowarning(gl):
assert len(items) > 20


deftest_list_as_list_false_nowarning(gl):
"""Using `as_list=False` will disable the warning"""
deftest_list_iterator_true_nowarning(gl):
"""Using `iterator=True` will disable the warning"""
with warnings.catch_warnings(record=True) as caught_warnings:
items = gl.gitlabciymls.list(as_list=False)
items = gl.gitlabciymls.list(iterator=True)
assert len(caught_warnings) == 0
assert len(list(items)) > 20


def test_list_as_list_false_warnings(gl):
"""Using `as_list=False` will disable the UserWarning but cause a
DeprecationWarning"""
with warnings.catch_warnings(record=True) as caught_warnings:
items = gl.gitlabciymls.list(as_list=False)
assert len(caught_warnings) == 1
for warning in caught_warnings:
assert isinstance(warning.message, DeprecationWarning)
assert len(list(items)) > 20
2 changes: 1 addition & 1 deletiontests/functional/api/test_projects.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,7 +15,7 @@ def test_create_project(gl, user):
sudo_project = gl.projects.create({"name": "sudo_project"}, sudo=user.id)

created = gl.projects.list()
created_gen = gl.projects.list(as_list=False)
created_gen = gl.projects.list(iterator=True)
owned = gl.projects.list(owned=True)

assert admin_project in created and sudo_project in created
Expand Down
4 changes: 2 additions & 2 deletionstests/unit/mixins/test_mixin_methods.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -107,7 +107,7 @@ class M(ListMixin, FakeManager):

# test RESTObjectList
mgr = M(gl)
obj_list = mgr.list(as_list=False)
obj_list = mgr.list(iterator=True)
assert isinstance(obj_list, base.RESTObjectList)
for obj in obj_list:
assert isinstance(obj, FakeObject)
Expand DownExpand Up@@ -138,7 +138,7 @@ class M(ListMixin, FakeManager):
)

mgr = M(gl)
obj_list = mgr.list(path="/others",as_list=False)
obj_list = mgr.list(path="/others",iterator=True)
assert isinstance(obj_list, base.RESTObjectList)
obj = obj_list.next()
assert obj.id == 42
Expand Down
Loading

[8]ページ先頭

©2009-2025 Movatter.jp