- Notifications
You must be signed in to change notification settings - Fork673
Description
Description of the problem, including code/CLI snippet
The artifact downloads support a streaming mode which wraps the iterator provided by therequests
library and allows the user to provide anaction
callable that is called with the chunks provided by the iterator as arguments.
I would like to be able to access theresponse.iter_content()
iterator directly.
Expected Behavior
My use case would be something like this:
iter_response=project.artifacts.download(ref_name="main",job="build",streamed=True,action='iterator')do_things_with_iterator(iter_response)
( In my real projectdo_things_with_iterator
is actually a fastapiStreamingResponse
which takes an iterator or generator. This would allow downloads of artifacts without first having to download the entire file server side and then forwarding it to the client)
Actual Behavior
As far as I can tell this is currently not possible.
I thought this would be relatively simple to implement, so I adapted theresponse_content
function inutils.py
to
defresponse_content(response:requests.Response,streamed:bool,action:Optional[Union[Callable,Literal["iterator"]]],chunk_size:int,)->Optional[Union[bytes,Iterator[Any]]]:ifstreamedisFalse:returnresponse.contentifactionisNone:action=_StdoutStream()ifaction=="iterator":returnresponse.iter_content(chunk_size=chunk_size)forchunkinresponse.iter_content(chunk_size=chunk_size):ifchunk:action(chunk)returnNone
But I ran into issues adapting the type annotations in other places, as some functions are not designed to handle iterators and will only handle bytes. I did not have a lot of time to spend on figuring this out though so if it seems worth the effort I can look into this further and create a pull request later.
Specifications
- python-gitlab version: 3.2.0
- API version you are using (v3/v4): v4
- Gitlab server version (or gitlab.com): gitlab.com