
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2017-11-04 22:32 bynirs, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 4279 | merged | nirs,2017-11-04 22:34 | |
| Messages (5) | |||
|---|---|---|---|
| msg305571 -(view) | Author: Nir Soffer (nirs)* | Date: 2017-11-04 22:32 | |
blocksize is hardcoded to 8192 in send() and _read_readable(), preventingefficient upload when using file-like body.Users of the module that are not interested in chunked encoding can rewritethe copy loop using HTTPConnection.send(): conn = HTTPSConnection(...) conn.putrequest(...) conn.putheader(...) conn.endheaders() while True: chunk = file.read(512*1024) if not chunk: break conn.send(chunk)But fixing send() to use a configurable blocksize seems more useful.Also, users of requests do not have access the underlying connection, sothey cannot use preferred buffer size.When reading from /dev/zero and uploading to server that drop the receiveddata, larger buffer size gives 3X more throughput *and* 1/3 of cpu time.With real storage and network, the effect will probably be much smaller. | |||
| msg305624 -(view) | Author: Nir Soffer (nirs)* | Date: 2017-11-06 09:57 | |
When using highlevel request() api, users can control the block size bywrapping the file object with an iterator: class FileIter: def __init__(self, file, blocksize): self.file = file self.blocksize = blocksize def __iter__(self): while True: datablock = self.file.read(self.blocksize) if not datablock: break yield datablockAdding configurable block size will avoid this workaround. | |||
| msg305678 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-11-06 21:16 | |
New changesetad455cd9243319b896c86074ffeb3bf78a82f4ec by Victor Stinner (Nir Soffer) in branch 'master':bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)https://github.com/python/cpython/commit/ad455cd9243319b896c86074ffeb3bf78a82f4ec | |||
| msg305679 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-11-06 21:18 | |
Thank you Nir Soffer for this nice enhancement. Sadly, since it's a new feature, it cannot be backport to Python 3.6 nor 2.7, since it's a new feature. | |||
| msg305680 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2017-11-06 21:21 | |
The commit message contains much more information than the NEWS and What's New entries. Maybe the What's New entry can be completed, I don't know. In the meanwhile, I close the issue :-) Nir: Feel free to create a new PR if you want to complete the What's New entry ;-) | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:54 | admin | set | github: 76126 |
| 2017-11-06 21:21:57 | vstinner | set | status: open -> closed resolution: fixed messages: +msg305680 stage: patch review -> resolved |
| 2017-11-06 21:18:07 | vstinner | set | messages: +msg305679 |
| 2017-11-06 21:16:39 | vstinner | set | messages: +msg305678 |
| 2017-11-06 21:15:53 | vstinner | set | type: enhancement versions: - Python 2.7, Python 3.6, Python 3.8 |
| 2017-11-06 09:57:49 | nirs | set | messages: +msg305624 |
| 2017-11-05 23:28:20 | martin.panter | link | issue21790 superseder |
| 2017-11-04 22:34:44 | nirs | set | keywords: +patch stage: patch review pull_requests: +pull_request4241 |
| 2017-11-04 22:32:30 | nirs | create | |