Movatterモバイル変換


[0]ホーム

URL:


homepage

Issue31945

This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title:Configurable blocksize in HTTP(S)Connection
Type:enhancementStage:resolved
Components:Library (Lib)Versions:Python 3.7
process
Status:closedResolution:fixed
Dependencies:Superseder:
Assigned To:Nosy List: brett.cannon, nirs, serhiy.storchaka, vstinner, yselivanov
Priority:normalKeywords:patch

Created on2017-11-04 22:32 bynirs, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.

Pull Requests
URLStatusLinkedEdit
PR 4279mergednirs,2017-11-04 22:34
Messages (5)
msg305571 -(view)Author: Nir Soffer (nirs)*Date: 2017-11-04 22:32
blocksize is hardcoded to 8192 in send() and _read_readable(), preventingefficient upload when using file-like body.Users of the module that are not interested in chunked encoding can rewritethe copy loop using HTTPConnection.send():   conn = HTTPSConnection(...)   conn.putrequest(...)   conn.putheader(...)   conn.endheaders()   while True:       chunk = file.read(512*1024)       if not chunk:          break       conn.send(chunk)But fixing send() to use a configurable blocksize seems more useful.Also, users of requests do not have access the underlying connection, sothey cannot use preferred buffer size.When reading from /dev/zero and uploading to server that drop the receiveddata, larger buffer size gives 3X more throughput *and* 1/3 of cpu time.With real storage and network, the effect will probably be much smaller.
msg305624 -(view)Author: Nir Soffer (nirs)*Date: 2017-11-06 09:57
When using highlevel request() api, users can control the block size bywrapping the file object with an iterator:    class FileIter:        def __init__(self, file, blocksize):            self.file = file            self.blocksize = blocksize        def __iter__(self):            while True:                datablock = self.file.read(self.blocksize)                if not datablock:                    break                yield datablockAdding configurable block size will avoid this workaround.
msg305678 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2017-11-06 21:16
New changesetad455cd9243319b896c86074ffeb3bf78a82f4ec by Victor Stinner (Nir Soffer) in branch 'master':bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)https://github.com/python/cpython/commit/ad455cd9243319b896c86074ffeb3bf78a82f4ec
msg305679 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2017-11-06 21:18
Thank you Nir Soffer for this nice enhancement. Sadly, since it's a new feature, it cannot be backport to Python 3.6 nor 2.7, since it's a new feature.
msg305680 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2017-11-06 21:21
The commit message contains much more information than the NEWS and What's New entries. Maybe the What's New entry can be completed, I don't know. In the meanwhile, I close the issue :-) Nir: Feel free to create a new PR if you want to complete the What's New entry ;-)
History
DateUserActionArgs
2022-04-11 14:58:54adminsetgithub: 76126
2017-11-06 21:21:57vstinnersetstatus: open -> closed
resolution: fixed
messages: +msg305680

stage: patch review -> resolved
2017-11-06 21:18:07vstinnersetmessages: +msg305679
2017-11-06 21:16:39vstinnersetmessages: +msg305678
2017-11-06 21:15:53vstinnersettype: enhancement
versions: - Python 2.7, Python 3.6, Python 3.8
2017-11-06 09:57:49nirssetmessages: +msg305624
2017-11-05 23:28:20martin.panterlinkissue21790 superseder
2017-11-04 22:34:44nirssetkeywords: +patch
stage: patch review
pull_requests: +pull_request4241
2017-11-04 22:32:30nirscreate
Supported byThe Python Software Foundation,
Powered byRoundup
Copyright © 1990-2022,Python Software Foundation
Legal Statements

[8]ページ先頭

©2009-2026 Movatter.jp