
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2014-05-08 19:13 byneologix, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| socket_listen.diff | neologix,2014-05-08 19:13 | review | ||
| socket_listen-1.diff | neologix,2014-05-08 23:01 | review | ||
| Messages (11) | |||
|---|---|---|---|
| msg218121 -(view) | Author: Charles-François Natali (neologix)*![]() | Date: 2014-05-08 19:13 | |
Having to pass an explicit backlog value to listen() is a pain: most people don't know which value to pass, and often end up using a value too small which can lead to connections being rejected.For example, if you search throughout the standard library, you'll see many different hard-coded values.The patch attached uses a default value of SOMAXCONN for socket.listen(): it should give a good default behavior (that's what golang does for example: in fact they go even further by checking the runtime value from net.core.somaxconn).A second step would be to update the codebase to remove explicit backlogs (unless specific case). | |||
| msg218127 -(view) | Author: Antoine Pitrou (pitrou)*![]() | Date: 2014-05-08 22:12 | |
Is there a risk of SOMAXCONN being huge and therefore allocating a large amount of resources? | |||
| msg218135 -(view) | Author: Charles-François Natali (neologix)*![]() | Date: 2014-05-08 23:02 | |
> Is there a risk of SOMAXCONN being huge and therefore allocating a large amount of resources?On a sensible operating system, no, but better safe than sorry: thepatch attached caps the value to 128 (a common SOMAXCONN value).It should be high enough to avoid connection drops in commonworkloads, and still guard against non-sensical SOMAXCONN values. | |||
| msg218278 -(view) | Author: Charles-François Natali (neologix)*![]() | Date: 2014-05-11 15:36 | |
Any objection to the last version?If not, I'll commit it within a few days. | |||
| msg218279 -(view) | Author: Antoine Pitrou (pitrou)*![]() | Date: 2014-05-11 15:37 | |
Looks good to me! | |||
| msg218369 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2014-05-12 21:47 | |
> Py_MIN(SOMAXCONN, 128)On Windows, it's not the best choice to use this hardcoded limit. socket.SOMAXCONN is 2^31-1.listen() documentation says that Windows chooses a reasonable backlog value for you if you pass SOMAXCONN:http://msdn.microsoft.com/en-us/library/windows/desktop/ms739168%28v=vs.85%29.aspxYou should maybe use SOMAXCONN by default on Windows, and Py_MIN(SOMAXCONN, 128) by default on other platforms (UNIX). | |||
| msg218374 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2014-05-12 22:18 | |
An article suggests to use max(1024, socket.SOMAXCONN) (to "listen() backlog as large as possible") instead of socket.SOMAXCONN because the OS maximum can be larger than SOMAXCONN (and that it's not possible in Python to get the OS value):http://utcc.utoronto.ca/~cks/space/blog/python/AvoidSOMAXCONN?showcomments#commentsThe following article tries to explain why the default limit is 128 on Linux:https://derrickpetzold.com/p/somaxconn/Article giving the value chosen by Windows when SOMAXCONN is passed: it depends on the Windows version (between 5 and 50, hard limit of 200 on Windows 7):http://stackoverflow.com/questions/4709756/listen-maximum-queue-size-per-windows-version | |||
| msg218401 -(view) | Author: Charles-François Natali (neologix)*![]() | Date: 2014-05-13 04:50 | |
>> Py_MIN(SOMAXCONN, 128)>> On Windows, it's not the best choice to use this hardcoded limit. socket.SOMAXCONN is 2^31-1.I don't see what this would bring: 128 *is* a reasonable limit.> listen() documentation says that Windows chooses a reasonable backlog value for you if you pass SOMAXCONN:>http://msdn.microsoft.com/en-us/library/windows/desktop/ms739168%28v=vs.85%29.aspx>> You should maybe use SOMAXCONN by default on Windows, and Py_MIN(SOMAXCONN, 128) by default on other platforms (UNIX).Trying to come up with a good heuristic is a lost battle, since itdepends not only on the OS but also the application. The goal is tohave a default value that works, is large enough to avoid connectiondrops in common use cases, and not too large to avoid resourceconsumption.> An article suggests to use max(1024, socket.SOMAXCONN) (to "listen() backlog as large as possible")> instead of socket.SOMAXCONN because the OS maximum can be larger than SOMAXCONN (and that> it's not possible in Python to get the OS value):In theory we could use net.core.somaxconn sysctl & Co (that's what godoes), but IMO it's overkill. | |||
| msg218411 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2014-05-13 08:11 | |
Hum ok, thanks for your explanation Charles-François. socket_listen-1.diff looks good to me. | |||
| msg218910 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2014-05-22 18:48 | |
New changeset09371221e59d by Charles-François Natali in branch 'default':Issue#21455: Add a default backlog to socket.listen().http://hg.python.org/cpython/rev/09371221e59d | |||
| msg218919 -(view) | Author: Charles-François Natali (neologix)*![]() | Date: 2014-05-22 20:45 | |
Committed, thanks! | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:03 | admin | set | github: 65654 |
| 2014-05-22 20:45:13 | neologix | set | status: open -> closed resolution: fixed messages: +msg218919 stage: patch review -> resolved |
| 2014-05-22 18:48:21 | python-dev | set | nosy: +python-dev messages: +msg218910 |
| 2014-05-13 08:11:33 | vstinner | set | messages: +msg218411 |
| 2014-05-13 04:50:45 | neologix | set | messages: +msg218401 |
| 2014-05-12 22:18:22 | vstinner | set | messages: +msg218374 |
| 2014-05-12 21:47:05 | vstinner | set | messages: +msg218369 |
| 2014-05-11 15:37:46 | pitrou | set | messages: +msg218279 |
| 2014-05-11 15:36:58 | neologix | set | messages: +msg218278 |
| 2014-05-08 23:02:02 | neologix | set | files: +socket_listen-1.diff messages: +msg218135 |
| 2014-05-08 22:12:27 | pitrou | set | messages: +msg218127 |
| 2014-05-08 19:13:27 | neologix | create | |