
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2014-12-30 10:52 byserhiy.storchaka, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| ipaddress_pickle.patch | serhiy.storchaka,2014-12-30 10:52 | review | ||
| ipaddress_pickle_2.patch | serhiy.storchaka,2015-01-16 10:08 | Pickle addresses as ints | review | |
| ipaddress_pickle_3.patch | serhiy.storchaka,2015-01-16 10:08 | + optimization | review | |
| Messages (10) | |||
|---|---|---|---|
| msg233201 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2014-12-30 10:52 | |
Currently ipaddress classes support pickling, but the pickling is not efficient and is implementation depened. Proposed patch makes pickling more compact and implementation agnostic. | |||
| msg234118 -(view) | Author: Antoine Pitrou (pitrou)*![]() | Date: 2015-01-16 08:49 | |
Patch looks good to me. For further efficiency, addresses could be pickled as ints (but beware of interfaces and networks). | |||
| msg234121 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2015-01-16 10:06 | |
Yes, pickling (and especially unpickling) ints is more efficient, but the code will more complex. Interfaces should be pickled as strings for backward compatibility, and interfaces are subclasses of addresses.Here are microbenchmarks:./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]" -- "pickle.dumps(ips)"./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]" -- "pickle.dumps(ips)"./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]; pickled = pickle.dumps(ips)" -- "pickle.loads(pickled)"./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]; pickled = pickle.dumps(ips)" -- "pickle.loads(pickled)"Results for unpatched module:1000 loops, best of 3: 1.56 msec per loop1000 loops, best of 3: 1.62 msec per loop1000 loops, best of 3: 1.08 msec per loop1000 loops, best of 3: 1.09 msec per loopWith ipaddress_pickle.patch:100 loops, best of 3: 3.43 msec per loop100 loops, best of 3: 10.6 msec per loop100 loops, best of 3: 7.76 msec per loop100 loops, best of 3: 8.58 msec per loopWith ipaddress_pickle_2.patch:1000 loops, best of 3: 1.11 msec per loop1000 loops, best of 3: 1.16 msec per loop1000 loops, best of 3: 1.88 msec per loop100 loops, best of 3: 2.05 msec per loopWith ipaddress_pickle_3.patch:1000 loops, best of 3: 1.12 msec per loop1000 loops, best of 3: 1.15 msec per loop1000 loops, best of 3: 1.13 msec per loop1000 loops, best of 3: 1.15 msec per loop | |||
| msg234123 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2015-01-16 10:20 | |
ipaddress_pickle_3.patch breaks one test (testMissingAddressVersion). Is this test needed? | |||
| msg234124 -(view) | Author: Antoine Pitrou (pitrou)*![]() | Date: 2015-01-16 10:43 | |
I don't understand what the test is for. I think it's safe it's remove it. | |||
| msg234264 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2015-01-18 18:27 | |
Then I'll remove it. Could you please make a review of optimized patch? | |||
| msg234270 -(view) | Author: Antoine Pitrou (pitrou)*![]() | Date: 2015-01-18 20:13 | |
The patch looks fine to me. | |||
| msg234274 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2015-01-18 20:39 | |
New changeset781b54f7bccc by Serhiy Storchaka in branch 'default':Issue#23133: Pickling of ipaddress objects now produces more compact andhttps://hg.python.org/cpython/rev/781b54f7bccc | |||
| msg234276 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2015-01-18 20:53 | |
Thank you Antoine.And here is comparison of pickle size.Unpatched:>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]))2971>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]))4071>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in range(1, 101)]))19341>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in range(1, 101)]))22741>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in range(1, 101)]))10614>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i in range(1, 101)]))22741Patched:>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]))1531>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]))2631>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in range(1, 101)]))2963>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in range(1, 101)]))3256>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in range(1, 101)]))2938>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i in range(1, 101)]))3209 | |||
| msg234277 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2015-01-18 20:57 | |
New changeset712ac77b772b by Serhiy Storchaka in branch 'default':Fixed tests for issue#23133 (pickling of IPv4Network was not tested).https://hg.python.org/cpython/rev/712ac77b772b | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:11 | admin | set | github: 67322 |
| 2015-01-18 20:57:32 | python-dev | set | messages: +msg234277 |
| 2015-01-18 20:53:56 | serhiy.storchaka | set | status: open -> closed resolution: fixed messages: +msg234276 stage: patch review -> resolved |
| 2015-01-18 20:39:56 | python-dev | set | nosy: +python-dev messages: +msg234274 |
| 2015-01-18 20:13:29 | pitrou | set | messages: +msg234270 |
| 2015-01-18 18:27:23 | serhiy.storchaka | set | messages: +msg234264 |
| 2015-01-16 14:49:59 | serhiy.storchaka | set | assignee:serhiy.storchaka |
| 2015-01-16 10:43:24 | pitrou | set | messages: +msg234124 |
| 2015-01-16 10:20:34 | serhiy.storchaka | set | messages: +msg234123 |
| 2015-01-16 10:08:51 | serhiy.storchaka | set | files: +ipaddress_pickle_3.patch |
| 2015-01-16 10:08:25 | serhiy.storchaka | set | files: +ipaddress_pickle_2.patch |
| 2015-01-16 10:08:02 | serhiy.storchaka | set | files: -ipaddress_pickle_3.patch |
| 2015-01-16 10:07:53 | serhiy.storchaka | set | files: -ipaddress_lightweight_2.patch |
| 2015-01-16 10:07:43 | serhiy.storchaka | set | files: +ipaddress_pickle_3.patch |
| 2015-01-16 10:06:50 | serhiy.storchaka | set | files: +ipaddress_lightweight_2.patch messages: +msg234121 |
| 2015-01-16 08:49:12 | pitrou | set | nosy: +pitrou messages: +msg234118 |
| 2014-12-30 10:56:39 | serhiy.storchaka | link | issue23103 dependencies |
| 2014-12-30 10:52:49 | serhiy.storchaka | create | |