Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitdda70fa

Browse files
furkanondereendebakptencukoupicnixz
authored
gh-99631: Add customloads anddumps support for theshelve module (#118065)
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>Co-authored-by: Petr Viktorin <encukou@gmail.com>Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
1 parentc564847 commitdda70fa

File tree

4 files changed

+336
-36
lines changed

4 files changed

+336
-36
lines changed

‎Doc/library/shelve.rst

Lines changed: 61 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,8 @@ This includes most class instances, recursive data types, and objects containing
1717
lots of shared sub-objects. The keys are ordinary strings.
1818

1919

20-
..function::open(filename, flag='c', protocol=None, writeback=False)
20+
..function::open(filename, flag='c', protocol=None, writeback=False, *, \
21+
serializer=None, deserializer=None)
2122

2223
Open a persistent dictionary. The filename specified is the base filename for
2324
the underlying database. As a side-effect, an extension may be added to the
@@ -41,13 +42,32 @@ lots of shared sub-objects. The keys are ordinary strings.
4142
determine which accessed entries are mutable, nor which ones were actually
4243
mutated).
4344

45+
By default,:mod:`shelve` uses:func:`pickle.dumps` and:func:`pickle.loads`
46+
for serializing and deserializing. This can be changed by supplying
47+
*serializer* and *deserializer*, respectively.
48+
49+
The *serializer* argument must be a callable which takes an object ``obj``
50+
and the *protocol* as inputs and returns the representation ``obj`` as a
51+
:term:`bytes-like object`; the *protocol* value may be ignored by the
52+
serializer.
53+
54+
The *deserializer* argument must be callable which takes a serialized object
55+
given as a:class:`bytes` object and returns the corresponding object.
56+
57+
A:exc:`ShelveError` is raised if *serializer* is given but *deserializer*
58+
is not, or vice-versa.
59+
4460
..versionchanged::3.10
4561
:const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle
4662
protocol.
4763

4864
..versionchanged::3.11
4965
Accepts:term:`path-like object` for filename.
5066

67+
..versionchanged::next
68+
Accepts custom *serializer* and *deserializer* functions in place of
69+
:func:`pickle.dumps` and:func:`pickle.loads`.
70+
5171
..note::
5272

5373
Do not rely on the shelf being closed automatically; always call
@@ -129,7 +149,8 @@ Restrictions
129149
explicitly.
130150

131151

132-
..class::Shelf(dict, protocol=None, writeback=False, keyencoding='utf-8')
152+
..class::Shelf(dict, protocol=None, writeback=False, \
153+
keyencoding='utf-8', *, serializer=None, deserializer=None)
133154
134155
A subclass of:class:`collections.abc.MutableMapping` which stores pickled
135156
values in the *dict* object.
@@ -147,6 +168,9 @@ Restrictions
147168
The *keyencoding* parameter is the encoding used to encode keys before they
148169
are used with the underlying dict.
149170

171+
The *serializer* and *deserializer* parameters have the same interpretation
172+
as in:func:`~shelve.open`.
173+
150174
A:class:`Shelf` object can also be used as a context manager, in which
151175
case it will be automatically closed when the:keyword:`with` block ends.
152176

@@ -161,8 +185,13 @@ Restrictions
161185
:const:`pickle.DEFAULT_PROTOCOL` is now used as the default pickle
162186
protocol.
163187

188+
..versionchanged::next
189+
Added the *serializer* and *deserializer* parameters.
164190

165-
..class::BsdDbShelf(dict, protocol=None, writeback=False, keyencoding='utf-8')
191+
192+
..class::BsdDbShelf(dict, protocol=None, writeback=False, \
193+
keyencoding='utf-8', *, \
194+
serializer=None, deserializer=None)
166195
167196
A subclass of:class:`Shelf` which exposes:meth:`!first`,:meth:`!next`,
168197
:meth:`!previous`,:meth:`!last` and:meth:`!set_location` methods.
@@ -172,18 +201,27 @@ Restrictions
172201
modules. The *dict* object passed to the constructor must support those
173202
methods. This is generally accomplished by calling one of
174203
:func:`!bsddb.hashopen`,:func:`!bsddb.btopen` or:func:`!bsddb.rnopen`. The
175-
optional *protocol*, *writeback*, and *keyencoding* parameters have the same
176-
interpretation as for the:class:`Shelf` class.
204+
optional *protocol*, *writeback*, *keyencoding*, *serializer* and *deserializer*
205+
parameters have the same interpretation as in:func:`~shelve.open`.
206+
207+
..versionchanged::next
208+
Added the *serializer* and *deserializer* parameters.
177209

178210

179-
..class::DbfilenameShelf(filename, flag='c', protocol=None, writeback=False)
211+
..class::DbfilenameShelf(filename, flag='c', protocol=None, \
212+
writeback=False, *, serializer=None, \
213+
deserializer=None)
180214
181215
A subclass of:class:`Shelf` which accepts a *filename* instead of a dict-like
182216
object. The underlying file will be opened using:func:`dbm.open`. By
183217
default, the file will be created and opened for both read and write. The
184-
optional *flag* parameter has the same interpretation as for the:func:`.open`
185-
function. The optional *protocol* and *writeback* parameters have the same
186-
interpretation as for the:class:`Shelf` class.
218+
optional *flag* parameter has the same interpretation as for the
219+
:func:`.open` function. The optional *protocol*, *writeback*, *serializer*
220+
and *deserializer* parameters have the same interpretation as in
221+
:func:`~shelve.open`.
222+
223+
..versionchanged::next
224+
Added the *serializer* and *deserializer* parameters.
187225

188226

189227
.. _shelve-example:
@@ -225,6 +263,20 @@ object)::
225263
d.close() # close it
226264

227265

266+
Exceptions
267+
----------
268+
269+
..exception::ShelveError
270+
271+
Exception raised when one of the arguments *deserializer* and *serializer*
272+
is missing in the:func:`~shelve.open`,:class:`Shelf`,:class:`BsdDbShelf`
273+
and:class:`DbfilenameShelf`.
274+
275+
The *deserializer* and *serializer* arguments must be given together.
276+
277+
..versionadded::next
278+
279+
228280
..seealso::
229281

230282
Module:mod:`dbm`

‎Lib/shelve.py

Lines changed: 38 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -56,12 +56,17 @@
5656
the persistent dictionary on disk, if feasible).
5757
"""
5858

59-
frompickleimportDEFAULT_PROTOCOL,Pickler,Unpickler
59+
frompickleimportDEFAULT_PROTOCOL,dumps,loads
6060
fromioimportBytesIO
6161

6262
importcollections.abc
6363

64-
__all__= ["Shelf","BsdDbShelf","DbfilenameShelf","open"]
64+
__all__= ["ShelveError","Shelf","BsdDbShelf","DbfilenameShelf","open"]
65+
66+
67+
classShelveError(Exception):
68+
pass
69+
6570

6671
class_ClosedDict(collections.abc.MutableMapping):
6772
'Marker for a closed dict. Access attempts raise a ValueError.'
@@ -82,7 +87,7 @@ class Shelf(collections.abc.MutableMapping):
8287
"""
8388

8489
def__init__(self,dict,protocol=None,writeback=False,
85-
keyencoding="utf-8"):
90+
keyencoding="utf-8",*,serializer=None,deserializer=None):
8691
self.dict=dict
8792
ifprotocolisNone:
8893
protocol=DEFAULT_PROTOCOL
@@ -91,6 +96,16 @@ def __init__(self, dict, protocol=None, writeback=False,
9196
self.cache= {}
9297
self.keyencoding=keyencoding
9398

99+
ifserializerisNoneanddeserializerisNone:
100+
self.serializer=dumps
101+
self.deserializer=loads
102+
elif (serializerisNone)^ (deserializerisNone):
103+
raiseShelveError("serializer and deserializer must be "
104+
"defined together")
105+
else:
106+
self.serializer=serializer
107+
self.deserializer=deserializer
108+
94109
def__iter__(self):
95110
forkinself.dict.keys():
96111
yieldk.decode(self.keyencoding)
@@ -110,19 +125,17 @@ def __getitem__(self, key):
110125
try:
111126
value=self.cache[key]
112127
exceptKeyError:
113-
f=BytesIO(self.dict[key.encode(self.keyencoding)])
114-
value=Unpickler(f).load()
128+
f=self.dict[key.encode(self.keyencoding)]
129+
value=self.deserializer(f)
115130
ifself.writeback:
116131
self.cache[key]=value
117132
returnvalue
118133

119134
def__setitem__(self,key,value):
120135
ifself.writeback:
121136
self.cache[key]=value
122-
f=BytesIO()
123-
p=Pickler(f,self._protocol)
124-
p.dump(value)
125-
self.dict[key.encode(self.keyencoding)]=f.getvalue()
137+
serialized_value=self.serializer(value,self._protocol)
138+
self.dict[key.encode(self.keyencoding)]=serialized_value
126139

127140
def__delitem__(self,key):
128141
delself.dict[key.encode(self.keyencoding)]
@@ -191,33 +204,29 @@ class BsdDbShelf(Shelf):
191204
"""
192205

193206
def__init__(self,dict,protocol=None,writeback=False,
194-
keyencoding="utf-8"):
195-
Shelf.__init__(self,dict,protocol,writeback,keyencoding)
207+
keyencoding="utf-8",*,serializer=None,deserializer=None):
208+
Shelf.__init__(self,dict,protocol,writeback,keyencoding,
209+
serializer=serializer,deserializer=deserializer)
196210

197211
defset_location(self,key):
198212
(key,value)=self.dict.set_location(key)
199-
f=BytesIO(value)
200-
return (key.decode(self.keyencoding),Unpickler(f).load())
213+
return (key.decode(self.keyencoding),self.deserializer(value))
201214

202215
defnext(self):
203216
(key,value)=next(self.dict)
204-
f=BytesIO(value)
205-
return (key.decode(self.keyencoding),Unpickler(f).load())
217+
return (key.decode(self.keyencoding),self.deserializer(value))
206218

207219
defprevious(self):
208220
(key,value)=self.dict.previous()
209-
f=BytesIO(value)
210-
return (key.decode(self.keyencoding),Unpickler(f).load())
221+
return (key.decode(self.keyencoding),self.deserializer(value))
211222

212223
deffirst(self):
213224
(key,value)=self.dict.first()
214-
f=BytesIO(value)
215-
return (key.decode(self.keyencoding),Unpickler(f).load())
225+
return (key.decode(self.keyencoding),self.deserializer(value))
216226

217227
deflast(self):
218228
(key,value)=self.dict.last()
219-
f=BytesIO(value)
220-
return (key.decode(self.keyencoding),Unpickler(f).load())
229+
return (key.decode(self.keyencoding),self.deserializer(value))
221230

222231

223232
classDbfilenameShelf(Shelf):
@@ -227,9 +236,11 @@ class DbfilenameShelf(Shelf):
227236
See the module's __doc__ string for an overview of the interface.
228237
"""
229238

230-
def__init__(self,filename,flag='c',protocol=None,writeback=False):
239+
def__init__(self,filename,flag='c',protocol=None,writeback=False,*,
240+
serializer=None,deserializer=None):
231241
importdbm
232-
Shelf.__init__(self,dbm.open(filename,flag),protocol,writeback)
242+
Shelf.__init__(self,dbm.open(filename,flag),protocol,writeback,
243+
serializer=serializer,deserializer=deserializer)
233244

234245
defclear(self):
235246
"""Remove all items from the shelf."""
@@ -238,8 +249,8 @@ def clear(self):
238249
self.cache.clear()
239250
self.dict.clear()
240251

241-
242-
defopen(filename,flag='c',protocol=None,writeback=False):
252+
defopen(filename,flag='c',protocol=None,writeback=False,*,
253+
serializer=None,deserializer=None):
243254
"""Open a persistent dictionary for reading and writing.
244255
245256
The filename parameter is the base filename for the underlying
@@ -252,4 +263,5 @@ def open(filename, flag='c', protocol=None, writeback=False):
252263
See the module's __doc__ string for an overview of the interface.
253264
"""
254265

255-
returnDbfilenameShelf(filename,flag,protocol,writeback)
266+
returnDbfilenameShelf(filename,flag,protocol,writeback,
267+
serializer=serializer,deserializer=deserializer)

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp