PEP 416 – Add a frozendict builtin type
- Author:
- Victor Stinner <vstinner at python.org>
- Status:
- Rejected
- Type:
- Standards Track
- Created:
- 29-Feb-2012
- Python-Version:
- 3.3
I’m rejecting this PEP. A number of reasons (not exhaustive):
- According to Raymond Hettinger, use of frozendict is low. Thosethat do use it tend to use it as a hint only, such as declaringglobal or class-level “constants”: they aren’t really immutable,since anyone can still assign to the name.
- There are existing idioms for avoiding mutable default values.
- The potential of optimizing code using frozendict in PyPy isunsure; a lot of other things would have to change first. The sameholds for compile-time lookups in general.
- Multiple threads can agree by convention not to mutate a shareddict, there’s no great need for enforcement. Multiple processescan’t share dicts.
- Adding a security sandbox written in Python, even with a limitedscope, is frowned upon by many, due to the inherent difficulty withever proving that the sandbox is actually secure. Because of thiswe won’t be adding one to the stdlib any time soon, so this usecase falls outside the scope of a PEP.
On the other hand, exposing the existing read-only dict proxy as abuilt-in type sounds good to me. (It would need to be changed toallow calling the constructor.) GvR.
Update (2012-04-15): A newMappingProxyType type was added to the typesmodule of Python 3.3.
Add a new frozendict builtin type.
A frozendict is a read-only mapping: a key cannot be added nor removed, and akey is always mapped to the same value. However, frozendict values can be nothashable. A frozendict is hashable if and only if all values are hashable.
Use cases:
- Immutable global variable like a default configuration.
- Default value of a function parameter. Avoid the issue of mutable defaultarguments.
- Implement a cache: frozendict can be used to store function keywords.frozendict can be used as a key of a mapping or as a member of set.
- frozendict avoids the need of a lock when the frozendict is sharedby multiple threads or processes, especially hashable frozendict. It wouldalso help to prohibe coroutines (generators + greenlets) to modify theglobal state.
- frozendict lookup can be done at compile time instead of runtime because themapping is read-only. frozendict can be used instead of a preprocessor toremove conditional code at compilation, like code specific to a debug build.
- frozendict helps to implement read-only object proxies for security modules.For example, it would be possible to use frozendict type for __builtins__mapping or type.__dict__. This is possible because frozendict is compatiblewith the PyDict C API.
- frozendict avoids the need of a read-only proxy in some cases. frozendict isfaster than a proxy because getting an item in a frozendict is a fast lookupwhereas a proxy requires a function call.
- frozendict has to implement the Mapping abstract base class
- frozendict keys and values can be unorderable
- a frozendict is hashable if all keys and values are hashable
- frozendict hash does not depend on the items creation order
- Add a PyFrozenDictObject structure based on PyDictObject with an extra“Py_hash_t hash;” field
- frozendict.__hash__() is implemented using hash(frozenset(self.items())) andcaches the result in its private hash attribute
- Register frozendict as a collections.abc.Mapping
- frozendict can be used with PyDict_GetItem(), but PyDict_SetItem() andPyDict_DelItem() raise a TypeError
To ensure that a frozendict is hashable, values can be checkedbefore creating the frozendict:
importitertoolsdefhashabledict(*args,**kw):# ensure that all values are hashableforkey,valueinitertools.chain(args,kw.items()):ifisinstance(value,(int,str,bytes,float,frozenset,complex)):# avoid the compute the hash (which may be slow) for builtin# types known to be hashable for any valuecontinuehash(value)# don't check the key: frozendict already checks the keyreturnfrozendict.__new__(cls,*args,**kw)
namedtuple may fit the requirements of a frozendict.
A namedtuple is not a mapping, it does not implement the Mapping abstract baseclass.
frozendict can be implemented in Python using descriptors” and “frozendictjust need to be practically constant.
If frozendict is used to harden Python (security purpose), it must beimplemented in C. A type implemented in C is also faster.
ThePEP 351was rejected.
ThePEP 351 tries to freeze an object and so may convert a mutable object to animmutable object (using a different type). frozendict doesn’t convert anything:hash(frozendict) raises a TypeError if a value is not hashable. Freezing anobject is not the purpose of this PEP.
Python has a builtin dictproxy type used by type.__dict__ getter descriptor.This type is not public. dictproxy is a read-only view of a dictionary, but itis not read-only mapping. If a dictionary is modified, the dictproxy is alsomodified.
dictproxy can be used using ctypes and the Python C API, see for example themake dictproxy object via ctypes.pythonapi and type() (Python recipe 576540)by Ikkei Shimomura. The recipe contains a test checking that a dictproxy is“mutable” (modify the dictionary linked to the dictproxy).
However dictproxy can be useful in some cases, where its mutable property isnot an issue, to avoid a copy of the dictionary.
Whitelist approach.
- Implementing an Immutable Dictionary (Python recipe 498072) by Aristotelis Mikropoulos.Similar to frozendict except that it is not truly read-only: it is possibleto access to this private internal dict. It does not implement __hash__ andhas an implementation issue: it is possible to call again __init__() tomodify the mapping.
- PyWebmail contains an ImmutableDict type:webmail.utils.ImmutableDict.It is hashable if keys and values are hashable. It is not truly read-only:its internal dict is a public attribute.
- remember project:remember.dicts.FrozenDict.It is used to implement a cache: FrozenDict is used to store function callbacks.FrozenDict may be hashable. It has an extra supply_dict() class method tocreate a FrozenDict from a dict without copying the dict: store the dict asthe internal dict. Implementation issue: __init__() can be called to modifythe mapping and the hash may differ depending on item creation order. Themapping is not truly read-only: the internal dict is accessible in Python.
Blacklist approach: inherit from dict and override write methods to raise anexception. It is not truly read-only: it is still possible to call dict methodson such “frozen dictionary” to modify it.
Hashable dict: inherit from dict and just add an __hash__ method.
This document has been placed in the public domain.