- Notifications
You must be signed in to change notification settings - Fork1
A MessagePack-based storage extension to tinydb using thehttp://msgpack.org
License
alshapton/TinyMP
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
TinyMP is a storage backend for TinyDBhttps://github.com/msiemens/tinydb which is based around the MessagePack compressed JSON format (https://msgpack.org/index.html)
TinyMP extends the syntax of thetinydb
class using one of the optionalkwargs
as follows:
**kwargs
Value | Effect |
---|---|
storage=MsgPackStorage | Default option, will use theMsgPack library |
storage=MsgPackStorage,Lib='msgpack' | Will use theMsgPack library |
storage=MsgPackStorage,Lib='umsgpack' | Will use theU-MsgPack Library |
fromtinydbimportTinyDB,Queryfromtinympimport*db=TinyDB('data.msg',storage=MsgPackStorage)defdbins():db.insert({'type':'apple','count':7})dbins()
As you can see, it's a simple drop-in replacement for any storage engineand it can be nested and cached. Don't forget, you will need to install as a minimum,themsgpack-python
library usingpip install msgpack-python
and theU-MsgPack
library fromhttps://github.com/vsergeev/u-msgpack-python in order to use that option.
fromtinydbimportTinyDB,Queryfromtinympimport*db=TinyDB('data.msg',storage=MsgPackStorage,Lib='umsgpack')defdbins():db.insert({'type':'apple','count':7})dbins()
As you can see, it's a simple drop-in replacement for any storage engineand it can be nested and cached.
Looking at the statistics below, it's apparent that compared to the "standard"JSON Storage mechanism, MessagePack isn't as quick, however, the filesizes ondisc are smaller - consider the table below, with 1,000 JSON documents ofminute size - clearly, the MessagePack compressed format is smaller thanthe JSON format. Whether you choose the default MsgPack library, which ismarginally slower than the U-MsgPack library (at the cost of a small increasein storage footprint with U-MsgPack) is dependent on your use case.
Format | Run 1 | Run 2 | Run 3 | FileSize |
---|---|---|---|---|
JSON Write: | 2.147 | 2.011 | 2.040 | 37.0 Kb |
MsgPack Write: | 9.562 | 9.732 | 9.716 | 21.1 Kb |
U-MsgPack Write: | 9.354 | 9.066 | 8.949 | 24.1 Kb |
- Version 1.0.0-Beta5 - XX/XX/XXXX
- Added PiPy compatibility
- PEP-8 Compliant code style
- Version 1.0.0-Beta4 - 11/26/2017
- Added descriptions of benchmarking and tidied up repo
- Further added information to README about usage
- Version 1.0.0-Beta3 - 11/24/2017
- Added support for u-msgpack-python alternative msgpack library
- Version 1.0.0-Beta2 - 11/21/2017
- Fixed Testing issues
- Version 1.0.0-Beta - 11/19/2017
- Tidied up repo ready for Beta release
- Version 0.2.0 - 11/18/2017
- Tidied up repo
- Added initial benchmarking and example
- Added Github Community files.
- Version 0.1 - 11/13/2017
- Initial Release.
About
A MessagePack-based storage extension to tinydb using thehttp://msgpack.org