- Notifications
You must be signed in to change notification settings - Fork18
A small header-only library for converting data between json representation and c++ structs
License
Stiffstream/json_dto
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
- Table of Contents
- What Is json_dto?
- What's new?
- Obtain and build
- How to use it?
- License
Created bygh-md-toc
json_dto library is a small header-only helperfor converting data between json representationand c++ structs. DTO here stands for data transfer object.It was made and used as a part of a larger projectinsideStiffStream.And since Fall 2016 is ready for public. We are still using it forworking with JSON in various projects.
Several newto_json,from_json,to_stream andfrom_stream functionsthat accept Reader-Writer parameter. For example:
// Type to be serialized.classsome_data {public:...template<typename Io>voidjson_io(Io & io) {...}};// Special Reader-Writer for packing/unpacking instances of some_data.structsome_data_reader_writer {voidread( some_data & obj,const rapidjson::Value & from )const {...}voidwrite(const some_data & obj, rapidjson::Value & to, rapidjson::MemoryPoolAllocator<> & allocator )const {...}};...// Object to be serialized.some_data data_to_pack{...};// Serialization by using custom Reader-Writer object.auto json_string = json_dto::to_json(some_data_reader_writer{}, data_to_pack);
See a newdeserialization example that shows howa newfrom_json function can be used.
There is alsoa new example that shows how a custom Reader-Writer may change representation of an item.
Support for storing of several fields into an array added.
It's a bug-fix release.
Support forstd::int8_t andstd::uint8_t added to json-dto.
Version 0.3.0 introduces a couple of breaking changes that can affect some users.
Previous versions of json-dto calledset_attr_null_value function when 'null'value was found during deserialization. There were two overloads forset_attr_null_value: one fornullable_t<T> and another for all other cases.The overload fornullable_t<T> reset the nullable field. The overload for allother cases threw an exception.
It's important to note that the manopt_policy trait wasn't used for handling'null' values.
Since v.0.3.0 an updated approach of dealing with 'null' values is used. Nowthe manopt_policy is used when 'null' is found during deserialization:on_null method from manopt_policy is now called when 'null' is found.
Function templatesset_attr_null_value were removed. They were replaced bynew function templatesdefault_on_null that have the same logic (but underthe new names).
This change means that if you have your own implementation of manopt_policythen you have to add theon_null template method to it. For example:
structmy_manopt_policy{template<typename Field_Type>voidon_field_not_defined(Field_Type &)const { ... }template<typename Field_Type>boolis_default_value(Field_Type &)const { ... }template<typename Field_Type>voidon_null(Field_Type & f)const{json_dto::default_on_null(f);}}
Also it means that if you have your own specialization forbinder_read_from_implementation_t class template then you have to replace acall toset_attr_null_value by a call to manopt_policy'son_null method(see the description ofbinder_read_from_implementation_t for more info).
A new binder functionmandatory_with_null_as_default introduced. It allowsbinding a mandatory attribute that allows 'null' value in JSON. If 'null' isfound during deserialization then a field of typeT will receiveT{} as avalue (it means thatT has to be DefaultConstructible). For example:
structmy_data{int type_{-1};std::vector<std::string> headers_;...template<typename Json_Io>voidjson_io(Json_Io & io){io// In case of 'null' type_ will receive 0 (because int{} produces 0).&json_dto::mandatory_with_null_as_default("type", type_ )// In case of 'null' headers_ will be empty// (because std::vector<std::string>{} produces empty vector).&json_dto::mandatory_with_null_as_default("headers", headers_ )...;}};
A new overloadto_strean added that receives an instance ofpretty_writer_params_t with parameters for RapidJSON's PrettyWriter:
std::ofstream target_file{...};my_data my_obj;...// Make default serialization, without pretty-writer:json_dto::to_strean(target_file, my_obj);// Make serialization with pretty-writer:json_dto::to_stream(target_file, my_obj,json_dto::pretty_writer_params_t{}.indent_char('').indent_char_count(3u).format_options(rapidjson::kFormatSingleLineArray));A new overloadto_json added that receives an instance ofpretty_writer_params_t with parameters for RapidJSON's PrettyWriter:
my_data my_obj;...// Make default serialization, without pretty-writer:std::string my_obj_image = json_dto::to_json(my_obj);// Make serialization with pretty-writer:std::string my_obj_pretty_image = json_dto::to_json(my_obj,json_dto::pretty_writer_params_t{}.indent_char('').indent_char_count(3u).format_options(rapidjson::kFormatSingleLineArray));
Functionsmandatory,optional,optional_no_default, andoptional_null now accepts const- and rvalue references. That can be useful for types that have to be only serializable. For example:
structdemo {constint priority_;std::vector<int>version()const {return {1,2,3 }; }const std::string payload_;demo(int priority, std::string payload):priority_{priority}, payload_{std::move(payload)}{}template<typename Json_Io>voidjson_io(Json_Io & io) {io &json_dto::mandatory("priority", priority_)&json_dto::mandatory("version",version())&json_dto::mandatory("payload", payload_);}};
Please note that this code will lead to a compilation error in an attempt to deserialize an instance ofdemo type.
The class templatejson_dto::binder_t was refactored and now it uses several new customization points in the implementation:binder_data_holder_t,binder_read_from_implementation_t andbinder_write_to_implementation_t. Those customization points allow to add a new functionality without modifying the json-dto source code.
For example, a user now can do something like:
namespacetricky_stuff {template<typename F>structserialize_only_proxy {...};template<typename F> serialize_only_proxy<F>serialize_only(const F & f) {...}template<typename F>structdeserialize_only_proxy {...};template<typename F> deserialize_only_proxy<F>deserialize_only(F & f) {...}}// namespace tricky_stuffnamespacejson_dto {...// Several partial specializations of binder_data_holder_t,// binder_read_from_implementation_t and binder_write_to_implementation_t// for tricky_stuff::serialize_only_proxy and tricky_stuff::deserialize_only_proxy.}// namespace json_dtostructdemo {constint priority_;std::vector<int>version()const {return {1,2,3 }; }const std::string payload_;std::vector<std::string> obsolete_properties_;demo(int priority, std::string payload):priority_{priority}, payload_{std::move(payload)}{}template<typename Json_Io>voidjson_io(Json_Io & io) {io &json_dto::mandatory("priority",tricky_stuff::serialize_only(priority_))&json_dto::mandatory("version",tricky_stuff::serialize_only(version()))&json_dto::mandatory("payload", payload_)&json_dto::optional_no_default("properties",tricky_stuff::deserialize_only(obsolete_properties_);}};
Several examples of how stuff like that can be implemented are shown in json_dto'ssamples folder:serialize_only implementation,deserialize_only implementation,ignore_after_deserialization implementation.
Such functions likeserialize_only anddeserialize_only can be useful in data-transformation code. For example, when we have to read some old data in JSON format, modify the data read and write it in a slightly different JSON.
New typesmutable_map_key_t<T> andconst_map_key_t<T> are now used for (de)serializing keys of map-like structures. Seethe description below.
A new Reader_Writer proxyapply_to_content_t added to address an issue of using custom Reader_Writers to the content of containers,nullable_t andstd::optional. Seethe description below.
Another way of custom read/write operations added. It's based on specifying an instance of some user-supplied Reader_Writer type in the description of a field. SeeUsage of Reader_Writer for more details.
For support of that feature new overloads ofjson_dto::mandatory,json_dto::optional, andjson_dto::optional_no_default have been added.
There is also a newjson_dto::write_json_value overload:
voidwrite_json_value(const rapidjson::Value::StringRefType & s,rapidjson::Value & object,rapidjson::MemoryPoolAllocator<> & allocator );
Please note thatjson_dto::string_ref_t is just an alias forrapidjson::Value::StringRefType.
New overloads forfrom_json function:
// Parses null-terminated string and returns a new object.template<typename Type,unsigned Rapidjson_Parseflags = rapidjson::kParseDefaultFlags>Typefrom_json(constchar * json );// Parses null-terminated string into alredy existed object.template<typename Type,unsigned Rapidjson_Parseflags = rapidjson::kParseDefaultFlags>voidfrom_json(constchar * json, Type & o );// Parses a string-view and returns a new object.// NOTE. string_ref_t is just an alias for RapidJSON's StringRefType.template<typename Type,unsigned Rapidjson_Parseflags = rapidjson::kParseDefaultFlags>Typefrom_json(conststring_ref_t & json );// Parses a string-view into alredy existed object.template<typename Type,unsigned Rapidjson_Parseflags = rapidjson::kParseDefaultFlags>voidfrom_json(conststring_ref_t & json, Type & o );
Versions withstring_ref_t arguments are intended to be used in cases where a part of existing buffer should be parsed. For example:
std::vector<char> pdu = extract_data();// string_view from C++17 is used just for a demonstration.string_view headers;string_view payload;std::tie(headers, payload) = split_pdu_to_headers_and_payload(&pdu.front(), pdu.size());auto parsed_payload = json_dto::from_json<PayloadType>(json_dto::make_string_ref(payload.data(), payload.size());
Versions withconst char * are added to resolve ambious overloads withfrom_string(const std::string &) andfrom_string(const string_ref_t &)in the following cases:
auto payload = json_dto::from_json<PayloadType>(R"JSON({"id":10})JSON");
Please note thatstring_ref_t is notstd::string_view.string_ref_tis just an alias for RapidJSON'sStringRefType. And typeStringRefTypehas a constructor is the form:
StringRefType(constchar * ch, SizeType length);
But RapidJSON'sSizeType is notstd::size_t. So if someone writes:
std::vector<char> payload{...};auto data = json_dto::from_json<PayloadType>(json_dto::string_ref_t{&payload.front(), payload.size()} );
there could be a warning from the compiler about narrowing ofstd::size_ttoSizeType.
A set onmake_string_ref functions is added to json_dto to avoid such warnings:
string_ref_tmake_string_ref(constchar * v);string_ref_tmake_string_ref(constchar * v, std::size_t length);string_ref_tmake_string_ref(const std::string & v);
Use those functions to avoid warnings from the compiler:
std::vector<char> payload{...};auto data = json_dto::from_json<PayloadType>(json_dto::make_string_ref(&payload.front(), payload.size()) );
Support for STL containers likestd::deque,std::list,std::forward_list,std::set,std::unordered_set,std::map andstd::unordered_map is implemented.These types can be used as types of fields in a serialized type, for example:
#include<json_dto/pub.hpp>#include<deque>#include<set>#include<map>structmy_message {std::deque<int> ids_;std::set<std::string> tags_;std::map<std::string, some_another_type> props_;...template<typename Json_Io>voidjson_io(Json_Io & io) {io &json_dto::mandatory("ids", ids_)&json_dto::mandatory("tags", tags_)&json_dto::mandatory("properties", props_)...;}};
These types can also be used withjson_dto::from_json() andjson_dto::to_json() functions:
auto messages = json_dto::from_json< std::forward_list<my_message> >(...);...auto json = json_dto::to_json(messages);
A new example tutorial17 added. This example shows the usage of new features.
An important note about support forstd::multiset,std::unordered_multiset,std::multimap andstd::unordered_multimap: those containers are also supported.But json_dto doesn't do any checks for duplicate keys. In that aspect, json_dto relieson RapidJSON behavior. For example, if an instance ofstd::multimap contains severalvalues for some key all those values will be serialized.What happens to those values is dependent on RapidJSON.
Two new forms offrom_json added. It is possible now to deserialize a DTO from already parsed document. For example:
structupdate_period {...template<typename Json_Io>voidjson_io(Json_Io & io) {...}};structread_sensor {...template<typename Json_Io>voidjson_io(Json_Io & io) {...}};...voidparse_and_handle_message(const std::string & raw_msg ){rapidjson::Document whole_msg;whole_msg.Parse< rapidjson::kParseDefaultFlags >( raw_msg );if( whole_msg.HasParseError() )throwstd::runtime_error(std::string{"unable to parse message:" } +rapidjson::GetParseError_En( whole_msg.GetParseError() ) );const std::string msg_type = whole_msg["message_type" ].GetString();constauto & payload = whole_msg["payload" ];if("Update-Period" == msg_type ){auto dto = json_dto::from_json< update_period >( payload );...}elseif("Read-Sensor" == msg_type ){auto dto = json_dto::from_json< read_sensor >( payload );...}else...}
Fix: compilation problems on FreeBSD 12 with clang-6.0.1.
Fix: add check for reading fields of DTO to ensure that a source JSON value is of type Object.
Improvestd::optional availability check.
Support forstd::vector forjson_dto::to_json andjson_dto::from_jsonfunctions.
Modify cmake-scripts for vcpkg port (target namejson-dto::json-dto).
Add cmake support.
Make string value setter independent toRAPIDJSON_HAS_STDSTRING.
Bug fix in support ofstd::vector<bool>.
Bug fix in implementation ofstd::optional support.
New example tutorial6.1 added.
Some code style changes to meet expectations of some users.
New format ofread_json_value function.NOTE: this is a breaking change!
Support forstd::optional (andstd::experimental::optional) added. Note: thismay require to specify C++17 standard in compiler params (like/std:c++17 for MSVC or-std=c++17 for GCC).
To usejson_dto it is necessary to have:
- C++14 compiler (VC++15.0, GCC 5.4 or above, clang 4.0 or above)
- rapidjson
And for building with mxxru:
- rapidjson_mxxru(v.1.0.0 or above)
- Mxx_ru 1.6.13 or above
And for running test:
- CATCH2 2.5.0 or above
Assuming thatGit andMxx_ru are already installed.
git clone https://github.com/Stiffstream/json_dto.gitAnd then:
cd json_dto-0.2mxxruexternalsto download and extractjson_dto's dependencies.
Forjson_dto itself:
MxxRu::arch_externals:json_dtodo |e|e.url'https://github.com/Stiffstream/json_dto/archive/v.0.2.8.1.tar.gz'e.map_dir'dev/json_dto'=>'dev'end
Forrapidjson andrapidjson_mxxru dependencies:
MxxRu::arch_externals:rapidjsondo |e|e.url'https://github.com/miloyip/rapidjson/archive/v1.1.0.zip'e.map_dir'include/rapidjson'=>'dev/rapidjson/include'endMxxRu::arch_externals:rapidjson_mxxrudo |e|e.url'https://github.com/Stiffstream/rapidjson_mxxru/archive/v.1.0.1.tar.gz'e.map_dir'dev/rapidjson_mxxru'=>'dev'end
Whilejson_dto is header-only library test and samples require a build.
Compiling with Mxx_ru:
git clone https://github.com/Stiffstream/json_dtocd json_dtomxxruexternalscd devruby build.rbNOTE. It might be necessary to set upMXX_RU_CPP_TOOLSET environment variable,see Mxx_ru documentation for further details.
An important notice: if you do not use Mxx_ru for building your project thenadd the following defines for your project:
RAPIDJSON_HAS_STDSTRINGRAPIDJSON_HAS_CXX11_RVALUE_REFS
If you use Mxx_ru andrapidjson_mxxru/prj.rb then these definitions will beadded automatically.
To start usingjson_dto simply include<json_dto/pub.hpp> header.
The usage principle ofjson_dto is borrowed fromBoost serializationwhererapidjson::Value plays the role of archive.
Let's assume we have a c++ structure that must be serialized to JSONand deserialized from JSON:
structmessage_t{std::string m_from;std::int64_t m_when;std::string m_text;};
For integrating this struct withjson_dto facilities the struct must bemodified as follows:
structmessage_t{std::string m_from;std::int64_t m_when;std::string m_text;// Entry point for json_dto.template<typename Json_Io>voidjson_io(Json_Io & io){io &json_dto::mandatory("from", m_from)&json_dto::mandatory("when", m_when)&json_dto::mandatory("text", m_text);}};
Herejson_io() function is an entry point forjson_dto library.It describes how to read the data fromrapidjson::Value(that is usualy parsed from string) and how to set the datainrapidjson::Value.json_io() is a template function. It allows to have a singledescription for read and write operations.The template is instantiated withJson_Io=json_dto::json_input_tfor reading dto from JSON-value andJson_Io=json_dto::json_output_t for writingdto to JSON-value. Bothjson_dto::json_input_t andjson_dto::json_output_toverrideoperator& for splitting io functionality.
There are also iostream-like overrides foroperator<< andoperator>>:
template<typename Dto>json_input_t &operator>>(json_input_t & i, Dto & v);template<typename Dto>inlinejson_output_t &operator<<(json_output_t & o,const Dto & v);
But they are only helpful for top level read/write operations.
In generaljson_dto gets data fromrapidjson::Value and putsthe data intorapidjson::Value. So read/write operations look like this:
// Readrapidjson::Document document;// ...json_dto::json_input_t jin{ document };message_t msg;jin >> msg;// If no exceptions were thrown DTO contains data received from JSON.
// Writerapidjson::Document document;// ...json_dto::json_output_t jout{ document, document.GetAllocator() };constmessage_t msg = get_message();jout << msg;// If no exceptions were thrown document contains data received from DTO.
But usually it is enough to work withstd::string objects, sojson_dtocomes with handy to/from string helpers:
template<typename Dto>std::stringto_json(const Dto & dto);template<typename Type>Typefrom_json(const std::string & json);
See full example without to/from string helpers.
When it is unwanted to add an extra function to C++ structureit is possible to use a non intrusivejson_io() version.In previous example dto part will look like this:
structmessage_t{std::string m_from;std::int64_t m_when;std::string m_text;};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,message_t & msg){io &json_dto::mandatory("from", msg.m_from)&json_dto::mandatory("when", msg.m_when)&json_dto::mandatory("text", msg.m_text);}}/* namespace json_dto*/
Note that it is necessary to definejson_io() in namespacejson_dto.
Out of the boxjson_dto lib supports following types:
- Bool: bool;
- Numeric:std::int8_t, std::uint8_t,std::int16_t, std::uint16_t,std::int32_t, std::uint32_t,std::int64_t, std::uint64_t,double;
- Strings: std::string
- C++17 specific: std::optional (or std::experimental::optional)
Example:
structsupported_types_t{bool m_bool{false };std::int8_t m_int8{};std::uint8_t m_uint8{};std::int16_t m_int16{};std::uint16_t m_uint16{};std::int32_t m_int32{};std::uint32_t m_uint32{};std::int64_t m_int64{};std::uint64_t m_uint64{};double m_double{};std::string m_string{};};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,supported_types_t & obj){io &json_dto::mandatory("bool", obj.m_bool)&json_dto::mandatory("int8", obj.m_int8)&json_dto::mandatory("uint8", obj.m_uint8)&json_dto::mandatory("int16", obj.m_int16)&json_dto::mandatory("uint16", obj.m_uint16)&json_dto::mandatory("int32", obj.m_int32)&json_dto::mandatory("uint32", obj.m_uint32)&json_dto::mandatory("int64", obj.m_int64)&json_dto::mandatory("uint64", obj.m_uint64)&json_dto::mandatory("double", obj.m_double)&json_dto::mandatory("string", obj.m_string);}}/* namespace json_dto*/
Each data member (at least those of them which are considered to be present in JSON)in C++ struct binds to JSON field. Bind can be mandatory or optional.Optional bind is extended with default value, but it is also possibleto set optional fields without defaults.Also it is possible to add a value validator to the bind.
Binds are created bymandatory(),optional() andoptional_no_default() functions. These functions returns afield binder.Binder is an instantiation ofbinder_t template classwhich carries a part of internal logiccapable for handling field input/output operations.With the help ofbindersJson_Io object understands how read, write and validatethe underlying field.
Binders for mandatory fields are created viamandatory() function:
template<typename Field_Type,typename Validator =empty_validator_t>automandatory(string_ref_t field_name,Field_Type & field,Validator validator = Validator{});// Since v.0.2.10template<typename Reader_Writer,typename Field_Type,typename Validator =empty_validator_t>automandatory(Reader_Writer reader_writer,string_ref_t field_name,Field_Type & field,Validator validator = Validator{});
The parameterfield_name is of typestring_ref_twhich is an alias forrapidjson::Value::StringRefType.Typically it is enough to passstd::string orchar * args(seerapidjsondocumentationfor further details).The parameterfield is a reference to the instance of the field value.The parametervalidator is optional and it sets validator on fields value.Validators will be described later. By defaultempty_validator_tis used, and as it says it does nothing.
Binders for optional fields are created viaoptional() andoptional_no_default() functions:
template<typename Field_Type,typename Field_Default_Value_Type,typename Validator =empty_validator_t>autooptional(string_ref_t field_name,Field_Type & field,Field_Default_Value_Type default_value,Validator validator = Validator{});template<typename Field_Type,typename Validator =empty_validator_t>autooptional_no_default(string_ref_t field_name,Field_Type & field,Validator validator = Validator{});// Since v.0.2.10template<typename Reader_Writer,typename Field_Type,typename Field_Default_Value_Type,typename Validator =empty_validator_t>autooptional(Reader_Writer reader_writer,string_ref_t field_name,Field_Type & field,Field_Default_Value_Type default_value,Validator validator = Validator{});// Since v.0.2.10template<typename Reader_Writer,typename Field_Type,typename Validator =empty_validator_t>autooptional_no_default(Reader_Writer reader_writer,string_ref_t field_name,Field_Type & field,Validator validator = Validator{});
Parameters for functions are pretty much the same as formandatory() functon.
The only difference is the third parameter foroptional() function,it defines default value for a field if it is not defined in JSON.
In case of reading DTO, if optional field has default valueand JSON object doesn't define this field then default value is used.In case of writing DTO, if value equals to defaultthen this field wouldn't be included in JSON.
Foroptional() there is a partial specification that acceptsnullptr argument asdefault_value parameter, it is usefull fornullable_t<T> fields.
Example of using optional fields:
structmessage_t{std::string m_from;std::int64_t m_when;std::string m_text;std::string m_text_format;bool m_is_private{false };};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,message_t & msg){io &json_dto::mandatory("from", msg.m_from)&json_dto::mandatory("when", msg.m_when)&json_dto::mandatory("text", msg.m_text)&json_dto::optional("text_format", msg.m_text_format,"text/plain")&json_dto::optional_no_default("is_private", msg.m_is_private);}}/* namespace json_dto*/
Since v.0.2 it is possible to use C++17'sstd::optional template as a typefor field. In this casestd::nullopt can be passed as third argument tojson_dto::optional() function:
structemail_data_t{std::string m_from;std::string m_to;std::string m_subject;std::optional<std::vector<std::string>> m_cc;std::optional<std::vector<std::string>> m_bcc;...template<typename Json_Io>voidjson_io(Json_Io & io){io &json_dto::mandatory("from", m_from)&json_dto::mandatory("to", m_to)&json_dto::mandatory("subject", m_subject)&json_dto::optional("cc", m_cc, std::nullopt)&json_dto::optional("bcc", m_bcc, std::nullopt)...}};
Note. If a compiler doesn't havestd::optional but havestd::experimental::optional thenstd::experimental::optional andstd::experimental::nullopt can be used.
JSON arrays are supported byjson_dto, but there is one very importantlimitation: all elements of the array must have the same type.To set up an array simply usestd::vector<T>.If DTO member is ofstd::vector<T> type,then corresponding JSON field is considered to be an array.While for output the elements of the array-field will be automaticallyof the same type, for successful input it is mandatorythat all elements of the array are convertible to vector value type.
Example for array-fields:
structvector_types_t{std::vector<bool> m_bool{};std::vector<std::int8_t> m_int8{};std::vector<std::uint8_t> m_uint8{};std::vector<std::int16_t> m_int16{};std::vector<std::uint16_t> m_uint16{};std::vector<std::int32_t> m_int32{};std::vector<std::uint32_t> m_uint32{};std::vector<std::int64_t> m_int64{};std::vector<std::uint64_t> m_uint64{};std::vector<double> m_double{};std::vector<std::string> m_string{};};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,vector_types_t & obj){io &json_dto::mandatory("bool", obj.m_bool)&json_dto::mandatory("int8", obj.m_int8)&json_dto::mandatory("uint8", obj.m_uint8)&json_dto::mandatory("int16", obj.m_int16)&json_dto::mandatory("uint16", obj.m_uint16)&json_dto::mandatory("int32", obj.m_int32)&json_dto::mandatory("uint32", obj.m_uint32)&json_dto::mandatory("int64", obj.m_int64)&json_dto::mandatory("uint64", obj.m_uint64)&json_dto::mandatory("double", obj.m_double)&json_dto::mandatory("string", obj.m_string);}}/* namespace json_dto*/
Since v.0.2.6 it is possible to serialize array of objects into JSONbyjson_dto::to_json function. It is also possible to deserializeJSON with array of objects intostd::vector byjson_dto::from_jsonfunction. For example:
#include<json_dto/pub.hpp>#include<iostream>#include<algorithm>structdata_t {std::string m_key;int m_value;template<typename Json_Io>voidjson_io(Json_Io & io) {io &json_dto::mandatory("key", m_key)&json_dto::mandatory("value", m_value);}};intmain() {const std::string json_data{R"JSON([{"key":"first", "value":32}, {"key":"second", "value":15}, {"key":"third", "value":80}])JSON"};auto data = json_dto::from_json< std::vector<data_t> >(json_data);std::sort(data.begin(), data.end(),[](constauto & a,constauto & b) {return a.m_value < b.m_value; });std::cout <<"Sorted data:" <<json_dto::to_json(data) << std::endl;}
Since v.0.2.8 there is a support for STL containers likestd::deque,std::list,std::forward_list,std::set,std::unordered_set,std::map andstd::unordered_map.Those types can be used as types of fields of serialized struct/classes:
#include<json_dto/pub.hpp>#include<deque>#include<set>#include<map>structmy_message {std::deque<int> ids_;std::set<std::string> tags_;std::map<std::string, some_another_type> props_;...template<typename Json_Io>voidjson_io(Json_Io & io) {io &json_dto::mandatory("ids", ids_)&json_dto::mandatory("tags", tags_)&json_dto::mandatory("properties", props_)...;}};
Also STL containers are supported byjson_dto::from_json() andjson_dto::to_json() functions:
auto messages = json_dto::from_json< std::forward_list<my_message> >(...);...auto json = json_dto::to_json(messages);
See a special example with usage of STL containers
Note that support for those STL-containers is not hardcoded in json_dto.Instead, json_dto tries to detect a type of a container by inspecting the presence of typeslikevalue_type,key_type,mapped_type and methods likebegin()/end(),emplace(),emplace_back() and so on. It means that json_dto may work not only with STL-containers butwith other containers those mimics like STL-containers.
Note. Typestd::array is not supported now. If you have to deal withstd::array andwant to have a support of it in json_dto pleaseopen an issueand we'll discuss some corner cases related tostd::array.
An important note about support forstd::multiset,std::unordered_multiset,std::multimap andstd::unordered_multimap: those containers are also supported.But json_dto doesn't do any checks for duplicate keys. In that aspect, json_dto relieson RapidJSON behavior. For example, if an instance ofstd::multimap contains severalvalues for some key all those values will be serialized.What happens to those values is dependent on RapidJSON.
To support JSON null values,json_dto introducesnullable_t<T>.It is required that nullable field is explicitly defined asdata member of typenullable_t<T>.
Interface ofnullable_t<T> tries to mimicstd::optional interface.
Example fornullable_t<T> field:
structmessage_t{message_t() {}message_t(std::string from,std::int64_t when,std::string text):m_from{std::move(from) },m_when{ when },m_text{std::move(text) }{}std::string m_from;std::int64_t m_when;std::string m_text;// Log level.// By default is constructed with null value.json_dto::nullable_t<std::int32_t> m_log_level{};};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,message_t & msg){io &json_dto::mandatory("from", msg.m_from)&json_dto::mandatory("when", msg.m_when)&json_dto::mandatory("text", msg.m_text)&json_dto::optional("log_level", msg.m_log_level,nullptr);}}/* namespace json_dto*/voidsome_function( ... ){// ...auto msg = json_dto::from_json<message_t>(json_data);// ...// If field is defined then its value can be obtained and used.if( msg.m_log_level )use_value(*msg.m_log_level);// ...msg.m_log_level =1;// Set new value.// ...// equivalent to msg.m_log_level.reset();msg.m_log_level =nullptr;// Reset value.// ...}
Here default value for optional nullble field isnullptr.And it means that absence of value is a default state for a field.So when converting to JSON no-value nullable fieldwouldn't be included in JSON as"field":null piece.
Nullable fields can be used with arrays:
structmessage_t{message_t() {}message_t(std::string from,std::int64_t when,std::string text):m_from{std::move(from) },m_when{ when },m_text{std::move(text) }{}// Who sent a message.std::string m_from;// When the message was sent (unixtime).std::int64_t m_when;// Message text.std::string m_text;// Log level.// By default is constructed with null value.json_dto::nullable_t<std::int32_t> m_log_level{};json_dto::nullable_t< std::vector<std::string> > m_tags{};};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,message_t & msg){io &json_dto::mandatory("from", msg.m_from)&json_dto::mandatory("when", msg.m_when)&json_dto::mandatory("text", msg.m_text)&json_dto::optional("log_level", msg.m_log_level,nullptr)&json_dto::optional("tags", msg.m_tags,nullptr);}}/* namespace json_dto*/voidsome_function( ... ){// ...auto msg = json_dto::from_json<message_t>(json_data);// ...if( msg.m_tags )use_tags(*msg.m_tags);// ...}voidsome_other_function( ... ){message_t msg{ ... };// ...// Add tags:msg.m_tags.emplace();// equivalent to msg = std::vector<std::string>{};msg.m_tags->emplace_back("sample");msg.m_tags->emplace_back("tutorial");// ...}
json_dto allows to construct complex types with nested objects.Using nested objects is pretty much the same as using data of a simple types.Nested objects can be optional, nullable and be elements of array-fields.However there are some constraints:
- nested type must be itself integrated withjson_dto;
- type must be default-constructible (for input);
- for optional fields with default value equality operator must be defined(more precisely an equality operator between nested type and type of passed default value).
Suppose there is a type which is already integrated withjson_dto:
structmessage_source_t{std::int32_t m_thread_id{0 };std::string m_subsystem{};template<typename Json_Io>voidjson_io(Json_Io & io){io &json_dto::optional("thread_id", m_thread_id,0)&json_dto::mandatory("subsystem", m_subsystem);}};
Then it can be used as a nested object in other type:
structmessage_t{message_source_t m_from;std::int64_t m_when;std::string m_text;template<typename Json_Io>voidjson_io(Json_Io & io){io &json_dto::mandatory("from", m_from)// Exactly as with simple types.&json_dto::mandatory("when", m_when)&json_dto::mandatory("text", m_text);}};
And see full example using nested objects as nullable and arrays
json_dto works well with inheritance. It is possible to usebase implementation ofjson_io() function or completely override it.
For example derived class can use base class like this:
structderived_t :publicbase_t{//...template<typename Json_Io>voidjson_io(Json_Io & io){base_t::json_io(io);// Run io on base class.// Run io on extra data:io &json_dto::mandatory("some_field", m_some_field)// ...;}};
However for easier maintenance it is recommended to use non intrusivejson_io() function. Because if base class is integrated withjson_dtoin non intrusive manner, then the following wouldn't work:
template<typename Json_Io>voidjson_io(Json_Io & io){// Base class doesn't provide such member function.base_t::json_io(io);// Run io on base class.// ...}
So it is preferred to put inheritance this way:
structmessage_source_t{std::int32_t m_thread_id{0 };std::string m_subsystem{};};namespacejson_dto{template<typename Json_Io>voidjson_io(Json_Io & io,message_source_t & m){io &json_dto::optional("thread_id", m.m_thread_id,0)&json_dto::mandatory("subsystem", m.m_subsystem);}}/* namespace json_dto*/structmessage_t :publicmessage_source_t{std::int64_t m_when;std::string m_text;template<typename Json_Io>voidjson_io(Json_Io & io){json_dto::json_io(io,static_cast<message_source_t &>(*this));io &json_dto::mandatory("when", m_when)&json_dto::mandatory("text", m_text);}};
json_dto allows to set validator on each field.Validator is a function object (an object of a type supporting anoperator() member function) that receives a single parameter.
When handling inputjson_dto calls specified validatorand passes resulting field value as an argument.If validator returns without throwing exception,then field value considered to be valid, and execution continues.Otherwise exception is catched and another will be thrown:json_dto::ex_t. This exeption contains original exception descriptionsupplemented with field name information.
When handling ouputjson_dto calls specified validator beforetrying to assign field value of JSON object. In all other respectsvalidation is the same as for input.
A simple example of using validators:
voidcheck_all_7bit(const std::string & text){constauto it =std::find_if(std::begin(text),std::end(text),[](char c){return c &0x80; });if(std::end(text) != it ){throw std::runtime_error{"non 7bit char at pos" +std::to_string(std::distance(std::begin(text), it)) };}}structmessage_t{std::string m_from;std::int64_t m_when;// Message text. Must be 7bit ascii.std::string m_text;template<typename Json_Io>voidjson_io(Json_Io & io){io &json_dto::mandatory("from", m_from)&json_dto::mandatory("when", m_when)&json_dto::mandatory("text", m_text, check_all_7bit);}};
json_dto comes with some useful ready to use validators for simple types.They are defined in<json_dto/pub.hpp> header.
Standard validators curently available:
min_max_constraint_t<Num>- range validator, targeted for numeric types;one_of_validator_t<T>- validator for set of values.
Standard validators are template classes with overloadedoperator().And as they are template classes so for conveniencefor each validator there is an auxiliary function that helps deducetype of template instance from arguments:
template<typename Number>automin_max_constraint(Number min_value, Number max_value);template<typename Field_Type>autoone_of_constraint(std::initializer_list<Field_Type> values);
See full example with standard validators
Sometimes several values may be stored inside an array:
{"x": [1,"Hello!",0.3] }Since v.0.3.3 json_dto support such cases the following way:
structinner {int a;std::string b;double c;//NOTE: there is no json_io for `inner` type.};structouter {inner x;template<typename Io >voidjson_io( Io & io ) {io &json_dto::mandatory(// The use of special reader-writer that will pack// all described fields into one array value.json_dto::inside_array::reader_writer(// All fields to be (de)serialized must be enumerated here.// The order of the enumeration is important: (de)serialization// is performed in this order.json_dto::inside_array::member( x.a ),json_dto::inside_array::member( x.b ),json_dto::inside_array::member( x.c ) ),"x", x );}};...auto obj = json_dto::from_json<outer>(R"({"x":[1, "Hello!", 0.3]})" );
By defaultjson_dto::inside_array::reader_writer expects exact number offields. For example, if three fields are described then an array with exactly treevalues is expected during deserialization, otherwise an exception will be thrown.However, there may be cases when JSON contains less values. It means that Nfirst members are mandatory, and all other are optional. This can be expressedthat way:
structouter {inner x;template<typename Io >voidjson_io( Io & io ) {io &json_dto::mandatory(json_dto::inside_array::reader_writer<// Specify that the first field is mandatory and all remaining// are optional.json_dto::inside_array::at_least<1>>(// This is mandatory field,json_dto::inside_array::member( x.a ),// This is optional field and we specify a default value for// a case when it's missing.json_dto::inside_array::member_with_default_value( x.b, std::string{"Nothing" } ),// This is optional field and we doesn't specify a default value// for it. It it's missing than `double{}` will be used as a new// value for `x.c`.json_dto::inside_array::member( x.c ) ),"x", x );}};
There are a family ofjson_dto::inside_array::member andjson_dto::inside_array::member_with_default_value functions that allow to specify a custom reader-writer and/or a validator:
json_dto::inside_array::member( x.a, json_dto::min_max_constraint(-10,10));json_dtp::inside_array::member( my_custom_reader_writer{}, x.b );json_dtp::inside_array::member( my_custom_reader_writer{}, x.c, json_dto::min_max_constraint(-1,1) );
The inside_array functionality can be used for manual support ofstd::tuple:
structouter {std::tuple<int, std::string,double> x;template<typename Io >voidjson_io( Io & io ) {io &json_dto::mandatory(json_dto::inside_array::reader_writer(json_dto::inside_array::member( std::get<0>(x) ),json_dto::inside_array::member( std::get<1>(x) ),json_dto::inside_array::member( std::get<2>(x) ) ),"x", x );}};...auto obj = json_dto::from_json<outer>(R"({"x":[1, "Hello!", 0.3]})" );
It is possible to define custom IO logic for a specific type.It might be useful for types when using object is an overkill,for example time point that can be stored in format of 'YYYY.MM.DD hh:mm:ss'or some token composed of several small items like '--'.But introducing custom IO logic for some typerequires to work withrapidjson API directly.
There are two way to introduce custom IO logic.
The first way uses C++'s Argument Dependent Lookup feature:an user should defineread_json_value andwrite_json_value in the samenamespace where types are defined. The right implementations ofread_json_value andwrite_json_value will be found by C++ compiler automatically.For example:
namespaceimportance_levels{enumclasslevel_t{low,normal,high};// read_json_value and write_json_value for level_t are// defined in importance_levels namespace.// They will be found by argument dependent lookup.voidread_json_value(level_t & value,const rapidjson::Value & from){...}voidwrite_json_value(constlevel_t & value,rapidjson::Value & object,rapidjson::MemoryPoolAllocator<> & allocator){...}}/* namespace importance_levels*/
This approach also allows to defineread_json_value andwrite_json_valuefor user's template type. For example:
namespacedemo{template<typename T>classsome_template{...}template<typename T>voidread_json_value(some_template<T> & value,const rapidjson::Value & from){...}template<typename T>voidwrite_json_value(const some_template<T> & value,rapidjson::Value & object,rapidjson::MemoryPoolAllocator<> & allocator){...}}/* namespace demo*/structmy_data_t{demo::some_template<int> m_first;demo::some_template<double> m_second;...template<typename Json_Io>voidjson_io(Json_Io & io){io &json_dto::mandatory("first", m_first)&json_dto::mandatory("second", m_second)...}};
See full example with custom IO and ADL
The second way uses explicit template specialization for 2 functonsinsidejson_dto namespace:
namespacejson_dto{template<>voidread_json_value(Custom_Type & v,const rapidjson::Value & object){// ...}template<>voidwrite_json_value(const Custom_Type & v,rapidjson::Value & object,rapidjson::MemoryPoolAllocator<> & allocator){// ...}}/* namespace json_dto*/
json_dto will consider these specializations for using withspecifiedCustom_Type. This way can be used when it is impossibleto placeread_json_value andwrite_json_value into the namespace wherethe type if defined (for example if it is standard type likestd::filesystem::path).
See full example with custom IO
Suppose we have an enumerationlog_level defined such way:
enumclasslog_level { low, normal, high };
And we have two structs that use thatlog_level enumeration:
structlog_message{log_level level_;std::string msg_;};structlog_config{std::string path_;log_level level_;};
Serialization oflog_level to JSON should use numeric values of log levels, e.g.:{"level":0, "msg":"..."}, but the serialization oflog_config should use textual names instead of numeric values, e.g.:{"path":"/var/log/demo", "level":"low"}.
Such a task can't be implemented by writing overloads ofread_json_value andwrite_json_value functions. Custom Reader_Writers should be used in that case:
structnumeric_log_level{voidread( log_level & v,const rapidjson::Value & from )const{using json_dto::read_json_value;int actual;read_json_value( actual, from );v =static_cast<log_level>(actual);}voidwrite(const log_level & v,rapidjson::Value & to,rapidjson::MemoryPoolAllocator<> & allocator )const{using json_dto::write_json_value;constint actual =static_cast<int>(v);write_json_value( actual, to, allocator );}};structlog_message{log_level level_;std::string msg_;template<typename Json_Io >voidjson_io( Json_Io & io ){io &json_dto::mandatory( numeric_log_level{},"level", level_ )&json_dto::mandatory("msg", msg_ );}};structtextual_log_level{voidread( log_level & v,const rapidjson::Value & from )const{using json_dto::read_json_value;std::string str_v;read_json_value( str_v, from );if("low" == str_v ) v = log_level::low;elseif("normal" == str_v ) v = log_level::normal;elseif("high" == str_v ) v = log_level::high;elsethrow json_dto::ex_t{"invalid value for log_level" };}voidwrite(const log_level & v,rapidjson::Value & to,rapidjson::MemoryPoolAllocator<> & allocator )const{using json_dto::write_json_value;using json_dto::string_ref_t;switch( v ){case log_level::low:write_json_value(string_ref_t{"low" }, to, allocator );break;case log_level::normal:write_json_value(string_ref_t{"normal" }, to, allocator );break;case log_level::high:write_json_value(string_ref_t{"high" }, to, allocator );break;}}};structlog_config{std::string path_;log_level level_;template<typename Json_Io >voidjson_io( Json_Io & io ){io &json_dto::mandatory("path", path_ )&json_dto::mandatory( textual_log_level{},"level", level_ );}};
Note that Reader_Writer class should have two const methodsread andwrite those signatures are the same with the signatures ofread_json_value andwrite_json_value functions.
See full example with Reader_Writer
Custom Reader_Writer classes can also be used for handling non-standard representation of some values in JSON document. For example, sometimes string-values like"NAN" or"nan" are used for NaN (Not-a-Number) values. RapidJSON can only parsed special valueNaN, but not"NAN" nor"nan" values. In such case a custom Reader_Writer like the following one can be used:
structcustom_floating_point_reader_writer{template<typename T >voidread( T & v,const rapidjson::Value & from )const{if( from.IsNumber() ){json_dto::read_json_value( v, from );return;}elseif( from.IsString() ){const json_dto::string_ref_t str_v{ from.GetString() };if(equal_caseless( str_v,"nan" ) ){v = std::numeric_limits<T>::quiet_NaN();return;}elseif(equal_caseless( str_v,"inf" ) ){v = std::numeric_limits<T>::infinity();return;}elseif(equal_caseless( str_v,"-inf" ) ){v = -std::numeric_limits<T>::infinity();return;}}throw json_dto::ex_t{"unable to parse value" };}template<typename T >voidwrite(T & v,rapidjson::Value & to,rapidjson::MemoryPoolAllocator<> & allocator )const{using json_dto::write_json_value;using json_dto::string_ref_t;if(std::isnan(v) )write_json_value(string_ref_t{"nan"}, to, allocator );elseif( v > std::numeric_limits<T>::max() )write_json_value(string_ref_t{"inf"}, to, allocator );elseif( v < std::numeric_limits<T>::min() )write_json_value(string_ref_t{"-inf"}, to, allocator );elsewrite_json_value( v, to, allocator );}};structstruct_with_floats_t{float m_num_float;double m_num_double;template<typename Json_Io >voidjson_io( Json_Io & io ){io&optional( custom_floating_point_reader_writer{},"num_float", m_num_float,0.0f )&optional( custom_floating_point_reader_writer{},"num_double", m_num_double,0.0 );}};
Note also thatread andwrite methods of Reader_Writer class can be template methods.
A custom Reader_Writer can also be used to change representation of a field. For example, let suppose that we have astd::vector<some_struct> field, but this field has to be represented as a single object if it holds just one value, and as an array otherwise. Something like:
{ "message": { "from": "address-1", "to": "address-2", ..., "extension": {...} }, ...}if we have only one extension in a message or:
{ "message": { "from": "address-1", "to": "address-2", ..., "extension": [ {...}, {...}, ... ] }, ...}if there are several extensions.
A solution with a custom Reader_Writer can looks like:
structextension{ ...template<typename Json_Io >voidjson_io( Json_Io & io ) { ...// Ordinary serialization/deserialization code. }};// Reader_Writer for vector of `extension` objects.structextension_reader_writer{voidread( std::vector< extension > & to,const rapidjson::Value & from )const {using json_dto::read_json_value; to.clear();if( from.IsObject() ) {extension_t single_value;read_json_value( single_value, from ); to.push_back(std::move(single_value) ); }elseif( from.IsArray() ) {read_json_value( to, from ); }else {throw std::runtime_error{"Unexpected format of extension value" }; } }voidwrite(const std::vector< extension > & v, rapidjson::Value & to, rapidjson::MemoryPoolAllocator<> & allocator )const {using json_dto::write_json_value;if(1u == v.size() )write_json_value( v.front(), to, allocator );elsewrite_json_value( v, to, allocator ); }};// Message.structmessage{// Fields of a message. ...// Extension(s) for a message. std::vector< extension > m_extension;template<typename Json_Io >voidjson_io( Json_Io & io ) { io & ... &json_dto::mandatory( extension_reader_writer{},"extension", m_extension ) ; }};
The full example of such an approach can be seenhere
If a custom Reader_Writer is used then a reference to the whole field is passed to Reader_Writer's methods. For example:
structmy_int_reader_writer{voidread(int & v, ...)const {...}// Custom read procedure for an int.voidwrite(constint & v, ...)const {...}// Custom write procedure for int.};...structmy_data{int field_;...template<typename Io>voidjson_io(Io & io){io &json_dto::mandatory(my_int_reader_writer{},"field", field_)...;}};
In that case a reference to anint will be passed tomy_int_reader_writer'sread andwrite methods.
In the case whenmy_data isn'tint but astd::vector<int> then a reference tostd::vector<int> instance will be passed toread/write. And there will be a compiler error becauseread/write expects a reference to anint.
If we want our custom Reader_Writer to be applied for every member of a container thenjson_dto::apply_to_content_t proxy should be used as Reader_Writer type:
structmy_complex_data{std::vector<int> field_;...template<typename Io>voidjson_io(Io & io){io &json_dto::mandatory(json_dto::apply_to_content_t<my_int_reader_writer>{},"field", field_)...;}};
Theapply_to_content_t proxy works very simple way: it holds an instance of an actual Reader_Writer and applies that actual Reader_Writer to every member of a container (or to the content ofjson_dto::nullable_t andstd::optional, see bellow).
The same rule is applied tonullable_t andstd::optional:
structmy_data{std::optional<int> field_;...template<typename Io>voidjson_io(Io & io){io &json_dto::optional(my_int_reader_writer{},"field", field_, std::nullopt)...;}};
Such code leads to compiler error becausemy_int_reader_writer'sread andwrite methods expect a reference toint, not tostd::optional<int>. So we have to useapply_to_content_t here too:
structmy_data{std::optional<int> field_;...template<typename Io>voidjson_io(Io & io){io &json_dto::optional(// Now my_int_reader_writer will be applied to the content// of std::optional<int>, not to std::optional<int> itself.json_dto::apply_to_content_t<my_int_reader_writer>{},"field", field_, std::nullopt)...;}};
Note thatapply_to_content_t can be nested:
structmy_complex_data {json_dto::nullable_t< std::vector<int> > params_;...template<typename Io>voidjson_io(Io & io) {io &json_dto::mandatory(// The first occurence of apply_to_content_t is for nullable_t.json_dto::apply_to_content_t<// The second occurence of apply_to_content_t is for std::vector.json_dto::apply_to_content_t<// This is for the content of std::vector.my_int_reader_writer>>{},"params", params_)...;}};
Since v.0.2.11 json_dto (de)serializes keys of map-like containers (std::map,std::multimap,std::unordered_map and so on) by using new proxy typesconst_map_key_t andmutable_map_key_t.
A new typeconst_map_key_t<T> is used for serializing a key of type T.
A new typemutable_map_key_t<T> is used for deserializing a key of type T.
It means that if someone wants to make overloads ofread_json_value andwrite_json_value for types that are used as keys in map-like structures, then such overloads should be placed intojson_dto namespace and should have the following prototypes:
namespacejson_dto {voidread_json_value(mutable_map_key_t<UserType> key,const rapidjson::Value & from);voidwrite_json_value(const_map_key_t<UserType> key,rapidjson::Value & to,rapidjson::MemoryPoolAllocator<> & allocator);}/* namespace json_dto*/
See full example with overloading of read/write_json_value for mutable/const_map_key_t
The addition ofmutable_map_key_t/const_map_key_t in the v.0.2.11 means that custom Reader_Writers should take the presence of those types into the account.
For example, if a custom Reader_Writer is used for (de)serializing a content ofstd::map then that Reader_Writer should have implementations ofread/write methods for keys and values from the map:
structmy_kv_formatter{// Read a key.voidread(json_dto::mutable_map_key_t<KeyType> & key,const rapidjson::Value & from)const {...}// Read a value.voidread(ValueType & value,const rapidjson::Value & from)const {...}// Write a key.voidwrite(const json_dto::const_map_key_t<KeyType> & key,rapidjson::Value & to,rapidjson::MemoryPoolAllocator<> & allocator)const {...}// Write a value.voidwrite(const ValueType & value,rapidjson::Value & to,rapidjson::MemoryPoolAllocator<> & allocator)const {...}};
Please note that a references to instances ofmutable_map_key_t/const_map_key_t are passed toread/write methods.
See full example with overloading of Reader_Writer for mutable/const_map_key_t
json_dto is distributed underBSD-3-Clause license. See LICENSEfile for more information.
For the license ofrapidson library see LICENSE file inrapidsondistributive.
For the license ofrapidson_mxxru library see LICENSE file inrapidson_mxxrudistributive.
For the license ofCATCH library see LICENSE file inCATCHdistributive.
About
A small header-only library for converting data between json representation and c++ structs
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.