Repository files navigation

• Intro
• Default Mapping of Types
• API Documentation - Member mapping classes and methods
• Cookbook Get cooking and putting it all together
  • Mapping Overview
  • Aliases
  • Arrays
  • Classes
  • Classes from Array/JSON Tuples
  • Dates
  • Enums
  • Graphs
  • Inserting and Extracting Raw JSON
  • JSON Lines/NDJSON
  • JSON Schema Output
  • Key Values - Map and Dictionary like things
  • Mapping Deduction
  • Member Options - Options for the parse mappings
  • Nullable Concept - Trait for mapping Option/Nullable types used in deduction
  • Nullable JSON Values
  • Numbers
  • Output Options - Options for serialization
  • Parser Options - Options for parsing
  • Parsing Individual Members
  • Reflection(New)
  • Strings
  • Unknown JSON and Raw Parsing - Browsing the JSON Document and delaying of parsing of specified members
  • Variant
  • Automatic Code Generation
• Intro
• Installing/Using
• Performance considerations
• Benchmarks
• Escaping/Unescaping of member names
• Differences between C++17 and C++20
  • C++ 17 Naming of members
  • C++ 20 Naming of members
• Using data types
• Error Handling
• Parsing call
• Global
• Deserializing/Parsing
• Member Paths
• Serialization
• Build Configuration Points
• Requirements
• For building tests
• Limitations

The JSON Link library is a high performance, no allocation, C++ JSON Library supporting:

• Direct parsing of JSON to your own data structures via simple declarative mappings
• Earlier error reporting of unexpected data and errors
• Low memory requirement, parser requires a small amount of stack but does not allocate itself.
• Optimizations based on data types
• Does not require owning the mapped types and is not intrusive

The library supports other parsing modes that can be mixed too.

• Direct parsing into data structures
• A lazy(PULL) JSON dom parser that allows for exploration of the document and selective parsing
• A SAX(PUSH) event based parser for tasks like minimization
• For large JSON array documents, one can use thejson_array_iterator orjson_array_range types.
• For dynamic situations, ajson_value type that allows iteration/lazy parsing of document

Some other notable features are:

• Support for JSON tuple(heterogeneous arrays) and variant/tagged variant like mappings.
• Automatic mapping of Containers, including the Map based Associative Structures
• Able to parse to/from other numeric types likeboost::multiprecision::cpp_int or GNU BigNum/Rationalmpq_t
• Optional Comment support
• Trailing comma support
• JSON Lines/NDJSON support with a thread partitioning helper
• JSON Schema output support
• Easy integration with reflection like libraries e.gBoost.Describe Support
• JSON Path support
• Minified and optional formatted output

The library is using theBSL license

When the structure of the JSON document is known, parsing is like the following:

MyThing thing = daw::json::from_json<MyThing>( json_string );

or for array documents, where the root of the document is an array, there is a helper method to make it easier, and it can be parsed like the following:

std::vector<MyThing> things = daw::json::from_json_array<MyThing>( json_string2 );

If the structure of the JSON document is unknown, one can construct ajson_value that acts as a view and allows iteration and pull parsing on demand. The following is an example of opening ajson_value from JSON data:

json_value val = daw::json::json_value( json_string );

Thefrom_json andto_json methods allow access most of the parsing and serializing needs.

The event based parser(SAX) can be called viadaw::json::json_event_parser. It takes two arguments, a json document and an event handler. The event handler can opt into events by having any of the following members:

• handle_on_value
• handle_on_array_start
• handle_on_array_end
• handle_on_class_start
• handle_on_class_end
• handle_on_number
• handle_on_bool
• handle_on_string
• handle_on_null
• handle_on_error

• TheCookbook section has pre-canned tasks and working code examples
• Tests provide another source of working code samples.
• Some video walkthroughs:
  • Making a config parser
  • I Like BigInt's
• Links to other examples
  • Parsing a Config File
  • Parsing BigInt/Multiprecision Numbers
• Small samples below

Mapping of your classes to JSON documents is done by specializing the traitdaw::json::json_data_contract. A class that is mapped does not need to be mapped again if it is a member of another mapped class.There are two parts to the traitjson_data_contract, first is a type alias namedtype that maps the JSON members to our class's constructor. This gets around needing private access to the class, assuming that data we would serialize would also be needed to construct the class. For example:

structThing {int a;int b;};

The construct forThing requires 2 integers and if we had the following JSON:

{"a":42,"b":1234}

We could do the mapping like the following:

namespacedaw::json {template<>structjson_data_contract<Thing> {staticconstexprcharconst a[] ="a";staticconstexprcharconst b[] ="b";using type = json_member_list<      json_number<a,int>,      json_number<b,int>    >;  };}

This says that the JSON class, in the document, will have at least two members "a", and "b" that will be numbers that are integers. They will be passed to the constructor ofThing whendaw::json::from_json<Thing>( json_doc ); is called, or that another class has ajson_class<MemberName, Thing> member mapping. The above is the C++17 mapping method for the names, it works in future C++ versions too. But, in C++20 and later the names can be inline in the mapping e.g.json_number<"a", int>.The above is all that is needed for parsing JSON, for serializing a static member function is needed in the trait. Taking the previous example and extending it we could serializeThing with:

namespacedaw::json {template<>structjson_data_contract<Thing> {staticconstexprcharconst a[] ="a";staticconstexprcharconst b[] ="b";using type = json_member_list<      json_number<a,int>,      json_number<b,int>    >;  };staticautoto_json_data( Thingconst & v ) {returnstd::forward_as_tuple( v.a, v.b );  }}

The ordering of the members returned as a tuple need to match the mapping in the type aliastype. This allows for passing the result of accessor methods too, if the data members are not public. Also, the classThing must be constructible fromint, int. The library supports both regular constructors and aggregate init (Thing{ int, int } andThing( int, int ) ) in C++17.

• Note: The return type ofto_json_data does not have to return a tuple of references to the existing object members, but can return calculated values too. It does not allow rvalues through as they are often temporaries, and it can result in long distance debugging. The library will static_assert on this and suggest including<daw/daw_tuple_forward.h> and callingdaw::forward_nonrvalue_as_tuple( ... ) which store temporaries and forward other reference types.

The parsers work by constructing each argument in place in the call to the class's constructor. The individual argument parsers can be tuned for the specified circumstances of the data(e.g. floating point and integral numbers). Then with our type trait defining the arguments needed to construct the C++ class and their order we are able to look at each member in the JSON. Now we construct the value with the result of each parser; similar toT{ parse<0, json_string<"name">>( data ), parse<1, json_number<"age", unsigned>>( data ), parse<json_number<2, "number>>( data )}. For each member, the data stream will be moved forward until we find the member we need to parse, storing interested locations for later parsing. This process allows us to parse other classes as members too via thejson_class<"member_name", Type> mapping type. So that each mapping trait only has to deal with its specific members and not their details.

In unnamed contexts, such as the root value, array elements, some key value types, and variant element lists where the name would beno_name, one can use some native C++ data types instead of the JSON mapping types. This includes, integer, floating point, bool, std::string, std::string_view, associative containers, sequence containers, Nullable/Optional like types and previously mapped classes.

For example, to map an array of string's.

template<>structdaw::json::json_data_contract<MyType> {using type = json_member_list<json_array<"member_name", std::string>>;};

One can usevcpkg to grab the latest release, the port is calleddaw-json-link

find_package( daw-json-link )#...target_link_libraries( MyTarget daw::daw-json-link )

The library is header only and can be cloned, along with it's two dependencies, followed by adding theinclude/ subfolders of each to the compiler's include path

To use daw_json_link in your cmake projects, adding the following should allow it to pull it in along with the dependencies:

include( FetchContent )FetchContent_Declare(  daw_json_link  GIT_REPOSITORY https://github.com/beached/daw_json_link  GIT_TAG release)FetchContent_MakeAvailable(daw_json_link)#...target_link_libraries( MyTarget daw::daw-json-link )

On a system with bash, it is similar on other systems too, the following can install for the system

git clone https://github.com/beached/daw_json_linkcd daw_json_linkmkdir buildcd buildcmake ..cmake --install.

This will allow for a cmake find_package install or using it as a regular header as long as the install prefix's include folder is included in the include paths of the compiler

The following will build and run the tests.

git clone https://github.com/beached/daw_json_linkcd daw_json_linkmkdir buildcd buildcmake -DDAW_ENABLE_TESTING=On ..cmake --build. ctest.

After the build there the individual examples can be tested too.city_test_bin requires the path to the cities JSON file.

./tests/city_test_bin ../test_data/cities.json

The order of the members in the data structures should generally match that of the JSON data, if possible. The parser is faster if it doesn't have to back track for values. Optional values, when missing in the JSON data, can slow down the parsing too. If possible have them sent as null. The parser does not allocate. The parsed to data types may and this allows one to use custom allocators or a mix as their data structures will do the allocation. The defaults for arrays is to use the std::vector and if this isn't desirable, you must supply the type.

• Kostya results usingtest_dawjsonlink.cpp SeeKostya Benchmarks for latest results.

The library, currently, does not unescape/escape member names when serializing, they are expected to be valid and unescaped. This may be a future optional addition, as it does have a cost.

There are slight differences between C++17 and C++20 where C++20 allows for some code not available in C++17.

namespacedaw::json {template<>structjson_data_contract<MyType> {staticconstexprcharconst member_name[] ="memberName";using type = json_member_list<json_number<member_name>>;  };}

Both versions of C++ support this method for naming members.

When compiled within C++20 compiler, in addition to passing achar const * as in C++17, the member names can be specified as string literals directly. C++20 compiler support is still really early and here be dragons. There are known issues with g++9.x in C++20 mode, and it's only tested with g++10/11. Here be dragons

namespacedaw::json {template<>structjson_data_contract<MyType> {using type = json_member_list<json_number<"member_name">>;  };}

Once a data type has been mapped with ajson_data_contract, the library provides methods to parse JSON to them

MyClass my_class = from_json<MyClass>( json_str );

Alternatively, if the input is trusted, the less checked version can be faster

MyClass my_class = from_json<MyClass, options::parse_flags<options::CheckedParseMode::no>>( json_str );

JSON documents with array root's use thefrom_json_array function to parse

std::vector<MyClass> my_data = from_json_array<MyClass>( json_str );

Alternatively, if the input is trusted, the less checked version can be faster

std::vector<MyClass> my_data = from_json_array<MyClass, std::vector<MyClass>, options::parse_flags<options::CheckedParseMode::no>>( json_str );

If you want to work from JSON array data you can get an iterator and use the std algorithms toIterating over array's in JSON data can be done via thejson_array_iterator

usingiterator_t = json_array_iterator<MyClass>;auto pos = std::find(iterator_t( json_str ),iterator_t( ), MyClass( ... ) );

Alternatively, if the input is trusted you can call the less checked version

usingiterator_t = daw::json::json_array_iterator<MyClass, options::CheckedParseMode::no>;auto pos = std::find(iterator_t( json_str ),iterator_t( ), MyClass( ... ) );

For a DOM like api, often used for things like GUI's and providing code when the mappings are inadequate, one can usejson_value. This is used in thejson_to_cpp tool.

auto jv = daw::json::json_value( json_doc );

One can use a JSON Path to extract an integer

int foo = as<int>( jv["path.to.int"] );

Here,"path.to.int" is a JSON Path that represents drilling into a JSON class like

{"path": {"to": {"int":5    }  }}

One can, also, select via an array like syntax in the JSON path too,"path[5]" would select the 5th element/member of"path".If you want to serialize to JSON. The JSON Path syntax works withfrom_json,from_json_array, andjson_array_iterator too.

std::string my_json_data = to_json( MyClass{} );

Or serialize an array, collection, range, or view of things. Only requiresstd::begin(...) andstd::end(...) to work for the type. This allows serialization when the type isn't a constructible collection of things.

std::vector<MyClass> arry = ...;std::string my_json_data = to_json_array( arry );

Parsing errors default to throwing adaw::json::json_exception that includes information about the reason and location of failure.

If exceptions are disabled the library will callstd::terminate upon a parse error by default.

While, the error handling defaults to throwing adaw::json::json_exception on errors, or callingstd::terminate if exceptions are disabled. One can change this behaviour by setting the function pointerdaw::json::daw_json_error_handler. The only requirement is that the function does not return. An example that utilizes this is inerror_handling_bench_test.cpp

Error checking can be modified on a per-parse basis.from_json,from_json_array,json_value,json_array_iterator, and alike all support parsing options. calls can be supplied a Parser Option. The available options are documented in theparser_policies cookbook item.

daw::json::json_exception has a member functionstd::string_view reason( ) const akin tostd::exception'swhat( ) but returns astd::string with more context thanwhat( ) does. If you want to disable exceptions in an environment that has them, you can defineDAW_JSON_DONT_USE_EXCEPTIONS to disable exception throwing by the library or set the handler, this is no longer recommended as the handler can be set to one of the two defaultsdaw::json::default_error_handling_throwing ordaw::json::default_error_handling_terminating.

This can be accomplished by writing a specialization ofjson_data_contract in thedaw::json namespace. For example:

#include<daw/json/daw_json_link.h>#include<string>#include<string_view>#include<vector>structTestClass {int i =0;double d =0.0;bool b =false;  std::string s{};  std::vector<int> y{};TestClass(int Int,double Double,bool Bool, std::string S,            std::vector<int> Y)      : i(Int), d(Double), b(Bool), s(std::move( S ) ), y(std::move( Y )) {}};namespacedaw::json {template<>structjson_data_contract<TestClass> {using type =    json_member_list<      json_number<"i",int>,       json_number<"d">,       json_bool<"b">,      json_string<"s">,       json_array<"y",int>    >;};}// namespace daw::jsonintmain() {  std::string_view test_001_t_json_data =R"({    "i":5,    "d":2.2e4,    "b":false,    "s":"hello world",    "y":[1,2,3,4]    })";  std::string_view json_array_data =R"([{    "i":5,    "d":2.2e4,    "b":false,    "s":"hello world",    "y":[1,2,3,4]    },{    "i":4,    "d":122e4,    "b":true,    "s":"goodbye world",    "y":[4,3,1,4]    }])";  TestClass test_class = daw::json::from_json<TestClass>(test_001_t_json_data);  std::vector<TestClass> arry_of_test_class =      daw::json::from_json_array<TestClass>(test_001_t_json_data);}

See on Compiler Explorer

• C++ 20 - gcc
• C++ 17 - clang

Both aggregate and user constructors are supported. The description provides the values needed to construct your type and the order. The order specified is the order they are placed into the constructor. There are customization points to provide a way of constructing your type too. A class like:

#include<daw/json/daw_json_link.h>structAggClass {int a{};double b{};};namespacedaw::json {template<>structjson_data_contract<AggClass> {using type = json_member_list<      json_number<"a",int>,      json_number<"b">    >;  };}

Works too.Same but C++17

#include<daw/json/daw_json_link.h>structAggClass {int a{};double b{};};namespacedaw::json {template<>structjson_data_contract<AggClass> {staticinlineconstexprcharconst a[] ="a";staticinlineconstexprcharconst b[] ="b";using type = json_member_list<      json_number<a,int>,      json_number<b>    >;  };}

The class descriptions are recursive with their submembers. Using the previousAggClass one can include it as a member of another class

// See above for AggClassstructMyClass {  AggClass other;  std::string_view some_name;};namespacedaw::json {template<>structjson_data_contract<MyClass> {using type = json_member_list<      json_class<"other", AggClass>,      json_string<"id", std::string_view>    >;  };}

The above maps a classMyClass that has another class that is described AggClass. Also, you can see that the member names of the C++ class do not have to match that of the mapped JSON names and that strings can usestd::string_view as the result type. This is an important performance enhancement if you can guarantee the buffer containing the JSON file will exist as long as the class does.

Iterating over JSON arrays. The input iteratordaw::json::json_array_iterator<JsonElement> allows one to iterator over the array of JSON elements. It is technically an input iterator but can be stored and reused like a forward iterator. It does not return a reference but a value.

#include<daw/json/daw_json_link.h>#include<daw/json/daw_json_iterator.h>#include<iostream>structAggClass {int a{};double b{};};namespacedaw::json {template<>structjson_data_contract<AggClass> {using type = json_member_list<    json_number<"a",int>,    json_number<"b">  >;};}// namespace daw::jsonintmain() {  std::string json_array_data =R"([    {"a":5,"b":2.2},    {"a":5,"b":3.14},    {"a":5,"b":0.122e44},    {"a":5334,"b":34342.2}     ])";usingiterator_t = daw::json::json_array_iterator<AggClass>;auto pos =std::find_if(iterator_t(json_array_data),iterator_t(),        [](AggClassconst &element) {return element.b >1000.0;        }      );if(pos ==iterator_t()) {    std::cout <<"Not found\n";  }else {    std::cout <<"Found\n";  }}

Parsing can begin at a specific member or element. An optional member path tofrom_json,from_json_array,json_value,json_array_iterator, and alike can be specified.The format is a dot separated list of member names and optionally an array index such asmember0.member1 which is like parsing from:

{"member0": {"member1": {}  }}

ormember0[5].member1 which would start parsing at "member1" in a document like:

{"member0": ["a","b","c","d","e",    {"member1":""    }  ]}

or

{"member0": {"a":"","b":"","c":"","d":"","e":"","f": {"member1":""    }  }}

Comments are supported when the parser policy for them is used. Currently, there are two forms of comment policies.

• C++ style// line comments and C style/* */ comments.

{ // This is a comment    "a" /*this is also a comment*/: "a's value"}

• Hash style# line comments

{ # This is a comment    "a" #this is also a comment      : "a's value"}

The comment policy can be set viaPolicyCommentTypes. Seeparser_policies for more info.

To enable serialization one must create an additional static function in your specialization ofjson_data_contract calledto_json_data( Thing const & ); that returns a tuple of members. It will provide a mapping from your type to the arguments provided in the class description. To serialize to a JSON string, one callsto_json( my_thing ); wheremy_thing is a registered type or one of the fundamental types like Containers, Maps, Strings, bool, and numbers. The result of theto_json_data( Thing const & ) static method is atuple who's elements match order in the accompanyingjson_data_contract type aliastype. Because of the way the method is used, tuple's with rvalue elements will result in a use after destruction bug. The compiler will error if this happens. Including<daw/daw_tuple_forward.h> and the methoddaw::forward_nonrvalue_as_tuple instead will store the rvalues instead of passing them by reference. Often it is the result of calculated tuple elements. Using the example above lets add ato_json_data method

#include<daw/json/daw_json_link.h>#include<tuple>structAggClass {int a;double b;};namespacedaw::json {template<>structjson_data_contract<AggClass> {using type = json_member_list<      json_number<"a",int>,      json_number<"b">    >;staticconstexprautoto_json_data( AggClassconst & value ) {returnstd::forward_as_tuple( value.a, value.b );    }  };}//...AggData value =//...;std::string test_001_t_json_data = to_json( value );// orstd::vector<AggData> values =//...;std::string json_array_data = to_json_array( values );

Alternatively, one can output to anyWritableOutput type, by default this includes FILE*, iostreams, containers of Characters, and Character pointers. In your type'sjson_data_constract. Or if opted-into, one can get an ostream operator<< for their type that inserts the json into the output stream byadding a type alias namedopt_into_iostreams the type it aliases doesn't matter, and includedaw/json/daw_json_iostream.h . For example

#include<daw/json/daw_json_link.h>#include<daw/json/daw_json_iostream.h>#include<tuple>structAggClass {int a{};double b{};};namespacedaw::json {template<>structjson_data_contract<AggClass> {using opt_into_iostreams =void;using type = json_member_list<      json_number<"a",int>,      json_number<"b">    >;staticinlineautoto_json_data( AggClassconst & value ) {returnstd::forward_as_tuple( value.a, value.b );    }  };}//...AggData value =//...;std::cout << value <<'\n';// orstd::vector<AggData> values =//...;std::cout << values <<'\n';

A working example can be found atdaw_json_iostream_test.cpp or oncompiler explorer

• error: pointer to subobject of string literal is not allowed in a template argument
  • Your compiler does not support Class Non-Type Template Parameters, or is not in C++20 mode. If you do not have compiler support, you can the C++17 naming style above e.g.

constexprcharconst member_name[] ="member_name";//...json_link<member_name, Type>

There are a few defines that affect how JSON Link operates

• DAW_JSON_DONT_USE_EXCEPTIONS - Controls if exceptions are allowed. If they are not, astd::terminate() on errors will occur. This is automatic if exceptions are disabled(e.g-fno-exceptions)
• DAW_ALLOW_SSE42 - Allow experimental SSE42 mode, generally the constexpr mode is faster
• DAW_JSON_NO_CONST_EXPR - This can be used to allow classes without move/copy special members to be constructed from JSON data prior to C++ 20. This mode does not work in a constant expression prior to C++20 when this flag is no longer needed.

• C++ 17 compiler
• GCC(9/10/11/12) have been tested
• Clang(8/9/10/11/12/13/14) have been tested.
• MSVC 19.29 has been tested.

Older compilers may still work but in testing some resulted in ICE's or compile errors due to buggy C++17 support. Often not using constexpr can help too.

• git
• cmake
• c++17 compiler

• Darrell Wright
• json_link@dawdevel.ca

• When parsing classes,the default is unspecified when duplicate names are encounters as to which is used.
  One can guarantee that both can be parsed or order by usingjson_key_value parse type.
  When used withstd::multimap<std::string, T> orstd::vector<std::pair<std::string, T>> all members are preserved with the former in order. Alternatively, thejson_value type will allow iteration over the class members and lazy parsing of the correct one.SeeCookbook Key Values which demonstrates these methods.
• Trailing commas, the parser makes no effort to detect trailing commas.