- Notifications
You must be signed in to change notification settings - Fork623
.NET's fastest JSON, JSV and CSV Text Serializers
License
ServiceStack/ServiceStack.Text
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Follow@ServiceStack,view the docs, useStackOverflow orCustomer Forums for support.
FREE high-perf Text Serializers and Core Utils poweringservicestack.net
ServiceStack.Text is anindependent, dependency-free serialization library containing ServiceStack's core high-performance utils and text processing functionality, including:
- JSON,JSV andCSV Text Serializers
- AutoMapping Utils
- HTTP Utils
- Dump Utils
- Fast Reflection Utils
- Several String Extensions, Collection extensions, Reflection Utils and lots more.
Try outServiceStack.Text Live
A great way to try out ServiceStack.Text is ongist.cafe which lets you immediatelyrun and explore all ServiceStack.Text features from the comfort of your browser with zero software install:
Like most of the interfaces in ServiceStack, the API is simple. Methods that you would commonly use include:
stringToJson(T)TFromJson()stringToJsv(T)TFromJsv()stringToCsv(T)TFromCsv()stringToXml(T)TFromXml()
stringJsonSerializer.SerializeToString<T>(Tvalue)void JsonSerializer.SerializeToWriter<T>(Tvalue,TextWriterwriter)T JsonSerializer.DeserializeFromString<T>(stringvalue)TJsonSerializer.DeserializeFromReader<T>(TextReaderreader)
stringTypeSerializer.SerializeToString<T>(Tvalue)void TypeSerializer.SerializeToWriter<T>(Tvalue,TextWriterwriter)T TypeSerializer.DeserializeFromString<T>(stringvalue)TTypeSerializer.DeserializeFromReader<T>(TextReaderreader)
stringCsvSerializer.SerializeToString<T>(Tvalue)void CsvSerializer.SerializeToWriter<T>(Tvalue,TextWriterwriter)T CsvSerializer.DeserializeFromString<T>(stringvalue)TCsvSerializer.DeserializeFromReader<T>(TextReaderreader)
WhereT can be any .NET POCO type. That's all there is - the API was intentionally left simple :)
Dump / Diagnostic Extensions:
TDump()TPrint()TPrintDump()stringFmt(args)
JsonObject.Parse()JsonArrayObjects.Parse()
You can format JSON into a more readable format with theIndentJson()
extension method, e.g:
varprettyJson=dto.ToJson().IndentJson();
URL Extensions:
stringGetStringFromUrl()stringGetJsonFromUrl()stringGetResponseStatus()stringUrlEncode()/ UrlDecode()stringHexEscape()/ HexUnescape()stringUrlFormat()/ AppendPath()/ AppendPaths()/ WithTrailingSlash()stringAddQueryParam()/ SetQueryParam()AddHashParam()/SetHashParam()string WithoutExtension()/ParentDirectory()/ReadAllText()
StreamWriteTo(Stream)/ CopyTo()Stream ReadLines()StreamReadFully()/ReadExactly()
stringSplitOnFirst()/ SplitOnLast()stringIndexOfAny()stringStripHtml()/ ToCamelCase()stringSafeSubstring()stringToUtf8Bytes()/ FromUtf8Bytes()stringLeftPart()/ LastLeftPart()/ RightPart()/ LastRightPart()
more String, Reflection, List, Dictionary, DateTime extensions...
Although usually used to (de)serialize C#/.NET POCO types, it also includes a flexible API allowing you to deserialize anyJSON payload without it's concrete type, see these real-world examples:
- Parsing GitHub's v3 API with typed DTOs
- Parsing GitHub's JSON response
- Parsing Google Maps JSON Response
- Parsing Centroid
Also a thin.NET 4.0 Dynamic JSON wrapper around ServiceStack's JSON library is included in theServiceStack.Razorproject. It provides a dynamic, but more succinct API than the above options.
ServiceStack.Text APIs for deserializing arbitrary JSON requires specifying the Type to deserialize into. An alternative flexible approach to read any arbitrary JavaScript or JSON data structures is to use the high-performance and memory efficient JSON utils in#Script implementation of JavaScript.
JSON.parse("1")//= int 1JSON.parse("1.1")//= double 1.1JSON.parse("'a'")//= string "a"JSON.parse("{a:1}")//= new Dictionary<string, object> { {"a", 1 } }JSON.parse("[{a:1}]")//= new List<object> { new Dictionary<string, object> { { "a", 1 } } }
Since JS Utils is an essential part of#Script it allows for advanced scenarios like implementing a text DSL or scripting language for executing custom logic or business rules you want to be able to change without having to compile or redeploy your App. It uses#Script Context which lets you evaluate the script within a custom scope that defines what functionsand arguments it has access to, e.g.:
publicclassCustomMethods:ScriptMethods{publicstringreverse(stringtext)=>newstring(text.Reverse().ToArray());}varscope=JS.CreateScope(args:newDictionary<string,object>{{"arg","value"}},functions:newCustomMethods());JS.eval("arg",scope)//= "value"JS.eval("reverse(arg)",scope)//= "eulav"JS.eval("3.itemsOf(arg.reverse().padRight(8, '_'))",scope)//= ["eulav___", "eulav___", "eulav___"]//= { a: ["eulav___", "eulav___", "eulav___"] }JS.eval("{a: 3.itemsOf(arg.reverse().padRight(8, '_')) }",scope)
ServiceStack's JS Utils is available in theServiceStack.Common NuGet package.
PM> Install-Package ServiceStack.Text
From v4.0.62+ServiceStack.Text is now free!
Since September 2013, ServiceStack source code is available under GNU Affero General Public License/FOSS License Exception, see license.txt in the source. Alternative commercial licensing is also available, contactteam@servicestack.net for details.
Contributors need to approve theContributor License Agreement before any code will be reviewed, see theContributing wiki for more details.
For reasons outlinedin this blog post I decided to re-useTypeSerializer's text processing-core to create ServiceStack.JsonSerializer - the fastest JSON Serializer for .NET.Based on theNorthwind Benchmarks it's3.6x faster than .NET's BCL JsonDataContractSerializer and3x faster than the previous fastest JSON serializer benchmarked -JSON.NET.
A comprehensive set of other .NET benchmarks are maintained atservicestack.net/benchmarks andin the wiki.
As CSV is an important format in many data access and migration scenarios, it becamethe latest format included in ServiceStack which allows all your existing web services to take advantage of the new format without config or code-changes. As its built using the same tech that makes the JSON and JSV serializers so fast, we expect it to be the fastest POCO CSV Serializer for .NET.
Included in this project isTypeSerializer
- A fast and compact text-based serializer for .NET. It's a light-weight compact Text Serializer which can be used to serialize .NET data types inc custom POCO's and DataContract's. More info on its JSV Format can be found on theintroductory post.
Another useful library to have in your .NET toolbox is theT.Dump() Extension Method. Under the hood it uses aPretty Print Output of the JSV Format to recursively dump the contents of any .NET object. Example usage and output:
varmodel=newTestModel();model.PrintDump();//Example Output{Int:1,String:One,DateTime:2010-04-11, Guid: c050437f6fcd46be9b2d0806a0860b3e, EmptyIntList:[],IntList:[1,2,3],StringList:[one,two,three],StringIntMap:{a:1,b:2,c:3}}
ServiceStack's JsonSerializer is optimized for serializing C# POCO types in and out of JSON as fast, compact and cleanly as possible. In most cases C# objects serializes as you would expect them to without added json extensions or serializer-specific artefacts.
JsonSerializer provides a simple API that allows you to serialize any .NET generic or runtime type into a string, TextWriter/TextReader or Stream.
stringSerializeToString<T>(T)voidSerializeToWriter<T>(T,TextWriter)voidSerializeToStream<T>(T,Stream)stringSerializeToString(object,Type)voidSerializeToWriter(object,Type,TextWriter)voidSerializeToStream(object,Type,Stream)
TDeserializeFromString<T>(string)T DeserializeFromReader<T>(TextReader)objectDeserializeFromString(string,Type)objectDeserializeFromReader(reader,Type)objectDeserializeFromStream(Type,Stream)TDeserializeFromStream<T>(Stream)
stringToJson<T>(thisT)T FromJson<T>(thisstring)
ConvenientToJson/FromJson extension methods are also included reducing the amount of code required, e.g:
new[]{1,2,3}.ToJson()//= [1,2,3]"[1,2,3]".FromJson<int[]>()//= int []{ 1, 2, 3 }
JSON is a lightweight text serialization format with a spec that's so simple that it fits on one page:https://www.json.org.
The only valid values in JSON are:
- string
- number
- object
- array
- true
- false
- null
Where most allowed values are scalar and the only complex types available are objects and arrays. Although limited, the above set of types make a good fit and can express most programming data structures.
All C# boolean and numeric data types are stored as-is without quotes.
For the most compact output null values are omitted from the serialized by default. If you want to include null values set the global configuration:
JsConfig.Init(newConfig{IncludeNullValues=true});
All other scalar values are stored as strings that are surrounded with double quotes.
Because a C# struct is a value type whose public properties are normally just convenience properties around a single scalar value, they are ignored instead theTStruct.ToString() method is used to serialize and either thestatic TStruct.Parse() method ornew TStruct(string) constructor will be used to deserialize the value type if it exists.
Any List, Queue, Stack, Array, Collection, Enumerables including custom enumerable types are stored in exactly the same way as a JavaScript array literal, i.e:
[1,2,3,4,5]
All elements in an array must be of the same type. If a custom type is both an IEnumerable and has properties it will be treated as an array and the extra properties will be ignored.
The JSON object type is the most flexible and is how most complex .NET types are serialized. The JSON object type is a key-value pair JavaScript object literal where the key is always a double-quoted string.
Any IDictionary is serialized into a standard JSON object, i.e:
{"A":1,"B":2,"C":3,"D":4}
Which happens to be the same as C# POCO types (inc. Interfaces) with the values:
new MyClass { A=1, B=2, C=3, D=4 }
{"A":1,"B":2,"C":3,"D":4}
Only public properties on reference types are serialized with the C# Property Name used for object key and the Property Value as the value. At the moment it is not possible to customize the Property Name.
JsonSerializer also supports serialization of anonymous types in much the same way:
new { A=1, B=2, C=3, D=4 }
{"A":1,"B":2,"C":3,"D":4}
The default WCF Date that's returned in ServiceStack.Text can be converted with:
functiontodate(s){returnnewDate(parseFloat(/Date\(([^)]+)\)/.exec(s)[1]));};
Which if you're using theservicestack-client npm package can be resolved with:
import{todate}from"servicestack-client";vardate=todate(wcfDateString);
Or if usingss-utils.js that's built into ServiceStack:
vardate=$.ss.todate(wcfDateString);
If you change ServiceStack.Text default serialization of Date to either use the ISO8601 date format:
JsConfig.DateHandler=DateHandler.ISO8601;
It can be parsed natively with:
newDate(dateString)
Likewise when configured to return:
JsConfig.DateHandler=DateHandler.UnixTimeMs;
It can also be converted natively with:
newDate(unixTimeMs)
The JSON/JSV and CSV serialization can be customized globally by configuring theJsConfig
or type-specificJsConfig<T>
static classes with your preferred defaults. Global static configuration can be configured once onStartup usingJsConfig.Init()
, e.g:
JsConfig.Init(newConfig{DateHandler=DateHandler.ISO8601,AlwaysUseUtc=true,TextCase=TextCase.CamelCase,ExcludeDefaultValues=true,});
The following is a list ofbool
options you can use to configure many popular preferences:
Name | Alias |
---|---|
IncludeNullValues | inv |
IncludeNullValuesInDictionaries | invid |
IncludeDefaultEnums | ide |
IncludePublicFields | ipf |
IncludeTypeInfo | iti |
ExcludeTypeInfo | eti |
ExcludeDefaultValues | edv |
ConvertObjectTypesIntoStringDictionary | cotisd |
TreatEnumAsInteger | teai |
TryToParsePrimitiveTypeValues | ttpptv |
TryToParseNumericType | ttpnt |
ThrowOnDeserializationError | tode |
EscapeUnicode | eu |
EscapeHtmlChars | ehc |
PreferInterfaces | pi |
SkipDateTimeConversion | sdtc |
AlwaysUseUtc | auu |
AssumeUtc | au |
AppendUtcOffset | auo |
EscapeHtmlChars | ehc |
EscapeUnicode | eu |
EmitCamelCaseNames | eccn |
EmitLowercaseUnderscoreNames | elun |
TimestampOffset | to |
DCJSCompatible | dcjsc |
ISO8601 | iso8601 |
ISO8601DateOnly | iso8601do |
ISO8601DateTime | iso8601dt |
RFC1123 | rfc1123 |
UnixTime | ut |
UnixTimeMs | utm |
DurationFormat | df |
StandardFormat | sf |
Default | d |
PascalCase | pc |
CamelCase | cc |
SnakeCase | sc |
Strict | s |
Lenient | l |
If you need to use different serialization settings from the global static defaults you can useJsConfig.With()
to create a scoped configurationusing property initializers:
using(JsConfig.With(newConfig{TextCase==TextCase.CamelCase,PropertyConvention=PropertyConvention.Lenient})){returntext.FromJson<T>();}
You can also create a custom config scope from a string manually usingJsConfig.CreateScope()
where you can use the full config name or their aliases, e.g:
using(JsConfig.CreateScope("IncludeNullValues,EDV,dh:ut")){varjson=dto.ToJson();}
This feature is used to provide a number of differentJSON customizations in ServiceStack Services.
If you can't change the definition of a Type (e.g. because its in the BCL), you can specify a custom serialization /deserialization routine to use instead. E.g. here's how you can add support forSystem.Drawing.Color
and customize howGuid
andTimeSpan
Types are serialized:
JsConfig<System.Drawing.Color>.SerializeFn= c=>c.ToString().Replace("Color ","").Replace("[","").Replace("]","");JsConfig<System.Drawing.Color>.DeSerializeFn=System.Drawing.Color.FromName;JsConfig<Guid>.SerializeFn= guid=>guid.ToString("D");JsConfig<TimeSpan>.SerializeFn= time=>(time.Ticks<0?"-":"")+time.ToString("hh':'mm':'ss'.'fffffff");
By default ServiceStack Serializers will try to deserialize as much as possible without error, if you prefer you can opt-in to stricter parsing with:
Env.StrictMode=true;
Where it will instead fail fast and throw Exceptions on deserialization errors.
Although JsonSerializer is optimized for serializing .NET POCO types, it still provides some options to change the convention-based serialization routine.
This makes it possible to customize the serialization routine and provide an even more compact wire format.
E.g. Instead of using a JSON object to represent a point
{ Width=20, Height=10 }
You could use a struct and reduce it to just:
"20x10"
By overridingToString() and providing a staticSize ParseJson() method:
publicstructSize{publicdoubleWidth{get;set;}publicdoubleHeight{get;set;}publicoverridestringToString(){returnWidth+"x"+Height;}publicstaticSizeParseJson(stringjson){varsize=json.Split('x');returnnewSize{Width=double.Parse(size[0]),Height=double.Parse(size[1])};}}
Which would change it to the more compact JSON output:
newSize{Width=20,Height=10}.ToJson()// = "20x10"
That allows you to deserialize it back in the same way:
varsize="20x10".FromJson<Size>();
In addition to using a Struct you can optionally use a custom C# IEnumerable type to provide a strong-typed wrapper around a JSON array:
publicclassPoint:IEnumerable{double[]points=newdouble[2];publicdoubleX{get{returnpoints[0];}set{points[0]=value;}}publicdoubleY{get{returnpoints[1];}set{points[1]=value;}}publicIEnumeratorGetEnumerator(){foreach(varpointinpoints)yieldreturnpoint;}}
Which serializes the Point into a compact JSON array:
newPoint{X=1,Y=2}.ToJson()// = [1,2]
Because the same wire format shared between Dictionaries, POCOs and anonymous types, in most cases what you serialize with one type can be deserialized with another, i.e. an Anonymous type can be deserialized back into a Dictionary<string,string> which can be deserialized into a strong-typed POCO and vice-versa.
Although the JSON Serializer is best optimized for serializing and deserializing .NET types, it's flexible enough to consume 3rd party JSON apis although this generally requires custom de-serialization to convert it into an idiomatic .NET type.
- UsingJsonObject
- Using Generic .NET Collection classes
- Using Customized DTO's in the shape of the 3rd party JSON response
CentroidTests is another example that uses the JsonObject to parse a complex custom JSON response.
In order to be able to deserialize late-bound objects likeobject
,interface
properties orabstract
classes ServiceStack needs to emit type informationin the JSON payload. By default it uses__type
property name, but can be customized with:
JsConfig.TypeAttr="$type";
You can also configure what Type Information is emitted with:
JsConfig.TypeWriter= type=>type.Name;
Which will just emit the name of the Type (e.gDog
) instead of the full Type Name.
By default ServiceStack will scan all loaded Assemblies to find the Type, but you can tell it to use your own Type Resolver implementation by overridingTypeFinder
, e.g:
JsConfig.TypeFinder= typeInfo=>=>{varregex=newRegex(@"^(?<type>[^:]+):#(?<namespace>.*)$");varmatch=regex.Match(value);vartypeName=string.Format("{0}.{1}",match.Groups["namespace"].Value,match.Groups["type"].Value.Replace(".","+"));returnMyFindType(typeName);};
ServiceStack only allows you to serialize "known safe Types" in late-bound properties which uses a whitelist that's pre-populated with a safe-list ofpopular Data Types, DTOs and Request DTOs with the default configuration below:
// Allow deserializing types with [DataContract] or [RuntimeSerializable] attributesJsConfig.AllowRuntimeTypeWithAttributesNamed=newHashSet<string>{nameof(DataContractAttribute),nameof(RuntimeSerializableAttribute),// new in ServiceStack.Text};// Allow deserializing types implementing any of the interfaces belowJsConfig.AllowRuntimeTypeWithInterfacesNamed=newHashSet<string>{"IConvertible","ISerializable","IRuntimeSerializable",// new in ServiceStack.Text"IMeta","IReturn`1","IReturnVoid",};// Allow object property in ServiceStack.Messaging MQ classesJsConfig.AllowRuntimeTypeInTypesWithNamespaces=newHashSet<string>{"ServiceStack.Messaging",};
The above rules can be extended to allow your own conventions. If you just need to allow a specific Type you can instead just implement:
JsConfig.AllowRuntimeType= type=>type==typeof(MyType);
If you’re in a trusted intranet environment this can also be used to disable the whitelist completely by allowing all Types to be deserialized into object properties with:
JsConfig.AllowRuntimeType= _=>true;
You can use[EnumMember]
to change what Enum value gets serialized, e.g:
[DataContract]publicenumDay{[EnumMember(Value="MON")]Monday,[EnumMember(Value="TUE")]Tuesday,[EnumMember(Value="WED")]Wednesday,[EnumMember(Value="THU")]Thursday,[EnumMember(Value="FRI")]Friday,[EnumMember(Value="SAT")]Saturday,[EnumMember(Value="SUN")]Sunday,}classEnumMemberDto{publicDayDay{get;set;}}vardto=newEnumMemberDto{Day=Day.Sunday};varjson=dto.ToJson();//= {"Day":"SUN"}varfromDto=json.FromJson<EnumMemberDto>();fromDto.Day//= Day.Sunday
Out of the box .NET provides a fairly quick but verbose Xml DataContractSerializer or a slightly more compact but slower JsonDataContractSerializer.Both of these options are fragile and likely to break with any significant schema changes.TypeSerializer addresses these shortcomings by being both smaller and significantly faster than the most popular options.It's also more resilient, e.g. a strongly-typed POCO object can be deserialized back into a loosely-typed string Dictionary and vice-versa.
With that in mind, TypeSerializer's main features are:
- Fastest and most compact text-serializer for .NET
- Human readable and writeable, self-describing text format
- Non-invasive and configuration-free
- Resilient to schema changes
- Serializes / De-serializes any .NET data type (by convention)
- Supports custom, compact serialization of structs by overriding
ToString()
andstatic T Parse(string)
methods - Can serialize inherited, interface or 'late-bound objects' data types
- Respects opt-in DataMember custom serialization for DataContract dto types.
- Supports custom, compact serialization of structs by overriding
These characteristics make it ideal for use anywhere you need to store or transport .NET data-types, e.g. for text blobs in a ORM, data in and out of a key-value store or as the text-protocol in .NET to .NET web services.
As such, it's utilized within ServiceStack's other components:
- OrmLite - to store complex types on table models as text blobs in a database field and
- ServiceStack.Redis - to store rich POCO data types into the very fastredis instances.
You may also be interested in the very usefulT.Dump() extension method for recursively viewing the contents of any C# POCO Type.
Type Serializer is actually the fastest and most compacttext serializer available for .NET.Out of all the serializers benchmarked, it is the only one to remain competitive withprotobuf-net's very fast implementation ofProtocol Buffers - google's high-speed binary protocol.
Below is a series of benchmarks serialize the different tables in theNorthwind database (3202 records) with the most popular serializers available for .NET:
Combined results for serializing / deserialzing a single row of each table in the Northwind database 1,000,000 times
Serializer | Size | Peformance |
---|---|---|
Microsoft DataContractSerializer | 4.68x | 6.72x |
Microsoft JsonDataContractSerializer | 2.24x | 10.18x |
Microsoft BinaryFormatter | 5.62x | 9.06x |
NewtonSoft.Json | 2.30x | 8.15x |
ProtoBuf.net | 1x | 1x |
ServiceStack TypeSerializer | 1.78x | 1.92x |
number of times larger in size and slower in performance than the best - lower is better
Microsoft's JavaScriptSerializer was also benchmarked but excluded as it was up to 280x times slower - basically don't use it, ever.
Type Serializer uses a hybrid CSV-style escaping + JavaScript-like text-based format that is optimized for both size and speed. I'm naming this JSV-format (i.e. JSON + CSV)
In many ways it is similar to JavaScript, e.g. any List, Array, Collection of ints, longs, etc are stored in exactly the same way, i.e:
[1,2,3,4,5]
Any IDictionary is serialized like JavaScript, i.e:
{A:1,B:2,C:3,D:4}
Which also happens to be the same as C# POCO class with the values
new MyClass { A=1, B=2, C=3, D=4 }
{A:1,B:2,C:3,D:4}
JSV iswhite-space significant, which means normal string values can be serialized without quotes, e.g:
new MyClass { Foo="Bar", Greet="Hello World!"}
is serialized as:
{Foo:Bar,Greet:Hello World!}
Any string with any of the following characters:[]{},"
is escaped using CSV-style escaping where the value is wrapped in double quotes, e.g:
new MyClass { Name = "Me, Junior" }
is serialized as:
{Name:"Me, Junior"}
A value with a double quote is escaped with another double quote e.g:
new MyClass { Size = "2\" x 1\"" }
is serialized as:
{Size:"2"" x 1"""}
To better illustrate the resilience ofTypeSerializer
and the JSV Format check out a real world example of it when it's used toPainlessly migrate between old and new types in Redis.
Support for dynamic payloads and late-bound objects is explained in the postVersatility of JSV Late-bound objects.
About
.NET's fastest JSON, JSV and CSV Text Serializers
Topics
Resources
License
Contributing
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.