- Notifications
You must be signed in to change notification settings - Fork155
Clojure JSON and JSON SMILE (binary json format) encoding/decoding
License
dakrone/cheshire
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
'That depends a good deal on where you want to get to,' said the Cat.
'I don't much care where--' said Alice.
'Then it doesn't matter which way you go,' said the Cat.
'--so long as I get SOMEWHERE,' Alice added as an explanation.
'Oh, you're sure to do that,' said the Cat, 'if you only walk longenough.'
Cheshire is fast JSON encoding, based off of clj-json andclojure-json, with additional features like Date/UUID/Set/Symbolencoding and SMILE support.
- Cheshire is a built-in library in theBabashka project
clojure-json had really nice features (custom encoders), but was slow;clj-json had no features, but was fast. Cheshire encodes JSON fast,with added support for more types and the ability to use customencoders.
[cheshire"5.13.0"];; Cheshire v5.13.0 uses Jackson 2.17.0;; In your ns statement:(nsmy.ns (:require [cheshire.core:refer:all]))
;; generate some json(generate-string {:foo"bar":baz5});; write some json to a stream(generate-stream {:foo"bar":baz5} (clojure.java.io/writer"/tmp/foo"));; generate some SMILE(generate-smile {:foo"bar":baz5});; generate some JSON with Dates;; the Date will be encoded as a string using;; the default date format: yyyy-MM-dd'T'HH:mm:ss'Z'(generate-string {:foo"bar":baz (java.util.Date.0)});; generate some JSON with Dates with custom Date encoding(generate-string {:baz (java.util.Date.0)} {:date-format"yyyy-MM-dd"});; generate some JSON with pretty formatting(generate-string {:foo"bar":baz {:eggplant [123]}} {:prettytrue});; {;; "foo" : "bar",;; "baz" : {;; "eggplant" : [ 1, 2, 3 ];; };; };; generate JSON escaping UTF-8(generate-string {:foo"It costs £100"} {:escape-non-asciitrue});; => "{\"foo\":\"It costs \\u00A3100\"}";; generate JSON and munge keys with a custom function(generate-string {:foo"bar"} {:key-fn (fn [k] (.toUpperCase (name k)))});; => "{\"FOO\":\"bar\"}";; generate JSON without escaping the characters (by writing it to a file)(spit"foo.json" (json/generate-string {:foo"bar"} {:prettytrue}))
In the event encoding fails, Cheshire will throw aJsonGenerationException.
If Jackson's default pretty printing library is not what you desire, you canmanually create your own pretty printing class and pass to thegenerate-stringorencode methods:
(let [my-pretty-printer (create-pretty-printer (assocdefault-pretty-print-options:indent-arrays?true))] (generate-string {:foo [123]} {:pretty my-pretty-printer}))
See thedefault-pretty-print-options for a list of options that can bechanged.
;; parse some json(parse-string"{\"foo\":\"bar\"}");; => {"foo" "bar"};; parse some json and get keywords back(parse-string"{\"foo\":\"bar\"}"true);; => {:foo "bar"};; parse some json and munge keywords with a custom function(parse-string"{\"foo\":\"bar\"}" (fn [k] (keyword (.toUpperCase k))));; => {:FOO "bar"};; top-level strings are valid JSON too(parse-string"\"foo\"");; => "foo";; parse some SMILE (keywords option also supported)(parse-smile <your-byte-array>);; parse a stream (keywords option also supported)(parse-stream (clojure.java.io/reader"/tmp/foo"));; parse a stream lazily (keywords option also supported)(parsed-seq (clojure.java.io/reader"/tmp/foo"));; parse a SMILE stream lazily (keywords option also supported)(parsed-smile-seq (clojure.java.io/reader"/tmp/foo"))
In 2.0.4 and up, Cheshire allows passing in afunction to specify what kind of types to return, like so:
;; In this example a function that checks for a certain key(decode"{\"myarray\":[2,3,3,2],\"myset\":[1,2,2,1]}"true (fn [field-name] (if (= field-name"myset") #{} [])));; => {:myarray [2 3 3 2], :myset #{1 2}}
The type must be "transient-able", so use either #{} or []
Custom encoding is supported from 2.0.0 and up, if you encounter abug, please open a github issue. From 5.0.0 onwards, custom encodinghas been moved to be part of the core namespace (not requiring anamespace change)
;; Custom encoders allow you to swap out the api for the fast;; encoder with one that is slightly slower, but allows custom;; things to be encoded:(nsmyns (:require [cheshire.core:refer:all] [cheshire.generate:refer [add-encoder encode-str remove-encoder]]));; First, add a custom encoder for a class:(add-encoder java.awt.Color (fn [c jsonGenerator] (.writeString jsonGenerator (str c))));; There are also helpers for common encoding actions:(add-encoder java.net.URL encode-str);; List of common encoders that can be used: (see generate.clj);; encode-nil;; encode-number;; encode-seq;; encode-date;; encode-bool;; encode-named;; encode-map;; encode-symbol;; encode-ratio;; Then you can use encode from the custom namespace as normal(encode (java.awt.Color.123));; => "java.awt.Color[r=1,g=2,b=3]";; Custom encoders can also be removed:(remove-encoder java.awt.Color);; Decoding remains the same, you are responsible for doing custom decoding.
Custom and Core encoding have been combined in Cheshire 5.0.0, sothere is no longer any need to require a different namespace dependingon what you would like to use.
There are also a few aliases for commonly used functions:
encode -> generate-stringencode-stream -> generate-streamencode-smile -> generate-smiledecode -> parse-stringdecode-stream -> parse-streamdecode-smile -> parse-smileCheshire supports encoding standard clojure datastructures, with a fewadditions.
Cheshire encoding supports:
- strings
- lists
- vectors
- sets
- maps
- symbols
- booleans
- keywords (qualified and unqualified)
- numbers (Integer, Long, BigInteger, BigInt, Double, Float, Ratio,Short, Byte, primitives)
- clojure.lang.PersistentQueue
- Date
- UUID
- java.sql.Timestamp
- any java.util.Set
- any java.util.Map
- any java.util.List
- Stream encoding/decoding
- Lazy decoding
- Pretty-printing JSON generation
- Unicode escaping
- Custom keyword coercion
- Arbitrary precision for decoded values:
Cheshire will automatically use a BigInteger if needed fornon-floating-point numbers, however, for floating-point numbers,Doubles will be used unless the*use-bigdecimals?* symbol is boundto true:
(nsfoo.bar (require [cheshire.core:as json] [cheshire.parse:as parse]))(json/decode"111111111111111111111111111111111.111111111111111111111111111111111111");; => 1.1111111111111112E32 (a Double)(binding [parse/*use-bigdecimals?*true] (json/decode"111111111111111111111111111111111.111111111111111111111111111111111111"));; => 111111111111111111111111111111111.111111111111111111111111111111111111M (a BigDecimal)
- Replacing default encoders for builtin types
- SMILE encoding/decoding
Change log is available on GitHub.
Cheshire is about twice as fast as data.json.
Check out the benchmarks incheshire.test.benchmark; or runlein benchmark. If you have scenarios where Cheshire is not performing aswell as expected (compared to a different library), please let meknow.
In thecheshire.experimental namespace:
$ echo "Hi. \"THIS\" is a string.\\yep." > /tmp/foo$ lein repluser> (use 'cheshire.experimental)niluser> (use 'clojure.java.io)niluser> (println (slurp (encode-large-field-in-map {:id "10" :things [1 2 3] :body "I'll be removed"} :body (input-stream (file "/tmp/foo"))))){"things":[1,2,3],"id":"10","body":"Hi. \"THIS\" is a string.\\yep.\n"}nilencode-large-field-in-map is used for streamy JSON encoding whereyou want to JSON encode a map, but don't want the map in memory all atonce (it returns a stream). Check out the docstring for full usage.
It's experimental, like the name says. Based onTigris.
Seethisandthisfor a list of features that can be customized if desired. A customfactory can be used like so:
(nsmyns (:require [cheshire.core:as core] [cheshire.factory:as factory]))(binding [factory/*json-factory* (factory/make-json-factory {:allow-non-numeric-numberstrue})] (json/decode"{\"foo\":NaN}"true))))))
See thedefault-factory-options map infactory.cljfor a full list of configurable options. Smile factories can also becreated, and factories work exactly the same with custom encoding.
move away from using Java entirely, use Protocols for thecustom encoder(see custom.clj)allow custom encoders(see custom.clj)figure out a way to encode namespace-qualified keywordslook into overriding the default encoding handlers with custom handlersbetter handling when java numbers overflow ECMAScript's numbers(-2^31 to (2^31 - 1))handle encoding java.sql.Timestamp the same asjava.util.Dateadd benchmarking- get criterium benchmarking ignored for 1.2.1 profile
look into faster exception handling by pre-allocating an exceptionobject instead of creating one on-the-fly (maybe ask Steve?)- make it as fast as possible (ongoing)
Release under the MIT license. See LICENSE for the full license.
Thanks go to Mark McGranaghan for clj-json and Jim Duey for the namesuggestion. :)
About
Clojure JSON and JSON SMILE (binary json format) encoding/decoding