Repository files navigation

ninny-json is an experiment to look at what JSON type classes would look likeif they made a distinction between absent JSON fields, and fields withnullvalues.
This project does include its own AST, but the point here is really not tointroduce a new AST or look at the ergonomics of manipulating the AST directly.Thus, the AST included is kept simple.
Why not use json4s, the project created to provide one unifying AST? Read on.

In principle, we want our libraries to be as expressive as possible.
In practice, the limitations of libraries today make it hard or impossible toimplement things likeJSON merge patch orJSON-RPC.Whether a field will be included in the final JSON is also left up to theconfiguration of the JSON serializer (whether to include nulls or not) ratherthan the AST. When the AST doesn't match the JSON output, testability issuescan open up.

Let's look at three popular libraries and see how they deal with convertingOption[A] to and from JSON.

json4s uses the following type classes

traitReader[T] {defread(value:JValue):T}traitWriter[-T] {defwrite(obj:T):JValue}

These are fairly standard and pretty similar to Play JSON, with the differencethat they throw exceptions.

Interestingly json4s includes aJNothing in its AST. Technically there is nosuch thing as "nothing" in JSON, but I can see how it would allow for maximumflexibility with other JSON libraries given that's the goal of json4s.

JNothingwould let us distinguish betweenNone and a missing field.However, the defaultWriter[Option[A]] doesn't leverageJNothing,rather it just writesNone asJNull.The default reader forOption on the other hand just aggregates a failure toparse for any reason intoNone.

• It is technically possible to distinguish null from absent both when readingand writing JSON.

• Default readers/writers don't distinguish null from absent.
• JNothing makes for a strange AST. We can imagine bugs where

  myObj.obj.map(_._1).contains("myField")// true// and yetmyObj\"myField"// JNothing

  Some might suggest"well you should have donemyObj \ "myField" != JNothing instead",but ideally that's a mistake that wouldn't compile.

Play JSON uses the type classes

traitWrites[A] {defwrites(o:A):JsValue}traitReads[A] {defreads(json:JsValue):JsResult[A]}

with a more standard AST.

It does provide aWrites[Option[A]], which writesNone asnull.However, there is noReads[Option[A]] since the type class has no way to knowif the field was missing.

The nice thing about Play JSON is the macro based type class derivation, so youcan just writeimplicit val format = Json.format[MyModel].Now you might think"well, that's not very useful if there is noReads[Option]"andMyModel can't have any optional fields.However, that's not the case, and the macro generated codewill read anOption using some internal logic.This works for the common use case, but if we want to distinguish between anabsent field and some null, then we can't use the automatic format because weneed access to the fields on the outer object.

Reads(root=>JsSuccess(MyClass((root\"myField")match {caseJsDefined(JsNull)=>NullcaseJsDefined(value)=>Defined(value)caseJsUndefined()=>Absent})))

• Automatic format derivation (although circe will call it semi-automatic)

• Inconsistent handling ofOption betweenReads andWrites.
• If we want to take direct control, we lose the composability of type classes.

circe uses the type classes

traitEncoder[A] {defapply(a:A):Json}traitDecoder[A] {defapply(c:HCursor):Decoder.Result[A]deftryDecode(c:ACursor):Decoder.Result[A]= cmatch {casehc:HCursor=> apply(hc)case _=>Left(DecodingFailure("Attempt to decode value on failed cursor", c.history)      )  }}

TheEncoder here is the same as we've seen in the others (and it also encodesNone as null), but theDecoder is interesting. Since circe uses cursors tomove around the JSON, there is anACursor which has the ability to tell usthat the cursor was unable to focus on the field we're trying to decode (thefield wasn't there). circe can and does use this to decode missing fields intoNone, and we can use it to distinguish null from absent fields.

newDecoder[FieldPresence[A]] {deftryDecode(c:ACursor)= cmatch {casec:HCursor=>if (c.value.isNull)Right(Null)else        d(c)match {caseRight(a)=>Right(Defined(a))caseLeft(df)=>Left(df)        }casec:FailedCursor=>if (!c.incorrectFocus)Right(Absent)elseLeft(DecodingFailure("[A]Option[A]", c.history))  }}

Because this is aDecoder for the value rather than the object containing thevalue, we can still use circe's awesome fully automatic type class generationwhich doesn't even require us to invoke a macro method like we do in Play.

Sadly there is nothing we can do with theEncoder to indicate that we don'twant our field included in the output.

• Decoder can distinguish between null and absent fields.

• Encoder can't output an indication that the field should be absent.
• Cursors might be intimidating to the uninitiated.

Now that we have the lay of the land, what are we proposing to shore up thecons without losing the pros?

Two simple type classes (the signatures are what matter, not the names)

traitToJson[A] {// return None if the field should not be included in the JSONdefto(a:A):Option[JsonValue]}traitFromJson[A] {// None if the field was not present in the JSONdeffrom(maybeJson:Option[JsonValue]):Try[A]}

note:Try andOption aren't strictly required, anything that conceptuallyconveys the possibility of failure and absence will work.

ToJson[Option[A]] is implemented predictably

newToJson[Option[A]] {defto(a:Option[A])= a.flatMap(ToJson[A].to(_))}

FromJson[Option[A]] is pretty straightforward as well

newFromJson[Option[A]] {deffrom(maybeJson:Option[JsonValue])= maybeJsonmatch {caseSome(JsonNull)=>Success(None)caseSome(json)=>FromJson[A].from(json).map(Some(_))caseNone=>Success(None)  }}

If we want to distinguish between a null and absent field

newFromJson[FieldPresence[A]] {deffrom(maybeJson:Option[JsonValue])=Success(maybeJsonmatch {caseSome(JsonNull)=>NullcaseSome(json)=>Defined(json)caseNone=>Absent  })}

How are we doing with our pros and cons?

• Able to distinguish null from absent fields when reading and writing JSONfrom inside the type class.
• AST is predictable and closely models JSON.
• We can automatically (or semi-automatically) derive type classes using shapeless.
• Option is handled in the same way when reading and writing.

Always dealing withOption could get annoying,so some simple additions can alleviate that

Addition of a method that takes the AST directly saves us from having toconstantly invokeSome().

deffrom(json:JsonValue):Try[A]

Some types (likeString) will always result in a JSON output. Instances forthose types can be implemented withToSomeJson to remove theOption fromcreated AST.

traitToSomeJson[A]extendsToJson[A] {deftoSome(a:A):JsonValueoverridedefto(a:A)=Some(toSome(a))}

An example of updating a user profile which clears one field,sets the value of another, and leaves a third unchanged without overwriting itwith the existing value.