Movatterモバイル変換

One of the interesting additions to Swift 4 is theCodable protocol and the machinery around it. This is a subject near and dear to my heart, and I want to discuss what it is and how it works today.

Serialization
Serializing values to data that can be stored on disk or transmitted over a network is a common need. It's especially common in this age of always-connected mobile apps.

So far, the options for serialization in Apple's ecosystem were limited:

1. NSCoding provides intelligent serialization of complex object graphs and works with your own types, but works with a poorly documented serialization format not suitable for cross-platform work, and requires writing code to manually encode and decode your types.
2. NSPropertyListSerialization andNSJSONSerialization can convert between standard Cocoa types likeNSDictionary/NSString and property lists or JSON. JSON in particular is used all over the place for server communication. Since these APIs provide low-level values, you have to write a bunch of code to extract meaning from those values. That code is often ad-hoc and handles bad data poorly.
3. NSXMLParser andNSXMLDocument are the choice of masochists or people stuck working with systems that use XML. Converting between the basic parsed data and more meaningful model objects is once again up to the programmer.
4. Finally, there's always the option to build your own from scratch. This is fun, but a lot of work, and error-prone.

These approaches tend to result in a lot of boilerplate code, where you declare a property calledfoo of typeString which is encoded by storing theString stored infoo under the key"foo" and is decoded by retrieving the value for the key"foo", attempting to cast it to aString, storing it intofoo on success, or throwing an error on failure. Then you declare a property calledbar of typeString which....

Naturally, programmers dislike these repetitive tasks. Repitition is what computers are for. We want to be able to just write this:

structWhatever{varfoo:Stringvarbar:String}

And have it be serializable. It ought to be possible: all the necessary information is already present.

Reflection is a common way to accomplish this. A lot of Objective-C programmers have written code to automatically read and write Objective-C objects to and from JSON objects. The Objective-C runtime provides all of the information you need to do this automatically. For Swift, we can use the Objective-C runtime, or make do with Swift's Mirror and use wacky workarounds to compensate for its inability to mutate properties.

Outside of Apple's ecosystem, this is a common approach in many languages. This has led to varioushilarioussecuritybugs over the years.

Reflection is not a particularly good solution to this problem. It's easy to get it wrong and create security bugs. It's less able to use static typing, so more errors happen at runtime rather than compile time. And it tends to be pretty slow, since the code has to be completely general and does lots of string lookups with type metadata.

Swift has taken the approach of compile-time code generation rather than runtime reflection. This means that some of the knowledge has to be built in to the compiler, but the result is fast and takes advantage of static typing, while still remaining easy to use.

Overview
There are a few fundamental protocols that Swift's new encoding system is built around.

TheEncodable protocol is used for types which can be encoded. If you conform to this protocol and all stored properties in your type are themselvesEncodable, then the compiler will generate an implementation for you. If you don't meet the requirements, or you need special handling, you can implement it yourself.

TheDecodable protocol is the companion to theEncodable protocol and denotes types which can be decoded. LikeEncodable, the compiler will generate an implementation for you if your stored properties are allDecodable.

BecauseEncodable andDecodable usually go together, there's another protocol calledCodable which is just the two protocols glued together:

typealiasCodable=Decodable&Encodable

These two protocols are really simple. Each one contains just one requirement:

protocolEncodable{funcencode(toencoder:Encoder)throws}protocolDecodable{init(fromdecoder:Decoder)throws}

TheEncoder andDecoder protocols specify how objects can actually encode and decode themselves. You don't have to worry about these for basic use, since the default implementation ofCodable handles all the details for you, but you need to use them if you write your ownCodable implementation. These are complex and we'll look at them later.

Finally, there's aCodingKey protocol which is used to denote keys used for encoding and decoding. This adds an extra layer of static type checking to the process compared to using plain strings everywhere. It provides aString, and optionally anInt for positional keys:

protocolCodingKey{varstringValue:String{get}init?(stringValue:String)varintValue:Int?{get}publicinit?(intValue:Int)}

Encoders and Decoders
The basic concept ofEncoder andDecoder is similar toNSCoder. Objects receive a coder and then call its methods to encode or decode themselves.

The API ofNSCoder is straightforward.NSCoder has a bunch of methods likeencodeObject:forKey: andencodeInteger:forKey: which objects call to perform their coding. Objects can also use unkeyed methods likeencodeObject: andencodeInteger: to do things positionally instead of by key.

Swift's API is more indirect.Encoder doesn't have any methods of its own for encoding values. Instead, it providescontainers, and those containers then have methods for encoding values. There's one container for keyed encoding, one for unkeyed encoding, and one for encoding a single value.

This helps make things more explicit and fits better with portable serialization formats.NSCoder only has to work with Apple's encoding format so it just needs to put the same thing out that it got in.Encoder has to work with things like JSON. If an object encodes values with keys, that should produce a JSON dictionary. If it uses unkeyed encoding then that should produce a JSON array. What if the object is empty and encodes no values? With theNSCoder approach, it would have no idea what to output. WithEncoder, the object will still request a keyed or unkeyed container and the encoder can figure it out from that.

Decoder works the same way. You don't decode values from it directly, but rather ask for a container, and then decode values from the container. LikeEncoder,Decoder provides keyed, unkeyed, and single value containers.

Because of this container design, theEncoder andDecoder protocols themselves are small. They contain a bit of bookkeeping info, and methods for obtaining containers:

protocolEncoder{varcodingPath:[CodingKey?]{get}publicvaruserInfo:[CodingUserInfoKey:Any]{get}funccontainer<Key>(keyedBytype:Key.Type)->KeyedEncodingContainer<Key>whereKey:CodingKeyfuncunkeyedContainer()->UnkeyedEncodingContainerfuncsingleValueContainer()->SingleValueEncodingContainer}protocolDecoder{varcodingPath:[CodingKey?]{get}varuserInfo:[CodingUserInfoKey:Any]{get}funccontainer<Key>(keyedBytype:Key.Type)throws->KeyedDecodingContainer<Key>whereKey:CodingKeyfuncunkeyedContainer()throws->UnkeyedDecodingContainerfuncsingleValueContainer()throws->SingleValueDecodingContainer}

The complexity is in the container types. You can get pretty far by recursively walking through properties ofCodable types, but at some point you need to get down to some raw encodable types which can be directly encoded and decoded. ForCodable, those types include the various integer types,Float,Double,Bool, andString. That makes for a whole bunch of really similar encode/decode methods. Unkeyed containers also directly support encoding sequences of the raw encodable types.

Beyond those basic methods, there are a bunch of methods that support exotic use cases. KeyedDecodingContainer has methods calleddecodeIfPresent which return an optional and returnnil for missing keys instead of throwing. The encoding containers have methods for weak encoding, which encodes an object only if something else encodes it too (useful for parent references in a complex graph). There are methods for getting nested containers, which allows you to encode hierarchies. Finally, there are methods for getting a "super" encoder or decoder, which is intended to allow subclasses and superclasses to coexist peacefully when encoding and decoding. The subclass can encode itself directly, and then ask the superclass to encode itself with a "super" encoder, which ensures keys don't conflict.

ImplementingCodable
ImplementingCodable is easy: declare conformance and let the compiler generate it for you.

It's useful to know just what it's doing, though. Let's take a look at what it ends up generating and how you would do it yourself. We'll start with an exampleCodable type:

structPerson:Codable{varname:Stringvarage:Intvarquest:String}

The compiler generates aCodingKeys type nested insidePerson. If we did it ourselves, that nested type would look like this:

privateenumCodingKeys:CodingKey{casenamecaseagecasequest}

The case names matchPerson's property names. Compiler magic gives each CodingKeys case a string value which matches its case name, which means that the property names are also the keys used for encoding them.

If we need different names, we can easily accomplish this by providing our ownCodingKeys with custom raw values. For example, we might write this:

privateenumCodingKeys:String,CodingKey{casename="person_name"caseagecasequest}

This will cause thename property to be encoded and decoded underperson_name. And this is all we have to do. The compiler happily accepts our customCodingKeys type while still providing a default implementation for the rest ofCodable, and that default implementation uses our custom type. You can mix and match customizations with the compiler-provided code.

The compiler also generates an implementation forencode(to:) andinit(from:). The implementation ofencode(to:) gets a keyed container and then encodes each property in turn:

funcencode(toencoder:Encoder)throws{varcontainer=encoder.container(keyedBy:CodingKeys.self)trycontainer.encode(name,forKey:.name)trycontainer.encode(age,forKey:.age)trycontainer.encode(quest,forKey:.quest)}

The compiler generates an implementation ofinit(from:) which mirrors this:

init(fromdecoder:Decoder)throws{letcontainer=trydecoder.container(keyedBy:CodingKeys.self)name=trycontainer.decode(String.self,forKey:.name)age=trycontainer.decode(Int.self,forKey:.age)quest=trycontainer.decode(String.self,forKey:.quest)}

That's all there is to it. Just like withCodingKeys, if you need custom behavior here you can implement your own version of one of these methods while letting the compiler generate the rest. Unfortunately, there's no way to specify custom behavior for an individual property, so you have to write out the whole thing even if you want the default behavior for the rest. This is not particularly terrible, though.

If you were to do it all by hand, the full implementation ofCodable forPerson would look like this:

extensionPerson{privateenumCodingKeys:CodingKey{casenamecaseagecasequest}funcencode(toencoder:Encoder)throws{varcontainer=encoder.container(keyedBy:CodingKeys.self)trycontainer.encode(name,forKey:.name)trycontainer.encode(age,forKey:.age)trycontainer.encode(quest,forKey:.quest)}init(fromdecoder:Decoder)throws{letcontainer=trydecoder.container(keyedBy:CodingKeys.self)name=trycontainer.decode(String.self,forKey:.name)age=trycontainer.decode(Int.self,forKey:.age)quest=trycontainer.decode(String.self,forKey:.quest)}}

ImplementingEncoder andDecoder
You may never need to implement your ownEncoder orDecoder. Swift provides implementations for JSON and property lists, which take care of the common use cases.

You can implement your own in order to support a custom format. The size of the container protocols means this will take some effort. Fortunately, it's mostly a matter of size, not complexity.

To implement a customEncoder, you'll need something that implements theEncoder protocol plus implementations of the container protocols. Implementing the three container protocols involves a lot of repetitive code to implement encoding or decoding methods for all of the various directly encodable types.

How they work is up to you. TheEncoder will probably need to store the data being encoded, and the containers will inform theEncoder of the various things they're encoding.

Implementing a customDecoder is similar. You'll need to implement that protocol plus the container protocols. The decoder will hold the serialized data and the containers will communicate with it to provide the requested values.

I've been experimenting with a custom binary encoder and decoder as a way to learn the protocols, and I hope to present that in a future article as an example of how to do it.

Conclusion
Swift 4'sCodable API looks great and ought to simplify a lot of common code. For typical JSON tasks, it's sufficient to declare conformance toCodable in your model types and let the compiler do the rest. When needed, you can implement parts of the protocol yourself in order to handle things differently, and you can implement it all if needed.

The companionEncoder andDecoder protocols are more complex, but justifiably so. Supporting a custom format by implementing your ownEncoder andDecoder takes some work, but is mostly a matter of filling in a lot of similar blanks.

That's it for today! Come back again for more exciting serialization-related material, and perhaps even things not related to serialization. Until then, Friday Q&A is driven by reader ideas, so if you have a topic you'd like to see covered here, pleasesend it in!

Add your thoughts, post a comment:

Spam and off-topic posts will be deleted without notice. Culprits may be publicly humiliated at my sole discretion.