Posted onNov 29, 2017

A Cameo that is worth an Oscar

#akka #scala #designpattern #designpatterns

Originally posted on:Big ball of mud

Rarely, during my life as a developer, I found pre-packaged solutions that fit my problem so well. Design patterns are an abstraction of both problems and solutions. So, they often need some kind of customization on the specific problem. While I was developing my concrete instance ofActorbase specification, I came across theCameo pattern. It enlighted my way and my vision about how to use Actors profitably. Let's see how and why.

The problem: capturing context

Jamie Allen, in his short but worthwhile bookEffective Akka, begins the chapter dedicated to Actors patterns with the following words:

One of the most difficult tasks in asynchronous programming is trying to capture context so that the state of the world at the time the task was started can be accurately represented at the time the task finishes.

This is exactly the problem we are going to try to resolve.

Actors often model long-lived asynchronous processes, in which a response in the future corresponds to one or more messages sent earlier. Meanwhile, the context of execution of the Actor could be changed. In the case of an Actor, its context is represented by all the mutable variables owned by the Actor itself. A notable example is thesender variable that stores the sender of the current message being processed by an Actor.

Context handling in Actorbase actors

Let's make a concrete example. In Actorbase there are two types of Actors among the others:StoreFinder andStorekeeper. Each Actor of typeStoreFinder represents adistributed map or acollection, but it does not physically store the key-value couples. This information is stored byStorekeeper Actors. So, eachStoreFinder owns a distributed set of its key-value couples, which means that owns a set ofStorekeeper Actors that stores the information for it.

StoreFinder can route to itsStorekeeper many types of messages, which represent CRUD operations on the data stored. The problem here is that if aStoreFinder ownsnStorekeeper, tofind which value corresponds to akey (if any), it has to sendn messages of typeGet("key") to eachStorekeeper. Once all theStorekeeper answer to the query messages, theStoreFinder can answer to its caller with the requestedvalue.

The sequence diagram below depicts exactly the above scenario.

The number of answers ofStorekeeper Actors and the body of their responses represent the execution context ofStoreFinder Actor.

Actor's context handling

So, we need to identify a concrete method to handle the execution context of an Actor. The problem is that between the sending of a message and the time when the relative response is received, an Actor processes many other messages.

Naive solution

Using nothing that my ignorance, the first solution I depicted in Actorbase was the following.

classStoreFinder(valname:String)extendsActor{defreceive:Receive=nonEmptyTable(StoreFinderState(Map()))defnonEmptyTable(state:StoreFinderState):Receive={// Query messages from externl actorscaseQuery(key,u)=>// Route a Get message to each StorekeeperbroadcastRouter.route(Get(key,u),self)context.become(nonEmptyTable(state.addQuery(key,u,sender())))// Some other stuff...// Responses from Storekeepercaseres:Item=>context.become(nonEmptyTable(state.copy(queries=item(res,state.queries))))}// Handling a response from a Storekeeper. Have they all answer? Is there at least// a Storekeeper that answer with a value? How can a StoreFinder store the original// sender?privatedefitem(response:Item,queries:Map[Long,QueryReq]):Map[Long,QueryReq]={valItem(key,opt,id)=responsevalQueryReq(actor,responses)=queries(id)valnewResponses=opt::responsesif(newResponses.length==NumberOfPartitions){// Some code to create the messageactor!QueryAck(key,item,id)queries-id}else{queries+(id->QueryReq(actor,newResponses))}}}// I need a class to maintain the execution contextcaseclassStoreFinderState(queries:Map[Long,QueryReq]){defaddQuery(key:String,id:Long,sender:ActorRef):StoreFinderState={// Such a complex data structure!copy(queries=queries+(id->QueryReq(sender,List[Option[(Array[Byte],Long)]]())))}// Similar code for other CRUD operations}sealedcaseclassQueryReq(sender:ActorRef,responses:List[Option[(Array[Byte],Long)]])

A lot of code to handle only a bunch of messages, isn't it? As you can see, to handle the execution context I defined a dedicated class,StoreFinderState. For eachQuery message identified by aUUID of typeLong, this class stores the following information:

The original sender
The list of responses fromStorekeeper Actors for the message
The values theStorekeeper answered with

As you can imagine, the handling process of this context is not simple, as a singleStoreFinder has to handle all the messages that have not received a final response from all the relativeStorekeeper.

We can do much better, trust me.

Asking the future

A first attempt to reach a more elegant and concise solution might be the use of theAsk pattern withFuture.

This is a great way to design your actors in that they will not block waiting for responses, allowing them to handle more messages concurrently and increase your application’s performance.

Using the Ask pattern, the code that handles theQuery message and its responses will reduce to the following.

caseQuery(key,u)=>valfutureQueryAck:Future[QueryAck]=for{responses<-Future.sequence(routeesmap(ask(_,Get(key,u))).mapTo[Item])}yield{QueryAck(/* Some code to create the QueryAck message from responses */)}futureQueryAckmap(sender!_)

Whoah! This code is fairly concise with respect to the previous one. In addition, usingFuture and a syntax that is fairly declarative, we can achieve quite easily the right grade of asynchronous execution that we need.

However, there are a couple of things about it that are not ideal. First of all, it is using futures to ask other actors for responses, which creates a newPromiseActorRef for every message sent behind the scenes. This is a waste of resources.

Annoying.

Furthermore, there is a glaring race condition in this code—can you see it? We’re referencing the “sender” in our map operation on the result fromfutureQueryAck, which may not be the sameActorRef when the future completes, because theStoreFinder ActorRef may now be handling another message from a different sender at that point!

Even more annoying!

The Extra pattern

The problem here is that we are attempting to take the result of the off-thread operations of retrieving data from multiple sources and return it to whoever sent the original request to theStoreFinder. But, the actor will likely have move on to handling additional messages in its mailbox by the time the above futures complete.

The trick is capturing the execution context of a request in a dedicated inner actor. Let's see how our code will become.

caseQuery(key,u)=>{// Capturing the original sendervaloriginalSender=sender// Handling the execution in a dedicated actorcontext.actorOf(Props(newActor(){// The list of responses from Storekeepersvarresponses:List[Option[(Array[Byte],Long)]]=Nildefreceive={caseItem(key,opt,u)=>responses=opt::responsesif(responses.length==partitions){// Some code that creates the QueryAck messageoriginalSender!QueryAck(key,item,u)context.stop(self)}}}))}

Much better. We have captured the context for a single request toStoreFinder as the context of a dedicated actor. The original sender ofStoreFinder Actor was captured by the constantoriginalSender and shared with the anonymous Actor using aclosure.

It's easy, isn't it? This simple trick is known as theExtra pattern. However, we are searching for aCameo in our movie.

Finally presenting the Cameo pattern

The Extra pattern is very useful when the code inside the anonymous Actor is very small and trivial. Otherwise, it pollutes the main Actor with details that do not belong to its responsibility (one for all, Actor creation).

It is also similar to lambdas, in that using an anonymous instance gives you less information in stack traces on the JVM, is harder to use with a debugging tool, and is easier to close over state.

Luckily, the solution is quite easy. We can move the anonymous implementation of the Actor into its own type definition.

This results in a type only used for simple interactions between actors, similar to a cameo role in the movies.

Doing so, the code finally becomes the following.

classStoreFinder(valname:String)extendsActor{overridedefreceive:Receive={// Omissis...caseQuery(key,u)=>valoriginalSender=sender()valhandler=context.actorOf(Props(newQueryResponseHandler(originalSender,NumberOfPartitions)))broadcastRouter.route(Get(key,u),handler)}// Omissis...}// The actor playing the Cameo roleclassQueryResponseHandler(originalSender:ActorRef,partitions:Int){varresponses:List[Option[(Array[Byte],Long)]]=Niloverridedefreceive:Receive=LoggingReceive{caseItem(key,opt,u)=>responses=opt::responsesif(responses.length==partitions){// Some code to make up a QueryAck messageoriginalSender!QueryAck(key,item,u)context.stop(self)}}}

Much cleaner, such satisfying.

Notice that the router in theStoreFinder tells the routees to answer to the actor that handles the query messages,broadcastRouter.route(Get(key, u), handler). Moreover, remember to capture thesender in a local variable in the main actor, before passing its reference to the inner actor.

Make certain you follow that pattern, since passing the senderActorRef without first capturing it will expose your handler to the same problem that we saw earlier where the senderActorRef changed.

Conclusions

So far so good. We started stating that context handling is not so trivial when we speak about Akka Actors. I showed you my first solution to such problem in Actorbase, the database based on the Actor model I am developing. We agreed that we do not like it. So, we moved on and we tried to useFutures. The solution was elegant but suffered from race conditions. In the path through the final solution, we met theExtra pattern, which solved the original problem without any potential drawback. The only problem is that this solution was no clean enough. Finally, we approached the Cameo pattern, and it shined in all its beauty.Simple,clean,elegant.