Repository files navigation

Snooze is an URL router for Common Lisp designed aroundREST webservices.

An URL router lets you open URL routes to your application that arefriendlier, easier to remember and better supported by otherapplications, such as search engines. RESTful routes are near universalin web APIs andlook like this.

AllSnooze does is establish a tight fit between this type ofroute and plain old Common Lisp. For example, inSnooze, routesare just functions and HTTP conditions are just Lisp conditions.

Since you stay inside Lisp, if you know how to make a function,you know how to make a route.There are no regular expressions towrite or extra route-defining syntax to learn.

Snooze is web-server-backend-agnostic: it canwork with any web server.

Here's an example you can try out quickly: a micro-REST service toread and write Lisp docstrings over HTTP:

(defpackage#:readme-demo (:use#:cl#:snooze))(in-package#:readme-demo)(defunfind-symbol-or-lose (namepackage)  (or (find-symbol (string name) (find-packagepackage))      (http-condition404"Sorry, no such symbol")))(defroute lispdoc (:get:text/* name&key (package:cl) (doctype'function))  (or (documentation (find-symbol-or-lose namepackage) doctype)      (http-condition404"Sorry,~a doesn't have any~a doc" name doctype)))(defroute lispdoc (:post:text/plain name&key (package:cl) (doctype'function))  (setf (documentation (find-symbol-or-lose namepackage) doctype)        (payload-as-string)));; Let's use clack as a server backend(clack:clackup (snooze:make-clack-app):port9003)

This establishes two routes (GET for reading andPOST for writing)on the URIlocalhost:9003/lispdoc/<symbol>. Here's an illustrationof how they respond:

GET /lispdoc/defun                         => 200 OKGET /lispdoc/funny-syntax?package=snooze   => 404 Not foundGET /lispdoc/in/?valid=args                => 400 Bad RequestGET /lispdoc/defun                         => 406 Not AcceptableAccept: application/jsonPOST /lispdoc/scan?package=cl-ppcre        => 200 OKContent-type: text/plainPOST /lispdoc/defun                        => 415 Unsupported Media TypeContent-type: application/json

The error codes 400, 406 and 415 are error reporting that you get "forfree": if the HTTP client strays off these routes, be it for impropersyntax or unsupported content types, the correct HTTP condition issignalled.

The rest of this README contains therationale forbuildingSnooze and atutorial that builds on thesimple application presented above.

Ah,Snooze is kindaBETA. The usual disclaimer of warrantyapplies.

There are already some Common Lisp systems for HTTP routing, likecaveman,cl-rest-server,restasandningle. Unfortunately, they tend to make you learn someextra route-defining syntax.

On the contrarySnooze mapsREST/HTTP concepts to CommonLisp concepts:

This has many advantages, for example

• since every route is a method, you cantrace it like a regularfunction, find its definition withM-. or even use:aroundqualifiers;
• using a regular lambda-list guarantees that URI errors can bespotted early by your compiler;
• there is no need to write code to "extract" arguments from theURI.
• SinceSnooze knows the lambda-list of a route, it can use itto do the reverse of URI matching: generate URIs that perfectlymatch that same route.

Consider the code sample presentedabove. Let's pick upwhere we left off, and build a bit more oflispdoc, ourdocstring-manipulating application. We'll see how to:

• understandSnooze's REST resources
• dispatch on HTTP methods and content-types
• generate and encode compatible URIs
• grafully handle failure conditions
• control conversion of URI arguments
• refactor routes without changing the API
• hookSnooze into the backend of your choice

This tutorial assumes you're using a recent version ofquicklisp so start by entering this into your REPL.

(push"path/to/snoozes/parent/dir"quicklisp:*local-project-directories*)(ql:quickload:snooze)

Make sure you keep an eye on the docstrings of the functionsmentioned,they are where the real API reference lives. Find themall, appropriately, in theapi.lispfile.

An important detail that was elided from the initial sample is that,inSnooze:

• a RESTresource is implemented a CLOS genericfunction.
• Theoperations (GET,POST,DELETE, etc...) accepted by aresource are implemented as CLOS methods on that generic function

When a HTTP request is received,Snooze arranges for its URI to betranslated into a generic function name and its remaining properties(verb, content-type, additional URI bits) to be translated intoarguments for that function.Snooze then calls that function withthose arguments and CLOS does the rest:

• if one of the methods of this generic function matches, its body iscalled and the HTTP client sees a nice response;

• otherwise a condition is signalled andSnooze takes care that theHTTP client sees the correct error code.

Under the hood,defroute is actually a really thin wrapper ondefmethod. You can even usedefmethod directly if you prefer:

(defmethodlispdoc            ((snooze-verbs:http-verbsnooze-verbs:post)             (snooze-types:typesnooze-types:text/plain) name&key (package:cl) (doctype'function))   (setf (documentation (find-symbol-or-lose namepackage) doctype)         (payload-as-string)))

Likewise there is adefresource form that is equivalent todefgeneric. It may be left out since it is implicit in the firstdefroute call.

This means we could have defined the above application in anequivalent terser form:

(defresource lispdoc (verb content-type name&key)  (:route (:get:text/* name&key (package:cl) (doctype'function))    (or (documentation (find-symbol-or-lose namepackage) doctype)        (http-condition404"Sorry,~a doesn't have any~a doc" name doctype)))  (:route (:post:text/plain name&key (package:cl) (doctype'function))    (setf (documentation (find-symbol-or-lose namepackage) doctype)          (payload-as-string))))

Let's start by serving docstrings in HTML. As seen above, we alreadyhave a route which serves plain text:

(defroute lispdoc (:get:text/* name&key (package:cl) (type'function))  (or (documentation (find-symbol-or-lose namepackage)type)      (http-condition404"Sorry no~a doc for~a"type name)))

To add a similar route for the content-typetext/html, we justnotice thattext/htmlistext/*. Also because routes are reallyonly CLOS methods, the easiest way is:

(defroute lispdoc:around (:get:text/html name&key&allow-other-keys)  (formatnil"<h1>Docstring for~a</h1><p>~a</p>"          name (call-next-method)))

This will do fine for now. Of course, later we should probably escapethe HTML with something likecl-who'sescape-string-all. We might also consider removing the:aroundqualifier and use a helper function shared by two routes.

Let's try our hand at implementing an important part of the API:POST requests with JSON content:

(defroute lispdoc (:post"application/json" name&key (package:cl) (doctype'function))  (let* ((json (handler-case;; you'll need to quickload :cl-json                   (json:decode-json-from-string                    (payload-as-string))                 (error (e)                   (http-condition400"Malformed JSON (~a)!" e))))         (sym (find-symbol-or-lose namepackage))         (docstring (cdr (assoc:docstring json))))    (if (and sym docstring doctype)        (setf (documentation sym doctype) docstring)        (http-condition400"JSON missing some properties"))))

Our application has a growing number of routes that work fine,provided the use knows how to type them. Because this is increasinglycomplicated as more and more routes are added, it is very often thecase that we'll want parts of the our REST application to generateURI's that match its own routes. Probably, the most common case isproviding a link to a specific resource in an HTML response.

This is very easy to do inSnooze, as it can automatically generatethe URIs for a resource. You first need to get a "genpath" functionfor your resource. Just do:

(defgenpath lispdoc lispdoc-path)

Or, alternatively, pass:genpath todefresource.

(defresource lispdoc (verb ct symbol) (:genpath lispdoc-path))

The newly generatedlispdoc-path has an argument list that perfectlymatches your route's arguments:

(lispdoc-path'defroute:package'snooze);; => "/lispdoc/defroute?package=snooze"(lispdoc-path'defun);; => "/lispdoc/defun"(lispdoc-path'*standard-output*:doctype'variable);; => "/lispdoc/%2Astandard-output%2A?doctype=variable"(lispdoc-path'*standard-output*:FOO'hey);; error! unknown &KEY argument: :FOO

Notice the automatic URI-encoding of the* character and how thefunction errors on invalid keyword arguments that would produce aninvalid route.

Path generators are useful, for example, when write HTML links to yourresources. In our example, let's use it to guide the user to thecorrect URL when an easily-fixed 404 happens:

(defundoc-not-found-message (namepackage doctype)  (let* ((othertype (if (eq doctype'function)'variable'function))         (otherdoc (documentation (find-symbol-or-lose namepackage) othertype)))    (with-output-to-string (s)      (format s"There is no~a doc for~a." doctype name)      (when otherdoc        (format s"<p>But try <a href=~a>here</a></p>"                (lispdoc-path name:packagepackage:doctype othertype))))))(defroute lispdoc (:get:text/html name&key (package:cl) (doctype'function))  (or (documentation (find-symbol-or-lose namepackage) doctype)      (http-condition404 (doc-not-found-message namepackage doctype))))

If you now point your browser to:

http://localhost:9003/lispdoc/%2Astandard-output%2A?doctype=variable

You should see a nicer 404 error message.Except you don't (!),because, by default,Snooze is very terse with error messages and wehaven't told it not to be. So don't worry, the next sections explainshow to change that.

Errors and unexpected situations are part of normal HTTP life. Manywebsites and REST services not only return an HTTP status code, butalso serve information about the conditions that lead to an error, beit in a pretty HTML error page or a JSON object describing theproblem.

Snooze tries to make it possible to precisely control whatinformation gets sent to the client. It uses a generic function andtwo variables:

• explain-condition (condition resource content-type)
• *catch-errors*
• *catch-http-conditions*

Out of the box, there are no methods onexplain-condition and thetwo variables are set tot.

This means that any HTTP condition or a Lisp error in your applicationwill generate a very terse reply in plain-text containing only thestatus code and the standard reason phrase.

You can amend this selectively by addingexplain-conditionmethods that explain HTTP conditions politely in, say, HTML:

(defmethodexplain-condition ((condition http-condition)                              (resource (eql#'lispdoc))                              (ctsnooze-types:text/html))               (with-output-to-string (s)                 (format s"<h1>Terribly sorry</h1><p>You might have made a mistake, I'm afraid</p>")                 (format s"<p>~a</p>"condition)))

The above explains only HTTP conditions that are the client's fault, but you can use the same technique to explainany error, like so:

(defmethodexplain-condition ((errorerror) (resource (eql#'lispdoc)) (ctsnooze-types:text/html))               (with-output-to-string (s)                 (format s"<h1>Oh dear</h1><p>It seems I've messed up somehow</p>")))

Finally, you can play around with*catch-errors* and*catch-http-conditions (see their docstrings). I normally leave*catch-http-conditions* set tot and*catch-errors* set toeither:verbose ornil depending on whether I want to do debuggingin the browser or in Emacs.

You might have noticed already that the arguments passed to the CLOSgeneric functions that represent resources are actual Lisp symbolsextracted from the URI, whereas other frameworks normally pass them asstrings.

What are the advantages of this? Let's drift from thelispdocexample a bit. Consider this fragment of a Beatle-listing app.

(defclassbeatle () ((id:initarg:id)                     (name:initarg:name:accessor name)                     (guitars:initarg:guitars:accessor number-of-guitars)))(defparameter*beatles*           (list (make-instance'beatle:id1:name"John":guitars1)                 (make-instance'beatle:id2:name"Paul":guitars2)                 (make-instance'beatle:id3:name"Ringo":guitars0)                 (make-instance'beatle:id4:name"George":guitars10)))(defroute beatles (:get"text/plain"&key (key'number-of-guitars) (predicate'>))  (assert-safe-functions key predicate)  (formatnil"~{~a~^~%~}"          (mapcar#'name                  (sort (copy-list*beatles*) predicate:key key))))(defgenpath beatles beatles-path)

Thedefgenpath form makesbeatles-path be a function of twokeyword arguments,:key and:predicate that returns the perfectURI for accessing thebeatles route. Among other things you can nameregular functions (like< andstring-lessp in this example) bytheir symbols, as you would in pure Lisp.

CL-USER> (beatles-path:key'number-of-guitars:predicate'<)"/beatles/?key=number-of-guitars&predicate=%3C"CL-USER> (beatles-path:key'name:predicate'string-lessp)"/beatles/?key=name&predicate=string-lessp"

Sure enough, feeding these URIs to the HTTP client causes the functionbeatles to be called with exactly the same symbols that you passedtobeatles-path.

Now, if you're thinking that this doesn't fit needs, know that it ismerely a default behaviour, and entirely configurable: if you reallywant to have the URI pathfoo/bar/baz become the strings"foo","bar" and"baz" in your application you merely need to add a CLOSmethod to the each of the generic functionsread-for-resource andwrite-for-resource.

Nevertheless, I recommend you keep the default:

• The defaultread-for-resource uses a very locked down version ofcl:read-to-string that doesn't intern symbols (for security),allow any kind of reader macros or read anything more complicatedthan a number, a string or a symbol.

• The defaultwrite-for-resource does the inverse: it writes onto astring of any object so thatread-for-resource can reconstructthat object from the string (so long as the object is a secure thingto serialize over URI).

There is perhaps a better way to influence the mapping between URIsand arguments. To that effect, two other functions are discussed inthe next section:arguments-to-uri anduri-to-arguments.

Let's recall thelispdoc app. The routes we have until now arefunctions of a string. To convert them into actual symbols they need:

• thefind-symbol-or-lose helper;
• an additional:package keyword arg.

This isn't pretty: it would be nicer if routes were functions of asymbol. After all, in Common Lisp, passing symbols around shouldn'tforce you to pass their packages separately!

So basically, we want to write our methods like this:

(defroute lispdoc (:get:text/* (symsymbol)&key (doctype'function))  (or (documentation sym doctype)      (http-condition404 (doc-not-found-message sym doctype))))

Actually, this will workjust fine out of the box. Oh wait, now our routeslook like this:

(lispdoc-path'cl-ppcre:scan);; => "/lispdoc/cl-ppcre%3Ascan"(lispdoc-path'ql:quickload);; => "/lispdoc/quicklisp-client%3Aquickload"

Compare these to the routes at the very top of this document:

  /lispdoc/scan?package=cl-ppcre  /lispdoc/quickload?package=ql

You might be dissapointed that the new ones are not as human-readable(the%3A encoding for the: looks would look slightly bizarre to auser reading it in the browser's address bar). But even if you don'tcare about appearance and find them perfectly acceptable, it isconceivable that we had already published the routes of the older RESTAPI to the world.

So, to keep that API, we need to change the implementation withoutchanging the interface. This is whereuri-to-arguments and itsreciprocalarguments-to-uri come in handy. These generic functionshave default implementations for all resources that can be leveragedfor surgical tweaks like the one we need here:

• uri-to-arguments receives an URI string, and make it compute avalues-list of "plain" and keyword arguments that are passed to theroute (after the verb and content type). In our case, we first parsethe plain symbol name and:package usingcall-next-method, thencompute an actual symbol. We also make sure to not pass:packageto our new route, as it doesn't accept it.

• arguments-to-uri is the function that allows thegenpathfunction to produce matching URIs. It does the reverse, producing anURI string. In this case it takes a symbol inplain-args, Afterextracting its package and massaging it into an uninterned symbol,we also usecall-next-method to simplify things.

(defmethoduri-to-arguments ((resource (eql#'lispdoc)) uri)  (multiple-value-bind (plain-args keyword-args)      (call-next-method)    (let* ((sym-name (string (first plain-args)))           (package-name (or (cdr (assoc:package keyword-args))'cl))           (sym (find-symbol sym-namepackage-name)))      (unless sym        (http-condition404"Sorry, no such symbol"))      (values (cons sym (cdr plain-args))              (remove:package keyword-args:key#'car)))))(defmethodarguments-to-uri ((resource (eql#'lispdoc)) plain-args keyword-args)  (let ((sym (first plain-args)))    (call-next-method resource                      (list sym)                      (cons`(:package.,(make-symbol                                           (package-name (symbol-package sym))))                            keyword-args))))

We can now safely rewrite the remaining routes in much simplerfashion. Here's the rest of the application now (notice also howdoc-not-found-message was also simplified)

(defundoc-not-found-message (symbol doctype)  (let* ((othertype (if (eq doctype'function)'variable'function))         (otherdoc (documentationsymbol othertype)))    (with-output-to-string (s)      (format s"There is no~a doc for~a." doctypesymbol)      (when otherdoc        (format s"<p>But try <a href=~a>here</a></p>"                (lispdoc-pathsymbol:doctype othertype))))))(defroute lispdoc (:get:text/* (symsymbol)&key (doctype'function))  (or (documentation sym doctype)      (http-condition404"No doc found for~a" sym)))(defroute lispdoc (:post:text/plain (symsymbol)&key (doctype'function))  (setf (documentation sym doctype)        (payload-as-string)))(defroute lispdoc (:get:text/html (symsymbol)&key (doctype'function))  (or (documentation sym doctype)      (http-condition404 (doc-not-found-message sym doctype))))(defroute lispdoc (:post:application/json (symsymbol)&key (doctype'function))  (let* ((json (handler-case                   (json:decode-json-from-string                    (payload-as-string))                 (error (e)                   (http-condition400"Malformed JSON! (~a)" e))))         (docstring (cdr (assoc:docstring json))))    (setf (documentation sym doctype) docstring)))

Snooze is web-server agnostic: it's just an URL router. It doescome with two utility functions,make-clack-app andmake-hunchentoot-app that will plug into two popular web servers andquickly let you jump into the action:

;;; Use hunchentoot directly(push (snooze:make-hunchentoot-app)hunchentoot:*dispatch-table*)(hunchentoot:start (make-instance'hunchentoot:easy-acceptor:port9003));;; Use clack(clack:clackup (snooze:make-clack-app):port9003)

But Snooze doesn't "require" Clack or Hunchentoot in any sense. So ifyou want to use any other backend, I suggest you take a look at theimplementations ofmake-hunchentoot-app andmake-clack-appfunctions, particularly their use ofsnooze:handle-request.

To ask questions, report bugs, or just discuss matters open anissue or send me email.