In this article I’m approaching REST from the perspective of client/server interaction, most especially (and slightly unusually I think) on the needs of the client and you, the client’s developer. I hope you find it helpful.
A caveat (no apology): my interest is in structure and navigation. HTTP has mechanisms already for clients and servers to negotiate content types and I’ve made a conscious design decision to rely on their existence. In other words, I ignore it completely here. Orthogonality is a wonderful thing!
We start at the beginning:
The completely hand-crafted client
At the most extreme, you’re working against an undocumented, unsupported server API that lacks regular structure. Any complexity implies significant reverse-engineering effort, made worse if you’re tracking a moving target. If you’re lucky though, you’ll be working against a supported and stable API.
Relying on regularity
Servers built using frameworks such as Rails (and even many that aren’t) will tend to exhibit some predictable patterns. The details may be framework-specific, but a collection resource, say “users”, can be expected to support these (or similar) operations and subresources:
GET, POST http://www.example.com/users
GET http://www.example.com/users/new
GET, PUT, DELETE http://www.example.com/users/{user_id}
GET http://www.example.com/users/{user_id}/edit
and a nested collection resource (each user’s articles, say) can be expected to follow a similar structure, rooted on a specific user resource.
GET, POST http://www.example.com/users/{user_id}/articles
GET http://www.example.com/users/{user_id}/articles/new
GET, PUT, DELETE http://www.example.com/users/{user_id}/articles/{article_id}
GET http://www.example.com/users/{user_id}/articles/{article_id}/edit
Armed therefore with just the knowledge of the user and articles collections, your client’s internal object model might easily support expressions such as
app.users
app.users.new
app.users[user_id]
app.users[user_id].edit
app.users[user_id].articles
app.users[user_id].articles.new
app.users[user_id].articles[article_id]
app.users[user_id].articles[article_id].edit
You’ll pay a price when the application departs from its usual pattern, but most of the time you’ll be fine. In fact for many client applications the edit and new are irrelevant, so all you need is the basic hierarchical structure and knowledge of the representations you can GET, PUT, POST there, also how to DELETE things.
Model-driven development
One approach to speeding application development is to abstract out that object model and generate skeleton code for both client and server from some common, logical representation. Not only are you spared the task of building the object model, but (given the right toolset) you can expect to have the web client functionality baked in for you. Most importantly, it’s much cheaper to track any changes made on the server.
The downside? You are now tied to a toolset. That might be a price worth paying if you’re in control of both client and server (for an internal application, say), but for many this will be unpalatable.
The no-model option
[Stick with me - this apparent digression makes metadata-driven clients easier to introduce!]
Dynamic languages make it possible to support expression like
app.users[user_id].articles.new
even if your app object doesn’t actually have a users member (method, attribute or property) and articles and new exist nowhere either. See [1] and [2] to read how (generally) this can be done in Ruby and Python, and [3] for a description of path-to [4], an actual client that can work this way.
Even without these dynamic features, a similar effect can be achieved with operator overloading. This, for example, is perfectly valid Java:
app / users / member(user_id) / articles / _new
The objects returned by such expressions can easily have a URI representation, request methods and so on.
But beware! Suppose a user resource defines an “up” link. Take for example
app.users["dojo"].up
To which of these URIs (below) does this expression refer?
http://www.example.com/users/dojo/up
http://www.example.com/users
We’ll come back to this issue later.
Metadata-driven clients
You can take the no-model approach and validate each navigation against it, so that for a non-existent path foo,
app.foo
raises an error just as the model-driven version would have done. To your client application code, this implied object model looks no different to the generated one, even though (in a way) it doesn’t really exist! I like to think of this as the model-driven approach turned inside-out - instead of running code generated from a model, we run code that interprets a model.
If that metadata comes from the server (so much better than a build-time configuration step), you’re getting closer to that RESTful goal of application interaction driven by links, or
Hypertext as the engine of application state (HATEOAS) [5]
But before claiming this goal genuinely you will need to address a couple of issues that we’ve glossed over so far:
- how to generate URIs for collection members, for example turning
users["dojo"] into http://example.com/users/dojo
- how to generate URIs for navigations like
up and first
The cleanest answer to the first issue is the URI Template (a parameterised URI with a standardised syntax), sourced (naturally) from the server. With Fielding’s quote in mind it’s clear that the resolution to the second issue must lie also in server-provided information. In fact we’ll see both issues come together as we understand that the problem really isn’t one of describing URI structure but instead one of describing resource relationships and their manifestations in links.
URI Templates and Resource Templates
A URI Template that maps users["dojo"] into http://example.com/users/dojo might look this this:
http://example.com/users/{user_id}
The syntax isn’t that important (the draft standard looks set to change significantly), and yes, we’ve seen them here already without introduction. They’re pretty self-documenting and (with a good library implementation) easy to use. If you stick to a standard there’s nothing unRESTful about them - having your application create a URI from template is really no different from a browser creating a URI as the result of GET-based HTML form.
In more complex cases - in particular where there is a mixture of mandatory and optional parameters - it’s advantageous to wrap the URI Template in some basic metadata. WADL [6] is a well-supported example of this idea (well almost - see the comments [7]) ; another is my own Resource Template format [8] (actually more a logical schema than a format since it’s available in JSON, YAML and XML).
WADL and Resource Templates typically describe an entire application, each element describing a class of resource, its relationships, its URI template (so that relationships can be turned into links), parameters, supported HTTP methods and so on.
Integrated with Rails, Resource Templates are generated by the server at run-time from the application’s routes, information already available. Although (most typically) they can be generated and consumed for the whole application in one go, they can be generated for a single resource already pre-populated with that specific resource’s known parameters.
At last: links!
From those pre-populated, resource-specific Resource Templates we can generate links - as HTML or XML elements or as HTTP headers - that our client can consume and navigate, resource by self-described resource. Every resource becomes a potential entry point to a virtual application that can easily span multiple servers. Finally, we got there: true hyperlink-based client/server interaction, and where the framework support exists, for very little effort too.
Discussion
In this article I have described the journey I travelled myself over a period of weeks following my decision to expose a complex existing application to scripting via HTTP. For me, technology-neutrality was a determining principle, and selecting HTTP and REST was an easy decision, first to make and then to sell. A proof of concept and a good demo based on a hand-crafted object model came quickly; metadata and templates were the result of some serious refactoring. Resource Templates came after I left the project, as did Rails integration (not a project requirement) and the path-to client in both Ruby and Python.
At the time of writing, link generation is supported on the server side but not yet on the client. The purist in me likes the fact that it is theoretically possible and will strive to keep it that way; the engineer in me is for the moment content with slurping up a whole application’s worth of metadata in one go and thereafter restricting interactions and payloads to the application level. Even as a purist, I see elegance in a simple metadata format, avoidance of hand-coded generation of URIs, respect for the power of HTTP and the orthogonality of formats; and if you believe as I do that RESTfulness lies in how clients and servers interact and not just in how server resources support the HTTP methods, I’ve probably succeeded in taking it further than most.
References
- Dynamically extending object behaviour in Ruby and Python [positiveincline.com]
- Programmatically adding methods to classes and objects: more Ruby/Python comparisons [positiveincline.com]
- path-to is born: nice client-side APIs to web applications in Ruby, also articles tagged path-to [positiveincline.com]
- path-to (Ruby) and path-to-py (Python) [github.com]
- Representational State Transfer (REST) (Chapter 5 of Architectural Styles and the Design of Network-based Software Architectures) [www.ics.uci.edu]
- Web Application Description Language [wikipedia.com]
- Partial template expansion in described_routes (comments)” [positiveincline.com]
- described_routes: hierarchical, framework-neutral and machine-readable descriptions of Rails routes [positiveincline.com], described_routes (Ruby) and described_routes-py (Python) [github.com]