Positive Incline Mike Burrows (@asplake) moving on up, positively

May 13, 2009

Partial template expansion in described_routes

Filed under: Web Integration — Tags: , , , , — Mike @ 9:36 pm

In between a trip to the local dump, another meeting with HR (role officially redundant now, though still on the payroll for a few weeks) and getting our signatures witnessed for our house sale, I added partial template expansion to described_routes today.  This is potentially a lot more important than it sounds, but before I go into that, an explanation of the functionality.

What it does

Any query parameters passed to the included Rails controller will be used to pre-populate the generated resource templates and their URI templates.  In this example, the <code>article_id</code> and <code>format</code> parameters have been replaced with actual values, leaving the <code>article_id</code> unchanged:

$ curl "http://localhost:3000/described_routes/user.text?format=json&user_id=dojo"
user                 user                 GET, PUT, DELETE       http://localhost:3000/users/dojo.json
  edit               edit_user            GET                    http://localhost:3000/users/dojo/edit.json
  articles           user_articles        GET, POST              http://localhost:3000/users/dojo/articles.json
    new_user_article new_user_article     GET                    http://localhost:3000/users/dojo/articles/new.json
    recent           recent_user_articles GET                    http://localhost:3000/users/dojo/articles/recent.json
    {article_id}     user_article         GET, PUT, DELETE       http://localhost:3000/users/dojo/articles/{article_id}.json
      edit           edit_user_article    GET                    http://localhost:3000/users/dojo/articles/{article_id}/edit.json
  profile            user_profile         GET, PUT, DELETE, POST http://localhost:3000/users/dojo/profile.json
    edit             edit_user_profile    GET                    http://localhost:3000/users/dojo/profile/edit.json
    new              new_user_profile     GET                    http://localhost:3000/users/dojo/profile/new.json

More typically, JSON, YAML or XML format would be requested.  Here’s a JSON example (after pretty printing – the original output was all on one line):

$ curl "http://localhost:3000/described_routes/user_articles.yaml?user_id=dojo&format=json"
{
   "name":"user_articles",
   "rel":"articles",
   "path_template":"/users/dojo/articles.json",
   "uri_template":"http://localhost:3000/users/dojo/articles.json",
   "options":["GET", "POST"],
   "resource_templates":[
      {
         "name":"new_user_article",
         "options":["GET"],
         "path_template":"/users/dojo/articles/new.json",
         "uri_template":"http://localhost:3000/users/dojo/articles/new.json",
         "rel":"new_user_article"
      },
      {
         "name":"recent_user_articles",
         "options":["GET"],
         "path_template":"/users/dojo/articles/recent.json",
         "uri_template":"http://localhost:3000/users/dojo/articles/recent.json",
         "rel":"recent"
      },
      {
         "name":"user_article",
         "resource_templates":[
            {
               "name":"edit_user_article",
               "options":["GET"],
               "path_template":"/users/dojo/articles/{article_id}/edit.json",
               "uri_template":"http://localhost:3000/users/dojo/articles/{article_id}/edit.json",
               "rel":"edit",
               "params":["article_id"]
            }
         ],
         "options":["GET", "PUT", "DELETE"],
         "path_template":"/users/dojo/articles/{article_id}.json",
         "uri_template":"http://localhost:3000/users/dojo/articles/{article_id}.json",
         "params":["article_id"]
      }
   ]
}

Putting it to use, and why it’s important

Up to now, we’ve focussed on describing whole applications, providing client applications with metadata that helps them navigate the application effectively, even driving client APIs such as those generated dynamically by path-to.  This is an efficient way of doing things and it maps well onto the way familiar web APIs work now, but can we unify it with the RESTful concepts around hypertext (HATEOAS and all that)?  I think we can.

Imagine putting in the <head> section of the HTML representation of John Doe’s article collection at http://localhost:3000/users/dojo/articles a <link> element that references its own specific metadata at http://localhost:3000/described_routes/user_articles.json?user_id=dojo&format=json. A client reading that document can follow that link and now has in its possession the recipe for accessing all of the resources that belong to that collection – whether through complete URIs (e.g. for recent articles) or through parameterised templates (e.g. for specific articles, identified by id).

We can improve on this by returning these resource-specific links in HTTP headers. This makes them independent of the representation format (<head> and <link> are HTML-specific) and will work even on requests that return no body at all, HEAD in particular.  At the cost of one header (easily suppressed for clients that don’t need it), every resource is uniquely enhanced, minimising the amount of state that needs to be retained in the client.  For very lightweight clients (e.g. JavaScript libraries embedded in the browser) or for applications lacking a stable overall structure, this could be very useful indeed.

A mini path-to app centered on Mr Doe’s article collection could look something like this:

require "path-to/described_routes"
articles = PathTo::DescribedRoutes.bootstrap(
               "http://localhost:3000/users/dojo/articles")
articles.recent.get.first
articles["hateoas-and-other-7-letter-acronyms"].get
articles.post("title" => "The power of reflection", ...)

Here the bootstrap method creates an articles object that has an API that follows the structure given in the metadata discovered via headers found at http://localhost:3000/users/dojo/articles. Easy!

11 Comments

  1. Mike –

    While deriving from the JAX-RS model, and thus being fairly Java centric, have you looked at WADL?

    Gerald

    Comment by Gerald Beuchelt — May 15, 2009 @ 3:59 am

  2. Hi Gerald,

    Yes I did look at it, though I have to admit only briefly. Given its apparent lack of traction, I’m probably not alone in that.

    I think we have to get past protocols that are in any way technology or platform centric (mine isn’t even XML-centric), and should be very nervous about generating metadata from unstable application internals.

    By separating the structure concern from the formats concern I’ve come up with a data structure so simple that it can realistically be hand-crafted (see my Delicious example), but also generates easily (just a few dozen lines of code) from what should be one of the more stable parts of a Rails application, namely its routes. And to be clear, the format is not Rails-centric, it should apply fairly generally to applications that have a correspondence between URI structure and underlying resource structure, which is a big class of application.

    And the formats concern will stay separate. No apologies here for modest ambitions – HTTP has all the mechanisms it needs for that already!

    But thanks for the feedback, I haven’t completely given up the search…

    Mike

    UPDATE: I went back to WADL, and see that it is very strongly coupled to URI structure, not resource structure; they missed a trick not using an indirection such as URI Templates. Why for example the distinction between URI parameters and query parameters? I could go on, but it’s really not a coupling-minimising approach.

    Comment by Mike — May 15, 2009 @ 7:20 am

  3. Mike,

    I’d like to understand your objections to WADL better. WADL uses URI templates so I don’t understand why you say it misses a trick by not using them ? I also don’t understand what you mean by it being coupled to URI structure rather than resource structure. Sure it helps an application construct the right URIs (hence the distinction between path parameters and query parameters) but it also allows description of resource representations and the links they contain. Perhaps my latest blog entry on WADL would help clear up some misunderstandings: http://weblogs.java.net/blog/mhadley/archive/2009/04/hateoas_with_wa.html

    Marc.

    Comment by Marc Hadley — May 15, 2009 @ 1:44 pm

  4. Thanks Marc, I’ll give that a proper read and respond tonight or tomorrow.

    Just want to make sure that (to give a lame example) the app can switch between /users/dojo/articles/wadl, /articles?user=dojo&article=wadl and maybe even /articles/wadl with relative impunity.

    Perhaps in the meantime you could explain the rationale behind the separate identification of query parameters?

    Comment by Mike — May 15, 2009 @ 2:38 pm

  5. Quick update: clarifying a couple of things with Marc and looking at wadl.rb.

    First impressions on wadl.rb is that it makes a WADL implementation for path-to a pointless exercise (which is good thing) but it does need some TLC – its tests and some of its examples fail against Ruby 1.8.6.

    Comment by Mike — May 18, 2009 @ 10:10 am

  6. Final update:

    I got all but two tests passing, with the two remaining failures looking like problems with the tests rather than the lib. I have however given up before getting all the demos working.

    The good news is I’ve seen enough to determine that working wadl.rb would be a very much more complete Ruby client than a WADL implementation of path-to.

    Less happily, my initial comments about the coupling between WADL structure and URI structure still stand, so much so that an accurate rendition of Rails routes would require an unnatural flattening of the resource hierarchy.

    To give a further example (for simplicity’s sake pretending that {format} is mandatory),

      http://example.com/users.{format}

    can’t have a child resource of

      http://example.com/users/{user_id}.{format}

    as the URIs of nested resources are defined by paths relative to the URIs of their parents, not generated from independently-specified URI templates

    I guess the existing users of WADL are fortunate enough to have applications that can be modelled elegantly in it and don’t see this as a coupling issue in quite the way I do. I however can’t muster the enthusiasm to do the generation from Rails.

    To end on a positive note, the link stuff does look powerful. Some food for thought there!

    Comment by Mike — May 18, 2009 @ 7:52 pm

  7. […] Positive Incline Mike Burrows (asplake) moving on up « Partial template expansion in described_routes […]

    Pingback by Latest gems, URI Template update « Positive Incline — May 20, 2009 @ 3:39 pm

  8. Hi Mike,

    I really like the JSON structure you laid out to describe resources as API. Definitely HATEOAS at work!

    You could write this structure up as a http://json-schema.org/ … 😉

    Marc, I’m going to need to reread the WADL materials again to be able to compare/contrast these two API definitions. My main issue with WADL was the code-generation leading to strong coupling.

    Cheers,
    John

    Comment by John Heintz — May 28, 2009 @ 9:03 pm

  9. Thanks John, and I’ll take a look at http://json-schema.org/ today.

    Spookily (but somewhat off-topic), I read and enjoyed your Kanban presentation only yesterday!

    Comment by Mike — May 29, 2009 @ 7:25 am

  10. […] Feature-wise, it differs from Joe’s in that it supports a partial expansion mode (see this post for motivation).  Implementation-wise, it’s all done with regexps – no special parsers […]

    Pingback by Alternative URI Templates implementation in Python; fun with decorators « Positive Incline — June 4, 2009 @ 8:20 am

  11. […] list archives) but I have to say that I’m pretty happy with it.  It takes the ideas of Partial template expansion in described_routes and adds a standardised way to relate content and metadata, making it even easier clients to […]

    Pingback by Link headers, link elements, and REST « Positive Incline — June 19, 2009 @ 3:14 pm

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress