Moving on up

Tough times in the City of London, and I am “no longer required in the office”. The ugly black cloud hanging over the financial services industry (and investment banking in particular) has a major silver lining for me though: it creates the exciting opportunity to move to the beautiful Derbyshire Dales, and closer to my wife’s family. More on that closer to the time.

Meanwhile, if your organisation needs an effective technical leader (development manager, programme manager, agile project manager) with a strong sense for architecture, perhaps you should consider me. I enjoy travel, but I will need work that is either reasonably local (Chesterfield, Derby, Sheffield, Nottingham, say) or can be done mainly from home (Matlock Bath).

Why hire me?

I’ve been doing the “protocols and formats” thing for several years, with influence or direct control over integration protocols at enterprise and consortium level for the past eight or so years.  Returning to real-time front office development three years ago, I embraced REST as a tool for architectural improvement, reusability, testability, scriptability and so on.  So hire me if you need a system architecture that values these things over specific technologies.

I’m also big on learning, and although I do spend a lot of time reading and experimenting around my chosen areas of interest (to which my blog I hope testifies), this is not just for myself.  I have coached individuals and organised and empowered teams large and small for continuous improvement, and have had considerable success at improvement projects around capacity, reliability, business process and development process.  Again, some of these projects have had enterprise-wide impact.  So hire me (even for a short time) and know that I will leave your teams in good shape.

For further details, LinkedIn has a career summary, CV on request to mjb@asplake.co.uk.

Nested path-to/described_routes and HTTParty

I’ve finished the support for nested described_routes in path-to.  Example:

  require 'path-to/described_routes'
  app = PathTo::DescribedRoutes::Application.new(
      :json => Net::HTTP.get(URI.parse("http://example.com/described_routes.json")))

  app.users["user_id" => "dojo"].articles.recent
  #=> http://example.com/users/dojo/articles/recent
  app.users["user_id" => "dojo"].articles.recent.get
  #=> "<html>...</html>"

Or if you still prefer the named route:

  app.recent_user_articles("user_id" => "dojo").get   #=> "<html>...</html>"

And that’s it!  Is that a client-side API for free or what?  This just has to be better than hard-coded URIs (ugh).

I have also enhanced the Rails controller in described_routes so that you can get the JSON, YAML or XML for a given named route.  In this example we see HTTParty (path-to’s default HTTP client) parsing a JSON response (which I’ve reformatted just a little bit):

  app.described_route("route_name" => "admin_product", "format" => "json").get
  #=> {"name"=>"admin_product", "options"=>["GET", "PUT", "DELETE"],
       "resource_templates"=>[
          {"name"=>"edit_admin_product", "options"=>["GET"],
           "uri_template"=>"http://localhost:3000/admin/products/{product_id}/edit{-prefix|.|format}",
           "path_template"=>"/admin/products/{product_id}/edit{-prefix|.|format}", "rel"=>"edit",
           "optional_params"=>["format"], "params"=>["product_id"]
          }],
       "uri_template"=>"http://localhost:3000/admin/products/{product_id}{-prefix|.|format}",
       "path_template"=>"/admin/products/{product_id}{-prefix|.|format}",
       "optional_params"=>["format"], "params"=>["product_id"]}

One last thing: as previously mentioned you’ll need grab the latest addressable from github – the published gem is a fix or two behind.

described_routes at run time

The first described_routes gem went up on RubyForge yesterday.  Usual install:

  sudo gem install described_routes

then follow these very simple instructions to have your routes described in JSON, YAML (long and short versions) or XML from the command line.

Today I added a Rails controller so that you can GET any of these formats from your server at runtime.  This takes just two 1-line changes:

1) Add to your project a file <railsapp>/initializers/described_routes.rb containing the following:

  require 'described_routes/rails_controller'

2) Add this new route to your config/routes.rb file:

  map.resources :described_routes, :controller => "described_routes/rails"

You can then browse to <yourapp>/described_routes.json (or .yaml or .xml).  For the shorter yaml description, navigate to <yourapp>/described_routes.yaml?short=true.

Without a format extension you’ll get an error unless you provide a suitable HTML template, perhaps a suitable place for some human-friendly API documentation?

As an added bonus, reports generated at run time include full URI templates (I haven’t yet found out how to get to the application’s root url from Rake)

Examples:

I’ll be getting back to the path-to / described_routes integration after this.  Getting nested routes working shouldn’t be a huge task (separately, both libraries support them already) but there’s some refactoring to be done.

path-to and described_routes playing nicely – but is it RESTful?

I started to get path-to and described_routes to play together today.  With them I can:

1) Load a JSON or YAML description of an application into a generic client app like this:

  app = PathTo::DescribedRoutes::Application.new(
      DescribedRoutes.parse_json(json),
      "http://localhost")

2) See what it has to offer:

  app.resources_by_name.keys.sort.inspect
  #=> ["admin_product", "admin_products", "edit_admin_product", "edit_page",
  #    "edit_user", "edit_user_article", "edit_user_profile", "new_admin_product",
  #    "new_page", "new_user", "new_user_article", "new_user_profile", "page",
  #    "pages", "recent_user_articles", "root", "summary_page",
  #    "toggle_visibility_page", "user", "user_article", "user_articles",
  #    "user_profile", "users"]

3) Generate some URIs:

  app.users.uri
  #=> "http://localhost/users"
  app.recent_user_articles[:user_id => "dojo"].uri
  #=> "http://localhost/users/dojo/articles/recent"

4) And access them:

  app.users.get
  app.recent_user_articles[:user_id => "dojo"].get

OK I lied about that last bit – my server app is no more than Rails boilerplate with described_routes and an example config/routes.rb, but (thanks to the HTTP support in path-to provided by HTTParty) it would have worked, I promise!

Next Steps

Next up is properly integrating the chaining stuff in path-to.  I want that last example to look more like

  app.users[:user_id => "dojo"].articles.recent.get

Is it REST?

stilkov: @christo4ferris There’d only be HATEOAS in that REST API if the URI templates were retrieved from the server dynamically

Indeed, and I make no great claims that this style of interaction is 100% RESTful, though when you look again at

  app.users[:user_id => "dojo"].articles.recent.get

can you be sure whether it is or it isn’t?  Who knows what’s going on behind the scenes?  The point really is that given some minimal discoverability in the server, the predictability of the REST and a good scripting language, tricks like this are easy to pull off.  And that’s way better than the highly coupled client-side URI generation splattered all over many of the so-called RESTful examples I see out there on the web.

If all of that sounds defensive, be assured that I’d be only too delighted to see this superseded by something of similar power that has more universal acceptance.  As far as I can tell though, right now that’s a little way off.

described_routes: hierarchical, framework-neutral and machine-readable descriptions of Rails routes

I’ve pushed to github (rubyforge approval is still pending) a new gem described_routes which produces machine-readable (JSON, YAML, XML and Ruby) representations of the resources supported by your Rails app.

It’s very easy to use: just drop this into your project’s Rakefile:

  require 'tasks/described_routes'

Then see:

  $ rake --tasks described_routes
  rake described_routes:json        # Describe resource structure in JSON format
  rake described_routes:ruby        # Describe resource structure as a Ruby literal
  rake described_routes:xml         # Describe resource structure in XML format
  rake described_routes:yaml        # Describe resource structure in YAML format
  rake described_routes:yaml_short  # Describe resource structure in YAML format (basic structure only)

To get a flavour, check out the short form below. See how (1) the resource hierarchy is represented, and (2) (with the “rel” attributes) it has picked out named relationships between parent and child resources:

  $ rake described_routes:yaml_short
  - name: users
    path_template: /users{-prefix|.|format}
    resources:
    - name: user
      path_template: /users/{user_id}{-prefix|.|format}
      resources:
      - name: user_articles
        rel: articles
        path_template: /users/{user_id}/articles{-prefix|.|format}
        resources:
        - name: user_article
          path_template: /users/{user_id}/articles/{article_id}{-prefix|.|format}
          resources:
          - name: edit_user_article
            path_template: /users/{user_id}/articles/{article_id}/edit{-prefix|.|format}
            rel: edit
        - name: new_user_article
          path_template: /users/{user_id}/articles/new{-prefix|.|format}
          rel: new
        - name: recent_user_articles
          path_template: /users/{user_id}/articles/recent{-prefix|.|format}
          rel: recent
      - name: edit_user
        path_template: /users/{user_id}/edit{-prefix|.|format}
        rel: edit

The longer forms include “options” (the HTTP methods applicable to each resource) and parameter lists.

The path templates are URI Templates but relative to the application’s base. If someone can show me how to derive that base URI at Rake time, please let me know!

Do please note that there’s nothing Rails-specific about these formats. Although they’re potentially useful as project documentation, they’re really for the benefit of client applications, and clients of course shouldn’t care how their respective servers were built!  My next step is to add client support for one or more of these formats in my path-to gem (announcement, github, rubyforge).

path-to is born: nice client-side APIs to web applications in Ruby

An initial cut of path-to is now on github.  There’s no gem on Rubyforge just yet but it’s not far away.

Path-to’s basic premise is that it should be easy to describe client interfaces to typical web applications (RESTful ones especially), and have created for you a Ruby API, even in the absence of a discoverable representation of the web API on the server side. My longer term aim is to identify convenient discovery protocols, have them implemented in the server apps I have influence over, and (naturally) include support for them in path-to.

A very simple example (expanded in the README) looks like this:

    require "path-to"

    app = Application.new(
            :users    => "http://example.com/users/{user}",
            :articles => "http://example.com/users/{user}/articles/{slug}")

    app.users                                                   #=> http://example.com/users/
    app.users(:user => "dojo")                                  #=> http://example.com/users/dojo
    app.users[:user => "dojo"]                                  #=> http://example.com/users/dojo
    app.articles(:user => "dojo", :slug => "my-article")        #=> http://example.com/users/dojo/articles/my-article
    app.users[:user => "dojo"].articles[:slug => "my-article"]  #=> http://example.com/users/dojo/articles/my-article

A key theme of path-to is chaining, so that if app.x represents a resource, then so (typically) do app.x.y and app.x[params]. But you’re not forced into this style: you could go straight for app.x(params) which involves one less object under the covers. Furthermore, you have control over the classes of the objects created, and are free to override #[] so that you can (for example) have calls like

    app.users["dojo"].articles["my-article"]                    #=> http://example.com/users/dojo/articles/my-article

Another important theme is that client applications shouldn’t be forced to follow my preferences and dependencies. For example, I’m testing it with HTTParty (so that app.x.get is delegated to HTTParty.get), but you don’t have to. Similarly, it’s easy to swap out the URI Template support I’m getting from Addressable – which is just as well, as I’m entertaining serious second thoughts about the whole URI Templates thing!

Avoiding overcommitment

A couple of interesting posts came up in my morning reading today, one from Glen Alleman (Herding Cats) on The Mathematics of Project Management, the other from Jurgen Appelo (NOOP.NL) on How to Do Things Your Customer Didn’t Ask For.

I see iterative development as one tool that helps raise the certainty in development. Another is knowing what commitments I can sensibly make as a project manager, or conversely, how much room for manoeuvre do I need to keep.

There is no conflict between on the one hand managing projects iteratively and positively in a customer-centric way, and on the other (behind the scenes) doing the math. Quite the opposite in fact: by looking ahead, paying close attention to critical work, being able to align commitments to real or imagined deadlines with some level of risk quantification, our clients can deal with us with much greater confidence.

If Glen’s approach seems too much (and it doesn’t have to be intimidating – see this Joel Spolsky classic) I give you the wiggle room calculator and release calculator. They’re just crude, rule-of-thumb tools really, but 5 minutes experimentation with them might just help you avoid an overcommitment trap.

And no, avoiding overcommitment doesn’t mean a descent into negative (dare I say stereotypical?) project management. But that’s for another post…

Agile wiggle room calculator

What could better measure your release’s agility than a wiggle room calculator? It’s a companion to the release calculator described in an earlier post.

The release calculator’s job was to work out how many iterations you need to complete your critical functionality with 95% certainty. The wiggle room calculator assumes instead that you’ve roughly sized your release already and tells you how much of your remaining iterations can be committed to critical functionality and (conversely) how much you should keep up your sleeve. This contingency can be kept for the unforeseen and – fingers crossed – used later for the nice-to-have stuff.

The contingency calculation looks a little more involved this time. All it is though is a rearrangement of the original calculations having eliminated the “ri” variable.

Try out the wiggle room calculator now!

P.S. Many thanks to Mike Cohn for his feedback on the original calculator. I’ve made some changes already as a result; longer term I’ll allow for the entry of historical velocities from multiple iterations so that average velocity and variation can be calculated for you.