Positive Incline Mike Burrows (@asplake) moving on up, positively

December 21, 2009

DRY up your routes – a Pylons routing refactoring

Filed under: Programming,Web Integration — Tags: , , , , , — Mike @ 11:43 am

[See UPDATES]

This post is in several acts, each one a refactoring. Taking the stage will be some familiar-looking application code; behind the scenes lurks some enhanced framework code that makes the refactorings possible.

Our story starts with an old-school (pre-REST) routing configuration. It’s in Python, for Pylons and other Routes-based web frameworks, but let me reassure the Ruby audience that Routes borrows very heavily from Rails and their routing configurations look quite similar.

We will finish with collection(), an experimental (but I hope worthy) alternative to resource(), the routing helper that rose to prominence when Rails first made its big push to REST.

The starting point: textbook routing.py

from routes import Mapper

def make_map():

    ...

    # Releases - Collection
    mapper.connect('releases', '/releases',
              controller='release', action='index', conditions=dict(method='GET'))
    mapper.connect('create_release', '/releases',
              controller='release', action='create', conditions=dict(method='POST'))
    mapper.connect('new_release', '/releases/new',
              controller='release', action='new', conditions=dict(method='GET'))

    # Releases - Members
    mapper.connect('release', '/releases/{id}',
              controller='release', action='show',
              requirements=dict(id='d+'), conditions=dict(method='GET'))
    mapper.connect('update_release', '/releases/{id}',
              controller='release', action='update',
              requirements=dict(id='d+'), conditions=dict(method='PUT'))
    mapper.connect('delete_release', '/releases/{id}',
              controller='release', action='delete',
              requirements=dict(id='d+'), conditions=dict(method='DELETE'))
    mapper.connect('edit_release', '/releases/{id}/edit',
              controller='release', action='edit',
              requirements=dict(id='d+'), conditions=dict(method='GET'))
    mapper.connect('release_notes', '/releases/{id}/notes',
              controller='release', action='release_notes',
              requirements=dict(id='d+'), conditions=dict(method='GET'))

Full of duplication, verbose, even ugly. Just because it sits in a config directory, does it have to look that bad?

Refactoring 1: Introducing submapper()

New in Routes 1.11 (so really quite new), submapper()provides the means to pull out parameters previously shared across multiple connect() calls.

One particularly nice feature is that the SubMapper objects returned by submapper() support the Python managed object protocol so if you’re on Python 2.5 or above you can use them with the with syntax like this:

from release_tool.lib.mapper import Mapper

def make_map():

    ...

    # Releases - Collection
    with mapper.submapper(
                    controller='release',
                    path_prefix='/releases') as c:
        c.connect('releases', '', action='index',
                conditions=dict(method='GET'))
        c.connect('create_release', '', action='create',
                conditions=dict(method='POST'))
        c.connect('new_release', '/new', action='new',
                conditions=dict(method='GET'))

    # Releases - Members
    with mapper.submapper(
                    controller='release',
                    path_prefix='/releases/{id}',
                    requirements=dict(id='d+')) as m:
        m.connect('release', '', action='show',
                conditions=dict(method='GET'))
        m.connect('update_release', '', action='update',
                conditions=dict(method='PUT'))
        m.connect('delete_release', '', action='delete',
                conditions=dict(method='DELETE'))
        m.connect('edit_release', '/edit', action='edit',
                conditions=dict(method='GET'))
        m.connect('release_notes', '/notes', action='release_notes',
                conditions=dict(method='GET'))

That’s a definite improvement (and credit where credit is due – submappers are great innovation, though a small bug requires our local extensions to be imported here), but notice that there’s still some duplication between the two submappers blocks, the first one corresponding to the collection resource, the second to that same collection’s members. Wouldn’t it be cool if we could nest them?

Refactoring 2: Submapper nesting

On the surface, the Routes 1.11 API doesn’t appear to support submapper nesting, but the SubMapper objects do indeed nest. Adding a submapper() method to SubMapper is a trivial change, and it take only minor internal tweaks for deeper nestings to function correctly.

    # Releases
    with mapper.submapper(
                    controller='release',
                    path_prefix='/releases') as c:
        # Collection
        c.connect('releases', '', action='index',
                conditions=dict(method='GET'))
        c.connect('create_release', '', action='create',
                conditions=dict(method='POST'))
        c.connect('new_release', '/new', action='new',
                conditions=dict(method='GET'))
        # Members
        with c.submapper(
                    path_prefix='/{id}',
                    requirements=dict(id='d+')) as m:
            m.connect('release', '', action='show',
                    conditions=dict(method='GET'))
            m.connect('update_release', '', action='update',
                    conditions=dict(method='PUT'))
            m.connect('delete_release', '', action='delete',
                    conditions=dict(method='DELETE'))
            m.connect('edit_release', '/edit', action='edit',
                    conditions=dict(method='GET'))
            m.connect('release_notes', '/notes', action='release_notes',
                    conditions=dict(method='GET'))

We seem to be on to something here, but what about the repetition (in one guise or another) of the resource name “release”? And what’s with those strange empty string ('') parameters?

Refactoring 3: Links and actions

To fix the repetition issue it’s clear that we need helpers that will generate those connect() calls for us more intelligently. But before we do that, let’s recognise that there are really two kinds of routes being generated here:

  1. Links to singleton subresources that support only one HTTP method, the most ubiquitous examples being the new and edit representations for which GET is the applicable HTTP method, but special subresources that are the targets for POST actions are common too;
  2. Actions that correspond directly to HTTP methods: index and create for GET and POST[1] on collection resources, show, update, delete for GET, PUT, DELETE on member resources.

It’s that second type that give rise to those strange empty strings as there is no further navigation to be done relative to the target resource.

So let’s add link() and action() helpers:

    # Releases
    with mapper.submapper(
                    controller='release',
                    path_prefix='/releases') as c:
        # Collection
        c.action(action='index', name='releases')
        c.action(action='create', method='POST')
        c.link('new')
        # Members
        with c.submapper(
                    path_prefix='/{id}',
                    requirements=dict(id='d+')) as m:
            m.action(action='show', name='release')
            m.action(action='update', method='PUT')
            m.action(action='delete', method='DELETE')
            m.link(rel='edit')
            m.link(rel='notes', name='release_notes')

Things are still moving in the right direction, but given that most of those links and actions follow a very well-worn convention, how about specific helpers for those?

Refactoring 4: More helpers

I’ll be honest – this is just an intermediate stage. You can see where we’re headed now though:

    # Releases
    with mapper.submapper(
                    collection_name='releases',
                    controller='release',
                    path_prefix='/releases') as c:
        # Collection
        c.index()
        c.create()
        c.new()
        # Members
        with c.submapper(
                    path_prefix='/{id}',
                    requirements=dict(id='d+')) as m:
            m.show()
            m.update()
            m.delete()
            m.edit()
            m.link(rel='notes', name='release_notes')

Refactoring 5: Parameter-driven generation

We’ll be using those same names a lot across our resources, so let’s just identify the ones we want in a list:

    # Releases
    with mapper.submapper(
                    collection_name='releases',
                    controller='release',
                    path_prefix='/releases',
                    actions=['index', 'create', 'new']) as c:
        # Members
        with c.submapper(
                    path_prefix='/{id}',
                    requirements=dict(id='d+'),
                    actions=['show', 'update', 'delete', 'edit']) as m:
            m.link(rel='notes', name='release_notes')

But even this pattern will be repeated across resources, so let’s define a collection() helper that does it all:

Refactoring 6: Using the collection() helper

    with mapper.collection(
                    'releases',
                    'release',
                    member_options={
                        'requirements': {'id': 'd+'}}) as c:
        c.member.link(rel='notes', name='release_notes')

And we’re done.

Let’s compare this to the equivalent resource() call:

    # Releases
    mapper.resource(
                'release',
                'releases',
                controller='release',
                # no requirements!
                member = {'notes':'GET'})

Superficially similar, but I’m unable to specify requirements on member paths and I don’t get to name my custom route either – the generated name “notes_release” just isn’t right! The bottom line is that resource() is approaching a dead end; its only way forward is to make its parameters ever more complex.

So… how about we put resource() on notice and bring in something more flexible and – dare I say – Pythonic? And wouldn’t it look just as nice in Ruby too? Answers on a postcard please…

8 Comments

  1. Well done!

    I’ve decided to go ahead and pull down your changes migrate my routes over to collection(). It cleans up my routing.py a good bit.

    Comment by Chris — December 29, 2009 @ 8:49 am

  2. Thanks, nice! And I’m delighted to report that my changes will make it into dev Routes very soon.

    Comment by Mike — December 29, 2009 @ 9:25 am

  3. Question Mike,
    Say I’ve got an admin section and if I use
    map.collection(‘admin/events’, ‘event’, controller=’admin/event’)

    It will create the index url name as ‘admin/events’
    but the others it will omit the ‘admin’ bit.
    admin/events -> ‘admin/events’ GET
    create_event -> ‘admin/events’ POST
    new_event -> ‘admin/events/new’ GET

    I’m not sure what the expected behavior ought to be. Should all route names include the ‘admin/’
    ‘admin/create_event’, ‘admin/new_event’, etc ? or just the index/list route?

    Comment by Chris — January 16, 2010 @ 7:16 pm

  4. Hi Chris,

    I’m not sure there’s a good answer to this – and I can’t help thinking that Rails compatibility (as opposed to anything deeper) was the key driver for the naming conventions of resource().

    My reasoning for naming all routes generated by collection():
    1) The names are after all route names, not resource names (and for a given path template the name for GET can be treated as the resource name, as described_routes does). Without names, routes are hard to reference (for whatever reason you might want to reference them).
    2) By not naming them you are forced to build into your views the assumption that your delete action (for example) is never going to move to admin/events/{id}/delete. Indirecting through the delete_ name give you than tiny but more flexibility.

    More weakly:
    3) What harm does it do?
    4) All else being even, isn’t explicit better than implicit?

    Mike

    Comment by Mike — January 16, 2010 @ 8:05 pm

  5. I was with you until around step 4. Then you really deviated into the fantastic world of methods so loaded with assumptions and magic that your final result can in no way be considered Pythonic.

    This definitely can be considered par for Rails, if that is your motivation, in that it is so narrow, so specific, so presumptive and so magical that unless one programs under the very fragile and particular conditions for which this solution is best suited, it’s going to be break. And because it’s so magical, unpacking its behavior and attempting to understand the process deeply is going to be an exercise in both spelunking and mind-reading.

    Python is definitely about being explicit.

    Comment by Jesse Dhillon — August 4, 2010 @ 9:05 pm

  6. Hi Jesse,

    I have some sympathy with that view. I thought it important though that the new API gave the power to avoid generating more routes than was required, and without too much code. But see it as an intermediate step to #6, which does the job of the old API but with a lot more power there at your disposal if you need it.

    With hindsight my bigger regret is that I didn’t name methods named after the HTTP methods GET, PUT, POST, DELETE etc. But that’s another story…

    Comment by Mike — August 4, 2010 @ 9:23 pm

  7. I should have proofread my post as I sound more irate than I actually am. What you did in steps 1-3 was very helpful. I just don’t think the style demonstrated here is wholly in keeping with Python’s guiding principles.

    Comment by Jesse Dhillon — August 4, 2010 @ 11:55 pm

  8. No worries, thanks for the feedback. Perhaps I exposed too much implementation detail here.

    Comment by Mike — August 5, 2010 @ 3:44 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress