Report from QCon New York 2012

I'm back in Sydney after a week in New York attending QCon, the conference for developers organised by InfoQ. I was invited to speak on the topic of "loosely-coupled identity management", since the eBook co-authored by me was published by InfoQ recently.

It's always a nice experience attending tech conferences. There's an exciting buzz in the air with so many "new, improved" ways of doing things on display. Some are marketing hype by vendors, others genuine technology advancements. QCon is not a vendor-focused conference though. Most of the technologies being discussed were Open Source.

One of the keynote speakers was Cameron Purdy, who is currently a senior executive at Oracle. I remember when Cameron was CEO of his own company Tangosol, and their product Coherence was one of the leading persistence technologies for Java. I used to enjoy the sparring matches between Cameron and a bunch of other people (usually JBoss employees) on the now all-but-defunct site TheServerSide.com (I mean the site is still there, but the party has since moved on to other places, notably StackOverflow and InfoQ). "Peace" was always his sign-off line. Looks like he's moved up quite a bit in life now, with Oracle's purchase of Tangosol.

Cameron's keynote had one major takeaway for me. I agree with his assessment that we as an industry are once again in the midst of a once-in-fifteen-years technology paradigm shift, and this one involves three elements:

1. The Cloud

2. HTML 5

3. Mobile devices

I'm going to take his advice very seriously and focus my attention on these three areas. I strongly feel the combination is going to be formidable.

My old friend and colleague Jim Webber was there. Jim is now Chief Scientist at Neo Technology, the maker of the Open Source Neo4j graph database. Graph databases are quite possibly a much more powerful way of modelling relations between entities than relational databases are, and the performance seems in no way compromised. On the contrary, if Jim is to be believed, the performance of a graph database can be orders of magnitude better than that of a relational database when the paradigm fits. (To be fair to Jim, he doesn't sell graph databases as the solution to every problem, but there does seem to be a large and growing set of applications that can benefit from them.) I confess I haven't got a firm handle on exactly how graph databases work and exactly what use cases can benefit from them, but they seem to work very well for data points that have a rich set of relationships with one another. According to Neo Technology's site, graph databases can find answers to questions like “what music do my friends like that I don’t yet own,” or “if this power supply goes down, what web services are affected?”

I missed Jim's tutorial on Neo4j because some competing (and more basic!) tutorials caught my attention, but I did attend his other tutorial on REST, about which I blogged earlier.

I attended a number of other talks and tutorials. The most memorable for me were the ones by Kaazing on HTML5 WebSockets. Richard Clark's half-day tutorial was quite good, but since the contents were compressed into 3 hours from a full-day workshop, we couldn't do justice to all the examples. But WebSocket technology is definitely something I need to play with and understand.

Subbu Allamaraju's talk was very interesting too. I was intrigued when he started off by saying the client side of REST had been neglected so far, but his thinking proved to be very different from mine. I've been thinking about a client-side resource model applied to a peer-to-peer messaging protocol (I know it sounds silly to talk about clients in a P2P world, but what I mean by "client" is "initiating peer"), but Subbu's idea is about a structured way to manage application state held on the client across "service invocations", a sort of client-side orchestration capability. The capability seems quite well advanced judging from his demo. He calls it ql.io, and it's well worth a dekko.

I met some of the REST greats at the conference, notably Mike Amundsen and Leonard Richardson. I didn't realise until later that Leonard was one of the authors of my favourite REST book, "RESTful Web Services". Leonard has also recently published a Science Fiction novel called "Constellation Games", which makes him a multi-dimensional nerd.

My own talk went off OK. (The slides are here.)

 "Faster, Cheaper Identity Management Through Loose Coupling - The LIMA Approach"

I had only 16 attendees, which was a bit disappointing, but the organisers told me it's always hard to get interest in the security track because developers are so much more interested in the mainstream technology tracks. However, I did receive good feedback about my talk. They have a system of gauging audience response by letting attendees place different coloured cards in a bowl after the talk (green for 'good', yellow for 'OK', red for 'not good'). I got 12 green and 2 yellow cards (no reds), which I take as a pretty good endorsement :-).

It's a heck of a journey from Sydney to New York (13 hours to LA, a 3 hour layover, and then a further 4.5 hours to NY) and back again, but I enjoyed myself. Qantas economy is pretty comfortable and I didn't feel the strain much.

IANA and the Last Layer of REST's "Uniform Interface"

[Before you read any further, go and read Martin Fowler's simple description of Leonard Richardson's REST Maturity Model. What I'm writing about here will make a lot more sense after that. Trust me.]

One of the things I've liked about REST as compared to SOAP (other than the fact that SOAP-based Web Services introduce fresh dependencies in the form of schema and WSDL files, negating the dependency-reducing spirit of SOA) is that REST standardises vocabulary, allowing designers to construct solutions out of standard terminological "components" rather than having to invent their own.

For example, instead of the arbitrarily-named and "open set" of SOAP verbs CreateQuote, SubmitQuote, SearchOrders, AmendOrder, CancelOrder, AddItem, RemoveItem, etc., one merely creates resources like Quote, Order and Item (and others for convenience) and applies a "closed set" of standard verbs to them (GET, PUT, POST and DELETE).

This much has been clear, and I like this. But I confess I have never fully understood how exactly REST manages to shift the coordination model away from centralised orchestration (à la WS-BPEL) to one of decentralised choreography (which is what the HatEoAS principle is all about).

Yes, the server provides a context-dependent, constrained set of links with each response, and the client is forced to follow one of these links, and is therefore shepherded in certain well-defined directions so that a business process completes in a well-defined way. [In the past, I've even referred to HatEoAS as "Hypermedia as the Envelope (not the Engine) of Application State" in recognition of its function as a set of constraints rather than an active driver.] This is choreography all right, where it takes two to tango, rather than orchestration, where services march to a single drum (pardon the discordant musical references). What I didn't understand was how these links could be made intelligible to an automated client that wasn't primed to understand them a priori.

A link looks like this:

<link rel="logical-action" uri="/some-resource"/>

I didn't understand how the descriptive "logical-action" in the link relation above was made meaningful to the client. After all, URIs are meant to be opaque in the sense that clients are not expected to understand their internal patterns and "hack" them to make future requests. Clients need to be guided purely by link relations (the "rel" attribute).

This finally became clear to me yesterday when I attended Jim Webber's excellent full-day tutorial on REST at QCon New York. [Attendees at Jim's talks can be assured of three things: (1) an unending spate of wry Brit jokes, here mainly directed at the Yanks ("tax dodgers who drive on the wrong side of the road"), (2) lots of Doctor Who references, and (3) some absolutely top-class technical insight and instruction.]

Much as I enjoyed the Brit humour (even as some of it came at the expense of Australians ("convicts who speak degenerate English")) and the Doctor Who references, I think the major value for me from this talk was the clarification that link relations are standardised too! I learnt that IANA has published a standard list of link relations that servers can use to describe the "next steps" that a client can perform, and since these are by definition "well-understood" descriptions, clients do not need prior negotiation with the server to learn what they mean!

The last piece of the jigsaw puzzle has now fallen into place in my head.

Some of the most important IANA-defined links, in my opinion, are:

• "first", "last", "next" and "prev" (alias "previous"): useful for chunked navigation of lists
• "self", "up", "start", "index", "contents", "glossary", "appendix": useful for orientation, repositioning and meta-information about the content itself
• "item" and "collection": useful to navigate master-detail relationships in both directions
• "alternate", "duplicate", "canonical", "current": helpful to refine the ability to link to a resource more accurately
• "edit" and "edit-media": suggest how to change content
• "help": needs no explanation
• "search", "tag": useful for research and taxonomy
• "working-copy", "working-copy-of", "version-history", "successor-version", "predecessor-version": helpful when editing documents collaboratively
• "payment": extremely useful in e-commerce applications by providing a standardised label to nudge customers towards the checkout

[If an application requires link relations that are more specialised, then they need to be namespaced, e.g., "mycorp:specialised-function", with the term before the colon being the namespace. The client will have to understand terms in the specialised namespace of course, but that's only reasonable.]

I prefer the more verbose form of the link tag, which looks like this:

<link rel="mycorp:apply" uri="/applications" method="POST" 

         type="application/vnd.mycorp-application-v1_1+xml"/>

Now the client also knows what HTTP method to use, and what schema the submitted document should conform to.

I can now see that REST has done a lot to standardise the development of applications in a SOA-compliant way. My last major query about how REST communicates with clients to bring about choreographed processes has now been answered.

This is great stuff. Thanks once again to Jim Webber for a very instructive tutorial! I guess we have a more powerful weapon than Torchwood now, Jim :-).