Sunday, July 29, 2012

OAuth2 - Whom to Believe?


In the field of Identity Management, a new dawn has begun, if one is to believe authorities such as Eve Maler of Forrester. The old, complex era of XML-based standards (SAML, SPML, XACML) is being replaced as we speak by a more lightweight, web-scale set of standards which are loosely called "the OAuth2 family".

OAuth2 is the one that underpins the others. Eve Maler refers to it as "a means of enabling constrained delegated authorisation of access." In other words, if you need to allow a photo printing service to print some of the photos that you have hosted on Picasa, then you don't have to give the service your Picasa username and password. The OAuth2 protocol allows you to give the service an "access token" for a limited period of time that allows them to access your photos, without having to reveal your security credentials. It takes an interesting 3-way handshake to accomplish this, but once this pattern is established, it can be used to protect any resource, most importantly APIs or service invocations.

The other members of the "OAuth2 family" use this feature so that they no longer need to concern themselves with the authorisation part of security, since OAuth2 will ensure that only a client with the right authorisation can invoke a service. These other APIs are "OpenID Connect" for Single Sign-On and SCIM for user provisioning. [Eve Maler talks up UMA (User Managed Access) as the third piece of the puzzle instead of SCIM, but I view UMA as just OAuth2 at industrial scale. Authentication, Authorisation and Provisioning are the three major aspects of IAM (audit is a fourth), and these are rightfully represented by OpenID Connect, OAuth2/UMA and SCIM, respectively.]

However, just when it looks like things are finally coming together in a simple and sane way, the party has been rudely interrupted.

Just 3 days ago, Eran Hammer, one of the leading figures in the OAuth2 standardisation effort, resigned from the working group and withdrew his name from the effort. The reasons he provides, at least initially, are dismaying and lead one to believe that OAuth2 is doomed as a standard and that we are going to see some catastrophic security breaches very soon.

His reasons for this seem plausible.

  • When the standardisation effort behind OAuth moved from the WHAT-WG to the venerable IETF (Internet Engineering Task Force), the lightweight-and-agile web developer crowd was replaced almost overnight by the  enterprise-y crowd. They squashed the agility out of the standard and replaced it with the dead weight of Enterprise thinking. The OAuth2 standard is the camel that the committee came up with when it set out to design a horse.
  • The specification fails to deliver on its two main goals of security and interoperability. As evidence, he says it has been renamed from a protocol to a framework, and the spec explicitly warns that implementations are unlikely to be interoperable.
  • The main point that the "enterprise crowd" fought for and won was flexibility and extensibility. In Eran Hammer's view, that's a bad thing, because "it is this extensibility and required flexibility that destroyed the protocol. With very little effort, pretty much anything can be called OAuth 2.0 compliant."
Fortunately, I read through the comments following his blog post, especially this very different viewpoint expressed by Leif Johansson.

I now have a more nuanced picture of where OAuth2 stands, and it isn't a picture of doom and despair. I've been in this industry for over 25 years, and I've seen this sort of thing before. Eran Hammer is young and idealistic, and design-by-committee always frustrates the young and idealistic.

The IT industry knows three types of innovators:
  1. The young, smart and agile, who come up with a new way of doing something, which then suffers growing pains because its inventors didn't think of all the possible situations in which it would be used. The NoSQL movement exhibits several examples of this.
  2. The older, smarter and experienced, who have learnt from years of mistakes (their own and those of other people), and who now know how things are to be done and will brook no questions. They're insufferable and nobody can work with them. (The much-respected and avoided Marshall T Rose comes to mind.)
  3. The committees of people with vested interests, who design the ugly camel that no one likes.
I think the most successful products come from the first group. The most successful standards come from the third. (Probably the second group can write good books and tell war stories.)

What Eran Hammer describes about the IETF and what they have done to OAuth is unsurprising and hardly dismaying. He's describing the sausage machine at work, and nobody really wants to see this in operation.

It's important that he's qualifying his rant with these two statements:

  • "OAuth 2.0 [in the hands] of a developer with deep understanding of web security will likely result [in] a secure implementation"
  • "It is true that [OAuth] 2.0 can be profiled into a simple, straightforward protocol. But every single one of these profiles is slightly different."

What this tells me is that OAuth2 is entering the second phase of its standardisation, when interoperability profiles will need to be defined. The standard isn't inherently broken. We can and do have secure implementations today. We just need to make them interoperable.

As an industry, we know how to do this. We've done it before with Web Services. That was called the WS-I Basic Profile, where "I" stood for Interoperability. Indeed, when Eran began his rant, he called the OAuth2 protocol "a bad protocol. WS-* bad". In other words, not that bad.

In fact, compared to the WS-I Basic Profile, OAuth2 is more of an orthogonal concern. The only aspect of it that an API developer will see is the "bearer token" header. So once the OAuth2 infrastructure is set up, it'll be an unobtrusive part of the IAM fabric. I know people who shudder at the term "profiles", but I don't believe profiles in OAuth2 are going to be visible in the sense of WS-I Basic Profile. So no course corrections required, folks. It's still full steam ahead with OAuth2. We'll give the angry young man some time and space to settle down.

[I have a rant of my own, a different one altogether, about the whole Identity-Management-as-Technology mindset, and will shortly be writing about the neglected stepchild of IAM - The Data Model. Watch this space.]

Monday, July 09, 2012

A RESTful Protocol to Add/Modify/Delete Elements of a Collection


I've been reading the SCIM specification recently.

[What does SCIM stand for? There's a moral-of-the-story here, by the way. SCIM originally stood for Simple Cloud Identity Management. Now it stands for System for Cross-domain Identity Management. The moral of the story is, if you're swayed by the buzzword of the day, you will be forced into embarrassing retreat at some point. Cross-domain Identity Management was what it was always about. What a shame someone felt the need to jump onto the cloud bandwagon.]

Anyway, coming back to the topic of this post, I realised that one of the aspects dealt with by the SCIM specification was a much more general problem that would benefit from a standard approach.

I was struck by the similarity of 3 types of operations in the SCIM protocol, though not impressed by any of them.

One set of operations deals with adding, modifying and deleting multi-valued attributes of a user, e.g., the set of addresses or email IDs.

A second set deals with adding, modifying and deleting users as members of a group.

SCIM uses a RESTful API, so for these sorts of partial updates to a larger entity, it uses PATCH. A single PATCH command can be used to simultaneously add, modify and delete attributes of an entity, or to add, modify and delete members of a group. [PATCH when used to update one or more single-valued attributes is fairly straightforward, so I'm not talking about that here.] I suspect that PATCH is so new that best practice around its use has yet to emerge, which accounts for some of the ugliness I'm about to describe.

A third set of operations is called 'bulk operations', and deals with a collection of operations on multiple entities, which could include add, modify and delete operations. This is physically formatted as a POST with nested POST, PATCH and DELETE operations as attributes within the document.

To my mind, these three situations are conceptually the same (a set of fine-grained operations grouped into a single composite operation), therefore the same mechanism should ideally be used for all of them.

As it turns out, SCIM uses many different mechanisms, all ugly.


(In the spec document linked above, scroll up and down a bit to search for these examples)

Some of the ugliness comes from passing 'operation:"delete"' as an attribute of the document. After arguing for years that a GET request with URL parameter "?operation=delete" is not RESTful, we can't now barefacedly offer a "RESTful" API that does the same thing. True, this is not a GET, so it's probably not as awful, but this is a really bad "smell" that suggests poor design at some level.

The syntax to replace a user's primary email address with another one and simultaneously demote the previous primary email address to the status of just another email address is more than just ugly, it's horrific. Do have a look for yourself, if you can keep from gagging.


(In the spec document linked above, scroll up and down a bit to search for this example)

Deleting all members from a group is even uglier. There is nothing in the message to suggest that a deletion is taking place! There's a "meta" attribute that has a value equal to the set of attribute names whose (multiple) values are being deleted. If that confuses you, you now know what I mean.


The main operation is a POST to an arbitrary URI which could be called "/bulk" or something similar.
There are nested operations within it that reflect operations on various entities.

As you can see, these are 3 entirely different mechanisms for essentially the same desired behaviour. Can't we use the REST philosophy of polymorphic verbs to make this interface more uniform?

I would like to propose a simpler scheme that uses explicit identifiers for every sub-entity that may need to be independently manipulated (updated or deleted). This is similar to having resource identifiers, but at finer levels of granularity.

All three sets of operations should have a common structure like this:

PATCH /main-entities/29d70c13-8651-41ce-97f8-c82ccdfcd6d6
(or POST /bulk-operations)
Host: example.com
Accept: application/json
Authorization: Bearer h480djs93hd8
Content-Length: ...

{
  operations:
  [
    {
      operation: "POST",
      URI: " /sub-entities",
      sub-entity:
      {
      ...sub-entity attributes
      }
    },
    {
      operation: "PUT",
      URI: "/sub-entities/dced46ca-b96d-44dd-b494-41e5de9405b6",
      sub-entity:
      {
      ...sub-entity attributes
      }
    },
    {
      operation: "DELETE",
      URI: "/sub-entities/12feb1db-8031-4a23-8810-bccfffb13f56"
    }
  ]
}

i.e., the nested operations should be RESTful operations in their own right, with standard HTTP verbs and standard URIs using the enveloping entity's URI as their base URI. How do we know what the URIs of the sub-entities are? The server should generate one for every array element that is created (multi-valued attributes of an entity), and return those URIs with the "201 Created" response, as well as in response to any GET on the main entity. That will then provide a handle for the client to specify fine-grained operations on the sub-entities.

I also don't like the SCIM convention of returning a "200 OK" even for a bulk request where different sub-operations may have had different outcomes. This is as bad as HTTP tunnelling in SOAP where the HTTP response could be a cheery "200 OK", but the message inside contains a SOAP fault.

There is a WebDAV status code for exactly this kind of situation, - "207 Multi-Status". That's what should be used.

HTTP 1.1/207 Multi-Status
Content-type: application/json

{
  operation-results:
  [
    {
      operation: "POST",
      URI: "/sub-entities",
      status: "200 OK",
      Location: "/sub-entities/c44fcc7b-8881-4f0f-b5fb-b2344615cc34",
      request:
      {
      ...sub-entity attributes repeated for verification
      }
    },
    {
      operation: "PUT",
      URI: "/sub-entities/dced46ca-b96d-44dd-b494-41e5de9405b6",
      status: "409 Conflict"
    },
    {
      operation: "DELETE",
      URI: "/sub-entities/12feb1db-8031-4a23-8810-bccfffb13f56",
      status: "404 Not found"
    }
  ]
}

This is a clean syntax, easily understood, and uniformly applicable when dealing with attributes or sub-entities of an entity, or with bulk operations. All it requires is for resources and resource collections to be identifiable at finer levels of granularity as well, i.e., attributes that are array elements.

I don't believe that's too much of an ask. UUIDs are cheap and plentiful, and the alternative is an ugly mess.

Sunday, July 08, 2012

Architectural Lessons in Identity from the Higgs Boson


As I read the details of last week's virtual confirmation of the existence of the Higgs boson, I smacked myself on the head with a "D'oh!"

As an architect, I've been preaching against tight coupling for years, and I utterly failed to realise that there could be examples of tight coupling in the way we think about the real world.

What the discovery of the Higgs boson does is confirm what is called the Standard Model of Particle Physics, and one of the fundamental, though unintuitive, aspects of that model is that mass is not an inherent property of a particle. A particle may acquire mass through interaction with the Higgs field, but it's not necessary. That's why it's possible for particles like photons to have no mass.

It's ironical that in the months just prior to this discovery in the world of Physics, I made a modest discovery of my own that nevertheless shook the foundations of my conceptual world. In the realm of identity management, I realised that any given entity has no inherent properties (attributes) at all! All attributes that an entity may be deemed to have are only by association (just as a particle acquires mass independently of its existence), and therefore the only necessary aspect of an entity is a unique and meaning-free identifier to set it apart from other similar entities. Attributes can then be assigned and de-assigned to it at will through the mechanism of the identifier, and gradually a more sophisticated model can be built up. In creating such sophisticated models, it's important to remember that nothing is inherent to an entity, - not first name/last name, not primary email address, not social security number, - nothing! Just a unique identifier that is internal to the domain and kept invisible to the world outside the domain.

I've found this minimalist model of identity to be amazingly powerful and flexible. In addition to having arbitrary groups of attributes associated with an entity's identifier to form its properties, multiple external identifiers can also be independently associated with the (internal) identifier. The usual surrogate for identity, the username, is then just one such external identifier. Much of the conceptual confusion in identity management comes from mistaking external identifiers like username for the entity itself, leading to extremely clumsy and costly implementations. By creating an explicit internal identifier and treating any external identifier as an attribute associated with the internal identifier, such confusion and costly error can be avoided.

Cross-domain entity references can also be managed very cleanly by mapping both domains' internal identifiers to a shared external identifier, so that neither domain is coupled to the other domain's internal identifier for an entity. That's goodness!

Federated identity, such as when using one's FaceBook ID to enter a website, is nothing but treating the FaceBook ID as an external identifier and associating it with the website's internal identifier to establish the entity's identity. So multiple domains can independently maintain models about the same entity, and for most of the time, they can behave as though they are dealing with different entities. When it becomes necessary for the two domains to recognise these entities as the same, they then create a shared identifier that is external to both of them, and they each map this external identifier to their respective internal identifier.

This is a model of federation that does not make the assumption that there is a centre of the universe. No domain is the centre of the universe. Every domain is independent, and when they need to share their model of an entity, they loosely associate their entity references through a shared external identifier. There is now traceability across domains, but no implied control. I've found in practice that this is the only model that works, - technically, logistically and politically.

It's amazing that I should receive indirect confirmation of my identity model from as unlikely a source as Particle Physics, but I guess that's just a reflection of the fundamental nature of all things.

We have nothing inherent to ourselves but an identity. That's a pretty humbling thought that verges on the religious.