50 Data Principles For Loosely-Coupled Identity Management

It's been a while since our eBook on Loosely-Coupled IAM (Identity and Access Management) came out. In it, my co-author Umesh Rajbhandari and I had described a radically simpler and more elegant architecture for a corporate identity management system, an architecture we called LIMA (Lightweight/Low-cost/Loosely-coupled Identity Management Architecture).

Looking at developments since then, it looks like that book isn't going to be my last word on the subject.

IAM has quickly moved from within the confines of a corporate firewall to encompass players over the web. New technology standards have emerged that are in general more lightweight and scalable than anything the corporation has seen before. The "cloud" has infected IAM like everything else, and it appears that IAM in the age of the cloud is a completely different beast.

And yet, some things have remained the same.

I saw this for myself when reviewing the SCIM specification. This is a provisioning API that is meant to work across generic domains, not just "on the cloud". It's part of the OAuth 2.0 family of specifications, and OAuth 2.0 is an excellent, layerable protocol that can be applied as a cross-cutting concern to protect other APIs. SCIM too is OAuth 2.0-protected, but that's probably where the elegance ends.

The biggest problem with SCIM is its clumsy data model, which then impacts the intuitiveness and friendliness of its API. I critiqued SCIM on InfoQ, and in response to a "put up or shut up" challenge from some of the members of the SCIM working group, I began working on an Internet Draft to propose a new distributed computing protocol, no less. That's a separate piece of work that should see the light of day in a couple of months.

In the meantime, I began to work on IAM at another organisation, a telco this time. My experiences with IAM at a bank, an insurance company and then a telco, had by then given me a much better understanding of Identity as a concept, and I began to see that many pervasive ideas about Identity were either limiting or just plain wrong. Funnily enough, most of these poor ideas had more to do with the Identity data model than with technology. I also observed that practitioners tended to focus more on the "sexy" technology bits of IAM and less on the "boring" data bits, and that explained to me, very convincingly, why systems were so clumsy.

I then consciously began to set down some data-specific tips and recommendations that I saw being ignored or violated. The irony is that it doesn't cost much to follow these tips. All it costs is a change of mindset, but perhaps that's too high a price to pay for many! In dollar terms, the business benefits of IAM can be had for a song. Expensive technology is simply not required.

So that's the lesson I learnt once more, and the lesson I want to share. No matter what changes we think are occurring in technology, the fundamental concepts of Identity have not changed. The data model underlying Identity has not changed. Collectively, we have a very poor understanding of this data model and how we need to design our systems to work with this data model.

So here are 50 data principles for you, the architect of your organisation's Identity Management solution. I hope these will be useful.

The presentation on Slideshare:
http://slidesha.re/14uo3YY

The document hosted on mesfichiers.org:
http://atarj9.mesfichiers.org/en/

"What Are The Drawbacks Of REST?"

It seems the season for me to post comments in response to provocative topics on LinkedIn. 

A few days ago, Pradeep Bhat posed the question, "What Are The Drawbacks Of REST?" on the REST Architects LinkedIn Group. As before, I had an opinion on this too, which I reproduce below:

"I wouldn't say REST has "drawbacks" as such. It does what it says on the tin, and does that very well. But remember that the only implementation of the REST architecture uses the HTTP protocol. We can surely think of a future RESTian implementation that uses another transport protocol, and that is where some improvements could be made. 

1. HTTP is a synchronous, request/response protocol. This means the protocol does not inherently support server-initiated notifications (peer-to-peer), which are often required. That's why callbacks in RESTian applications require the use of application-level design patterns like Webhooks. Now that we have a bidirectional transport protocol in the form of WebSockets, perhaps the industry should be looking at layering a new application protocol on top of it that follows RESTian principles. 

2. The much-reviled WS-* suite of protocols has at least one very elegant feature. These are all end-to-end protocols layered on top of the core SOAP+WS-Addressing "messaging" capability. They resemble the TCP stack in that the basic protocol is IP, which only knows how to route packets. SOAP messages with WS-Addressing headers are analogous to IP packets. In the TCP world, end-to-end reliability is implemented through TCP over IP, and the SOAP world's analogy is WS-ReliableMessaging headers in SOAP messages. In the TCP stack, IPSec is the end-to-end security protocol (not TLS, which is point-to-point). The SOAP equivalent is WS-SecureConversation. Such Qualities of Service (QoS - reliability, security, transactions) can be specified by policy declaration (WS-PolicyFramework) and SOAP endpoints can apply them like an "aspect" to regular SOAP traffic. 

The REST world has nothing like this. Yes, an argument could be made that idempotence at the application level is a better form of reliability than automated timeouts and retries at the transport level. Similarly, we could argue that an application-level Try-Confirm/Cancel pattern is better than distributed transactions. But what remains is security. WS-SecureConversation with WS-Security is routable, unlike SSL/TLS, which is the only security mechanism in REST. With WS-Sec*, messages can also be partially encrypted, leaving some content in the clear to aid in content-based routing or switching. This is something REST does not have an elegant equivalent for. SSL is point-to-point, cannot be inspected by proxies and violates RESTian principles. It is just tolerated. 

The reason behind REST's inability to support such QoS in general is that all of these require *conversation state* to be maintained. Statefulness has known drawbacks (i.e., impacts to scalability and failure recovery), but with the advent of NoSQL datastores like Redis that claim constant-time, i.e., O(1), performance, it may be possible to delegate conversation state from memory to this datastore and thereby support shared sessions for multiple nodes for the purposes of QoS alone. I don't mean to use this for application-level session objects like shopping carts. If nodes can routinely use shared NoSQL datastores to maintain sessions, then the argument against statefulness weakens, and Qualities of Service can be more readily supported *as part of the protocol*. In RESTian terms, we can have a "uniform interface" for QoS.

3. While REST postulates a "limited" set of verbs, HTTP's verbs are too few! 

POST (add to a resource collection), PUT (replace in toto), PATCH (partially update), DELETE (remove from accessibility) and GET. These are actually not sufficient and they are frequently overloaded, resulting in ambiguity. 

I would postulate a more finely-defined set of verbs if defining a RESTian application protocol over a new peer-to-peer transport: 

INCLUDE (add to a resource collection and return a server-determined URI), PLACE (add to a resource collection with client-specified URI), REPLACE (in toto), FORCE (PLACE or REPLACE), AMEND (partial update, a container verb specifying one or more other verbs to specify operations on a resource subset), MERGE (populate parts of the resource with the supplied representation), RETIRE (a better word than DELETE) and SOLICIT (a GET replacement that is also a container verb, to tell the responding peer what to do to the initiator's own resource(s), because this is a peer-to-peer world now). Think of GET as a SOLICIT-POST to understand the peer-to-peer model. We also need a verb of last resort, a catch-all verb, APPLY, which caters to conditions not covered by any of the others. 

4. HTTP combines application-level and transport-level status codes (e.g., 304 Not Modified and 400 Bad Request vs 407 Proxy Authentication Required and 502 Bad Gateway). The next implementation of REST on another transport should design for a cleaner separation between the application protocol and the transport protocol. HTTP does double-duty and the results are often a trifle inelegant. 

So that's what I think could be done as an improvement to REST-over-HTTP. Apply the principles (which are very good) to a more capable peer-to-peer transport protocol, and design the combination more elegantly."

I'm in the process of writing an Internet Draft for a new application protocol that can be bound to any transport (Pub/Sub, Asynchronous Point-to-Point or Synchronous Request/Response). The protocol is part of a new distributed computing architecture that I call ROMA (Resource/Representation-Oriented Messaging Architecture) and covers not just the data model and message API but also higher levels (QoS, description and process coordination). It's been 5 years in the making and has reached 170 pages so far. It may take another couple of months to get to a reviewable state. Stay tuned.

"Can Anyone Explain SOA In Simple Terms?"

A few days ago, David Diamond posed a deceptively simple question on one of the LinkedIn Group sites (SOA SIG) - "Can Anyone Explain SOA In Simple Terms?"

The barrage of widely varying responses that followed was, in a way, an eloquent answer to that question!

I've had my own take on SOA for quite a while now, so this gave me the opportunity to validate my model against what other practitioners had to say. And I must say this: I'm more convinced than ever that the industry is horribly confused about SOA. There are those whose understanding of SOA is at a purely technology level (even some of those who profess to understand that SOA is not (just) about technology). And there are others who may understand SOA for all I know, but whose explanations tend to be couched in so much jargon that they're really hard to understand.

In hindsight, David Diamond could not have asked a more insightful question.

Well, this is my blog, so just as history is written by the victors, the one correct answer to David's question is to be found here :-).

Here's what I wrote (put together from more than one post that I made to that topic):

My initial one-paragraph answer: "SOA is the science of analysing and managing dependencies between systems. That means the elimination of unnecessary dependencies and the formalisation of legitimate dependencies into readily understood contracts. The more dependencies there are between systems, the less agile an organisation is (because of the number of related systems that have to change when one of them has to change), the higher its operating costs (because of all the unnecessary extra work to coupled systems) and the higher its operational risk (because of the number of things that could break when something changes). Dependencies exist at all of the BAIT layers - Business, Applications, Information (Data) and Technology. That's why a technology-only view of SOA does not solve an organisation's real problems. SOA should have been called DOT instead ("Dependency-Oriented Thinking")."

After a few days of reading other responses and feeling dissatisfied, I posted again:

"Many of the comments here emphasise reuse as part of the *definition* of SOA. Is reuse a core feature of SOA or just a side-benefit? If the latter, what are SOA's defining features (which is what the original question was about)? Also, while we use the word "services" a lot, how do we define the term?

Let me try and address these two points.

SOA is an organising principle for the enterprise, and the fundamental skill that an architect requires to apply this organising principle is the ability to see dependencies between systems, - to be able to eliminate the ones that shouldn't exist and formalise the legitimate ones into "contracts" maintained in a single place and covering all the dependencies between two systems. This approach greatly reduces the cost of change, improves the speed with which changes are made (agility) and reduces the risk of making changes, all because the number of dependencies (aspects of an interaction affected by a change) are now smaller, one can tell at a glance what they are, and there are no surprises because there are no dependencies outside what is documented by the contract. This is not limited to technology interactions. One can apply this thinking to the design of business processes just as naturally.

When we look through a dependency lens at an organisation, our tasks are quite distinct at its four layers (Business, Applications, Information (Data) and Technology).

At the Business layer, it is more of a BPR (Business Process Re-engineering) exercise, because we end up rationalising processes when we weed out unnecessary dependencies. When we finish, we have a traceability matrix linking the following:

Vision (Our idea of Utopia)

Mission (Our strategy to bring about that Utopia)

Functions (The main groups of activities we need to be doing as part of that strategy)

Processes (The detailed and related sequences of steps comprising each function)

Process Steps (The basic building blocks of these processes)

[At the business layer, we will come across some *potential* reuse when we look at the definition of some of the Process Steps (operations) we arrive at. Only further analysis at the Information layer will tell us if reuse is actually possible or these are independent operations.]

The Application layer is all about grouping "related" operations, and the dependency principle used is that of "cohesion and coupling". In other words, we need to determine which process steps belong together and which do not. This cannot be done independently but must involve the Information (data) layer as well. [That's why architectural frameworks like TOGAF combine the two into a single step (Phase C)].

The Information layer looks at data dependencies (shared models) and classifies data into two groups - "data on the outside" and "data on the inside". "Data on the inside" is the set of internal domain models for operations that other operations do not need to see. "Data on the outside" is what goes "over the wire" between operations.

When we apply the dependency principle of cohesion and coupling to the combined Application and Information layers, we have two ways of grouping operations together. Operations that share a domain model ("data on the inside") coalesce into Applications that are called Products. Operations that share an interface data model ("data on the outside") coalesce into Applications that are called Services. So this is where Services fit into SOA - as a bundle of related operations sharing an interface data model.

The Technology layer deals with "implementation". As others have pointed out as well, implementation need not have anything to do with SOAP, ESBs, etc. We need distinct components to host implementations of exposed operations (Service Containers), to mediate interactions (Brokers) and to coordinate operations (Process Coordinators). Other components merely support these (Rules engines, registries, monitoring tools).

This is SOA :-)."

I would have posted more, but I exceeded the word count for the site, so I had to post my thoughts about the Technology layer separately:

"I must add that when viewed through a dependency lens, the Technology layer often introduces artificial dependencies of its own. There is a reason why many people prefer REST to SOAP. It's because WSDL is a dependency hotspot. Think about it. If a WSDL file describes just one service, and that service comprises 5 operations, each with an input document and an output document, then the version of the WSDL file has a dependency on the version of 10 document schemas. If any one of them changes, the WSDL will have to change! That's why we have so much version churn in organisations.

In addition, because we don't build explicit interface data models with type hierarchies, our operation interfaces are too rigid and low-level, requiring a fresh *version* whenever a new *variant* is to be supported.

A second major dependency introduced by the technology layer is through the ESB, or more correctly, through incorrect use of the ESB. The dependency principle at the Technology layer is to use the right tool for the job and to use it the right way. If we use the ESB to host business logic, we are making it perform the role of a Service Container. If we use the ESB to orchestrate a process, we are making it perform the role of a Process Coordinator. Both of these mistakes create dependencies that reduce performance and increase the cost of change. 

The other ESB-related mistake is its deployment in a hub-and-spokes architecture. Then the ESB becomes both a performance bottleneck and a single point of failure - both symptoms of a needless topological dependency that was created at the Technology layer. IT organisations often ask for funds to buy an ESB because they want to "do SOA", then implement it in a topology that creates dependencies and thereby violates SOA principles. What an irony! 

So one of the reasons why SOA has acquired a bad name is that its practice often introduces dependencies at the Technology layer even as it tries to reduce dependencies at the Business, Application and Information layers. Worse, because organisations are often too technology-focused, they don't do enough of the dependency-reduction at these higher layers and their net effect is to introduce new, technology-related dependencies to an existing set of business processes and data structures. The net effect of SOA on such organisations is then entirely negative.

I'm in the process of writing a white paper on "Dependency-Oriented Thinking" based on my experiences with SOA in large organisations. Stay tuned :-)."

Well, this represents my current thinking about SOA in a nutshell (a fairly large nutshell, I'll grant). The coming white paper on Dependency-Oriented Thinking will elaborate on these points. The workshops on "Dead Simple SOA" that I've been conducting through my company (Eigner Pty Ltd) along with my colleague Rahul Singh, address these very topics.

JEM (JSON with Embedded Metadata) - A Simpler Alternative to JSON Schema?

I've long been a supporter of the JSON Schema initiative, and I was also happy to see developments like Orderly, which offered a simpler and less verbose format than JSON Schema. But Orderly introduces its own format, which necessitates conversion to JSON Schema before it can be used. Both approaches are unsatisfactory in their own way. One is too verbose and the other needs translation.

All of this made me wonder if we aren't approaching the problem the wrong way. JSON Schema is a conscious effort to replicate in the JSON world the descriptive capability that XML Schema brings to XML. But is this the best way to go about it?

I would like descriptive metadata about documents to be capable of being embedded inside the document itself, rather like annotations in Java programs. Indeed, this metadata should be capable of forming a "scaffold" around the data that then allows the data itself to be stripped out, leaving behind a template or schema for other data instances.

So I'm proposing something that I think is a whole lot simpler. It does require one fundamental naming convention to be followed, and that is this:

Any attribute name that begins with an underscore is metadata. Everything else is data.

Let's take this simple JSON document:

We can embed metadata about this document in two different ways. Click diagram to expand.

I'm calling the first style "Metadata Markup", where the data elements of the JSON document retain their primacy, and the metadata around them is secondary and serves to add more detail to these data elements. One can readily see that "_value" is now just one of the possible attributes of an element, and many more such attributes can therefore be added at will.

I call the second style "Metadata Description", where the primary elements are metadata, and any data elements (whether keys or values) are modelled as the values of metadata elements. Note that describing a document as an array (a nested array in the general case) rather than as a dictionary (or nested dictionary) of elements allows the default order of the elements to be retained. This is quite useful when this format is used to publish data for human consumption.

The first style, Metadata Markup, is more suitable for document instances, because a lot of detailed meta-information can accompany a document and can be hidden or stripped out at will. It is easy for a recipient to distinguish data from metadata because of the leading underscore naming convention. There is no need to pre-negotiate a dictionary of metadata elements. (Click to expand.)

The second style, Metadata Description, is more suitable for schemas, because in this format, all elements pertaining to instance data (both keys and values) are just values. If only the values representing keys are retained, we get a "scaffold" structure describing the document, and more metadata elements representing constraints can be added, turning it into a schema definition. (Click to expand.)

Obviously, this system will not work for everyone. I'm sure there are JSON documents out there that have underscores for regular data (HAL?), so adoption of this convention won't be feasible in such domains. But if a significant subset of the JSON-using crowd finds value in this approach, they're more than welcome to adopt it.

The Happy Confluence of IAM, SOA and Cloud

Someone pointed me to this Gartner blog post on IAM, and I was once again reminded why Gartner doesn't get it, (or when they do, they get it much after everyone else).

The Gartner analyst in his presentation makes a big deal of the fact that LDAP, being a hierarchical data structure, is incapable of modelling the various complex relationships between entities in an IAM system. This is one of the reasons he believes we need to "kill IAM in order to save it". But is this limitation in traditional IAM systems really new? I'm no fan of LDAP, and it has been known in IAM circles for at least 5 years that LDAP directories are suited for nothing other than the storage of authentication credentials (login names and passwords)! Everything else should go into a relational database, which is much better at modelling complex relationships. A meaning-free identifier links an LDAP entry with its corresponding record in the relational database. I describe this hybrid design in a fair amount of detail in my book "Identity Management on a Shoestring". And this wasn't even my original idea. It was one of the pieces of advice my team got from a consultant (Stan Levine) that my employer hired to review our IAM plans.

Seriously, where has Gartner been?

Another big point made by the Gartner analyst was that IAM should not be "apart from" the rest of an organisation's systems but become "a part of" them. Joining the dots with my cynical knowledge of where Gartner tends to go with this kind of argument, I can see them making the case for big vendors that do everything including IAM. The cash registers at SAP, Oracle and Salesforce.com must have started ringing already, since Gartner has given those vendors' product strategies their all-important blessing.

Um, no. If there's anything we've learnt in the last few years (especially from SOA thinking), it's the great benefits that are gained from loose coupling. IAM should neither be "apart from" (decoupled) nor "a part of" (tightly coupled) with respect to an organisation's other, business-related systems. IAM needs to be loosely-coupled with respect to them.

What does this mean in practical terms? It means IAM needs to be a cross-cutting concern that can be transparently layered onto business systems to enforce access policies, but without disrupting those systems with IAM-related logic.

That's really what the latest IAM technology, OAuth 2, brings to the table. But the Gartner analyst, while dwelling for quite a while on how great OAuth is, completely omits to define its true contribution.

Eve Maler of Forrester says it much better in her presentations. She defines OAuth as a way to delegate authorisation, and positions it as a way to protect APIs. Can you see the confluence of IAM, SOA and the Cloud in that simple characterisation?

Let's take those two aspects one by one and have a closer look.

OAuth as a way to delegate authorisation:

The traditional model of authorisation works like this. There is an entity that owns as well as physically controls access to a resource. When a client requests access to that resource, the owning entity does three things:

1. Authenticates the client (i.e., establishes that they are who they claim to be)

2. Checks the authorisation of the authenticated client to access the resource (i.e., acts as a Policy Decision Point)

3. Allows (or denies) the client access to the resource (i.e., acts as a Policy Enforcement Point)

What OAuth does is recognise that the Policy Decision Point and the Policy Enforcement Point may be two very different organisational entities, not just two systems within the same organisational entity. The PDP role is typically performed by the owner of the resource. The PEP role is performed by the custodian of the resource. The owner need not be the custodian.

Under the OAuth model, there is a three-way handshake between the owner of a resource, the custodian of the resource and a client. Three separate trust relationships are established between the three pairs of entities in this model, and authentication is obviously required in setting these up (owner-to-client, owner-to-custodian and client-to-custodian-through-owner). Once the owner's permission to access the resource for a certain window of time is recorded in the form of an access token that the client stores, the owner's presence is no longer required when such access takes place. The custodian is able to verify the token and allow access in accordance with the owner's wishes even in the owner's absence. This is delegated authorisation.

And since the resource doesn't even know it's being protected, this is loose coupling. IAM is neither "apart from" nor "a part of" the business system with OAuth.

OAuth as a way to protect APIs:

The delegated authorisation model can be used to protect resources that are not just "things" but also "actions". In other words, OAuth can be used to control who can invoke what logic, and do so in a delegated manner. In other words, owners of business logic can grant access to clients to invoke business logic, and custodians that host such business logic can validate the access tokens presented by clients and allow or deny access in accordance with the wishes of the owners.

Now why does this development in the IAM world bring it into confluence with the SOA and cloud worlds?

The SOA bit is easy to understand. We did mention that an API is a form of resource. If all business logic can be reduced to service operations exposed through endpoints, then these form an API. Endpoints can be protected by OAuth as we saw, so OAuth can be an effective security mechanism for SOA.

The cloud bit isn't hard to understand either. If business logic can be abstracted behind APIs, then does it matter where that logic sits? Bingo - cloud! The cloud also forces separation of owner and custodian roles, with the cloud platform performing the role of custodian, and the cloud customer performing the role of resource owner or API owner. With OAuth as the authorisation mechanism, the cloud model becomes viable from an access control perspective as well.

So that's really what OAuth signifies. It's not just a development in IAM. It has profound implications for SOA security and the viability of the cloud model.

Watch for Gartner to break this news to their clients in 3 to 5 years' time...

(Meanwhile, someone at Gartner or elsewhere ought to tell that analyst that "staid" is not spelled "stayed". This presentation has irritated me on so many levels - spiritually, ecumenically, grammatically, as Captain Jack Sparrow said.)

How to Implement An Atomic "Get And Set" Operation In REST

This question came up yesterday at work, and it's probably a common requirement.

You need to retrieve the value of a record (if it exists), or else create it with a default value. An example would be when you're mapping identifiers between an external domain and your own. If the external domain is passing in a reference to an existing entity in your domain, you need to look up the local identifier for that entity. If the entity doesn't yet exist in your domain, you need to create (i.e., auto-provision) it and insert a record in the mapping table associating the two identifiers. The two operations have to be atomic because you can't allow two processes to both check for the existence of the mapping record, find out it doesn't exist, then create two new entity instances. Only one of the processes should win the race.

(Let's ignore for a moment the possibility that you can rely on a uniqueness constraint in a relational database to prevent this situation from occurring. We're talking about a general pattern here.)

Normally, you would be tempted to create an atomic operation called "Get or Create". But if this is to be a RESTian service operation, there is no verb that combines the effects of GET and POST, nor would it be advisable to invent one, because it would in effect be a GET with side-effects - never a good idea.

One solution is as follows (and there could be others):

Step 1:

GET /records/{external-id}

If a record exists, you receive a "200 OK" status and the mapping record containing the internal ID.

Body:

{

  "external-id" :  ...

  "internal-id" :  ...

}

If the record does not exist, you get a "404 Not found" and a one-time URI in the "Location" header.

Location: /newrecords/84c5d65a-2198-42eb-8537-b16f58733791

(The server will also use the header "Cache-control: no-cache" to ensure that intermediate proxies do not cache this time-sensitive response but defer to the origin server on every request.)

Step 2 (Required only if you receive a "404 Not found"):

2a) Generate an internal ID.

2b) Create a new entity with this internal ID and also create a mapping record that associates this internal ID with the external ID passed in. This can be done with a single POST to the one-time URI.

POST /newrecords/84c5d65a-2198-42eb-8537-b16f58733791

Body:

{

  "external-id" :  ...

  "internal-id" :  ... (what you just generated)
  "other-entity-attributes" : ...

}

The implementation of the POST will create a new local entity instance as well as insert a new record in the mapping table - in one atomic operation (which is easy enough to ensure on the server side).

If you win the race, you receive a "201 Created" and the mapping record as a confirmation.

Body:

{

  "external-id" :  ...

  "internal-id" :  ... (what you generated)

}

If you lose the race, you receive a "409 Conflict" and the mapping record that was created by the previous (successful) process.

Body:

{

  "external-id" :  ...

  "internal-id" :  ... (what the winning process generated)

}

Either way, the local system now has an entity instance with a local (internal) identifier, and a mapping from the external domain's identifier to this one. Subsequent GETs will return this mapping along with a "200 OK". The operation is guaranteeably consistent, without having to rely on an atomic "Get or Create" verb.

One could quibble that a GET that fails to retrieve a representation of a resource does have a side-effect - the creation of a one-time URI with the value "84c5d65a-2198-42eb-8537-b16f58733791" being inserted somewhere. This is strictly true, but the operation is idempotent, which mitigates its impact. The next process to do an unsuccessful GET on the same value must receive the same one-time URI.

It's a bit of work on the server side, but it results in an elegant RESTian solution.

Learnings From My "Dead Simple SOA" Workshop on Saturday

As announced earlier, the workshop that we (my business partner Rahul Singh and I) organised on "Dead Simple SOA" took place yesterday (Saturday the 8th Dec), and here's what we learnt from it.

Fundamentally, the ideas we are trying to popularise are seen as useful and worth pursuing. That was a very welcome piece of validation. However, we do need to refine our message and target our audience better.

We got largely positive feedback from the two participants (the low registration rate is itself a symptom of poor messaging and targeting), but also some valuable feedback on how we could improve it.

1. There are two sets of messages in the workshop, and this dilutes its appeal. This is because the draft white paper on which the workshop is based ("Slicing the Gordian Knot of SOA Governance") itself deals with two separate (though related) themes.

One theme is "How to do SOA". This had a number of new and useful ideas that the participants commented favourably on.

The other theme is about "How to do SOA Governance and Management". While the ideas behind this were also appreciated, here's what seemed to be lacking:

a. This topic is of interest to a different group of people, - perhaps managers and enterprise architects, whereas "How to do SOA" is of interest to solution architects, designers and perhaps even developers.

b. There is likely to be unease over the fact that my method challenges the accepted industry view of SOA Governance as a technology-related practice, which creates dissonance and may make people reluctant to pay for a "heterodox" course. If I want my approach of governing the entire enterprise through dependency-oriented thinking to be adopted, I should rename it to something else rather than try and redefine the term "SOA Governance". That train has left the station.

I seem to have a choice here. In the candid words of one of the participants, I have to "choose whether I want to be a philosopher or to build a business". Because if I want to build a business, his suggestion was to quietly align with the industry model instead of trying to challenge it, and then make money by offering to train people in the "universally accepted way". [I do think I can carve out a niche by being different without aligning submissively with conventional thinking, but there is a point here. A "flanking" strategy of coining a new term may work better than a "head-on" strategy of challenging a widely-held set of views.]

2. We need to improve our marketing, both in reach and in targeting. I don't know if we managed to reach most of our target audience with news of the workshop, and whether they could identify themselves as the target based on the workshop brochure. We relied on our membership of various LinkedIn groups to reach the professionals we were connected to, and some of our friends helped out by relaying the message to their contacts, but I'm not sure what proportion of our target audience is reached in this way. Some more research and refinement is required here.

[Although we didn't get direct feedback on this aspect, I believe the price we are charging for a full-day workshop is reasonable ($450 plus 10% GST = $495, with an early-bird price of $400 plus GST = $440), and this includes morning and afternoon tea as well as lunch. The room and audio-visual equipment that we rented at the UTS Haymarket campus was OK, but not great. The air-conditioning couldn't be adjusted, and the data projector's screen obscured the fixed whiteboard, making it impossible to use both simultaneously. The catering was satisfactory in terms of quality but their commercial terms were somewhat unfavourable. We had to pay for a minimum of 10 people regardless, which was wasteful.]

Action items:

1. Over the next few days, I will split my draft white paper into two parts, one dealing with "How to do SOA", aimed at solution architects and designers, and the other dealing with "How to do SOA Governance and Management", aimed at enterprise architects and managers. In the latter, I will touch upon the differences between my approach and the industry definition of "SOA Governance", and use a different name for mine.

2. Future workshops will be better targeted. We will design our SOA workshops to focus on one or the other of the above topics, not both combined.

3. We will look for newer and better ways to communicate news of forthcoming workshops to industry professionals. Some brainstorming will happen in the weeks ahead.