Namespace-Time: Unifying the SOAP/WS-* and REST Models

I have always believed that the SOAP/WS-* and REST models were duals, and I have a model to explain the seeming dichotomy. I wrote about it earlier.

Now here's the picture to provide the kiloword equivalent.

My First Mashup

So WSO2's Mashup Server has finally been released. As I've blogged before, this is a very exciting product for a number of reasons.

My first mashup is now public at mooshup.com, the hosted online version of WSO2 Mashup Server, where anyone can get an account and start kicking the tyres of this revolutionary technology.

The mashup is really simple, but it illustrates some of the power of the platform. I have two independent "services", a Grocery List service and a Grocery Prices service (Click the "Try it" link on each). The idea is that I create my grocery list independently knowing only what products I need and the quantities of each that I need. What I don't know are the (unit) prices of these products. Another service provides the prices for a number of grocery products, which is a superset of the products in my list.

My grocery list:

<order>
<line-item prod="tomato" qty="4" />
<line-item prod="potato" qty="3" />
<line-item prod="onion" qty="6" />
<line-item prod="chilli" qty="40" />
</order>

Grocery prices (independently available):

<prices>
<prod id="potato" unit-price="1.00" />
<prod id="tomato" unit-price="2.00" />
<prod id="onion" unit-price="3.00" />
<prod id="beans" unit-price="4.00" />
<prod id="chilli" unit-price="0.10" />
</prices>

What I need to do is "mash up" my grocery list with the price list that is independently published, so that I can see my itemised grocery bill, with individual costs and total cost.

To anyone who shudders at the thought of the XML manipulation this entails, the brevity of the following code should be vastly reassuring:

var total = 0.0;
for each ( i in gList.descendants("line-item") )
{
i.@amount = i.@qty * gPrices..prod.(@id==i.@prod)["@unit-price"];
total += Number( i.@amount );
}
gList..order["total-amt"] = total;

That's really all it takes. The "for" loop iterates through all elements in the grocery list with the tag "line-item". The first line inside the loop is the most interesting one. It's basically multiplying the quantity by the unit price to arrive at the amount, but this is where the "mashup" happens. It's using the unit price of the product in the grocery price list whose id matches the id of the product in the grocery list. It's like this SQL statement:

UPDATE T_GROCERY_LIST GL
SET GL.AMOUNT =
(SELECT GL.QTY * GP.UNIT_PRICE
FROM T_GROCERY_PRICES GP
WHERE GL.PROD_ID = GP.ID)

Then of course, there's the running total which gets inserted as a new subelement ("total-amt") under the "order" element of the grocery list. Note that if an element or attribute doesn't exist, it gets quietly added (e.g., the new attribute "amount" and the new element "tot-amt"). What distinguishes attributes from elements is the "@" symbol. If you avoid hyphens altogether in element and attribute names, the syntax is much simpler, but I prefer to use hyphens for complex tags, so I wear the extra syntactic overhead, but it's not much really.

What I finally get is the "mashed-up" XML:

<order>
<line-item prod="tomato" qty="4" amount="8" />
<line-item prod="potato" qty="3" amount="3" />
<line-item prod="onion" qty="6" amount="18" />
<line-item prod="chilli" qty="40" amount="4" />
<tot-amt>33</tot-amt>
</order>

which I can redisplay as a table.

My JavaScript is a bit rusty these days, but E4X has excited me enough to start coding again. Those of you who have to work with XML on a daily basis and curse the experience should look at E4X. It might just cause you to desert Java or C# for JavaScript.

If you do a View Source on the HTML page, you'll see that I've even used E4X to manipulate HTML from within JavaScript! Who wants to deal with the DOM? Treat whatever HTML structures you're working with as XML (well, it's all supposed to be XHTML anyway, so that's not an invalid assumption, right?). Once you've created the HTML structure you want, just convert it to a string using "toXMLString()" and use (the admittedly nonstandard but universally supported) "innerHTML" to insert that string into the existing HTML document.

Think of the possibilities. This goes far beyond content aggregation, which is what mashups are about. Mashup Server can talk both SOAP and REST (as consumer and producer). You can use JavaScript to code arbitrary logic, even simple processes. I'm sure someone will write a JavaScript library very soon to match the capabilities of WS-BPEL, and a translator to convert BPEL scripts into JavaScript, which will let them run more complex processes on Mashup Server. A process server! Next, combine all this with AJAX. You now have a one-stop shop for all your enterprise and collaborative computing needs. I wonder if there are nice programmatic interfaces to databases, directories, message queues, e-mail, etc. Something like the Spring templates in JavaScript would be nice.

I never had much regard for JavaScript when I was a Java programmer, but I'm sure developing a lot of respect for it now. A friend recently pointed me to Sun's research project, the Lively Kernel, which is essentially recreating Java applets using the more lightweight and ubiquitous JavaScript. Sun's Phobos library makes server-side JavaScript more feasible. Could it be that JavaScript is enjoying a resurgence?

E4X is the clincher for me. I know I'm not going to manipulate XML any other way right now. And WSO2 Mashup Server is my new hammer which makes every integration problem look like a mashup-able nail.

Namespace-Time: Why REST and SOAP Composition Models are so Different

There's a mini-controversy going on over the importance of hyperlinks, and whether they're only usable by humans. Jonathan Marsh seems to think that hyperlinks are less useful for machine-to-machine interaction. The REST viewpoint seems to be that services without hyperlinks "dead-end" the Web, hence the absence of solid support for links in SOAP is a further argument against its usefulness.

I have (as usual) a different viewpoint on this altogether. I think SOAP and REST are duals. They both model the Universe of Discourse, but in different ways. With apologies to Einstein, I think the universe being modelled exists in a Namespace-Time continuum. REST lies on the Namespace axis where hyperlinks play (URIs form a namespace). SOAP lies on the Time axis, which is not surprising considering its verb-orientation. That also explains why there is much more emphasis on process orchestration in the SOAP/WS-* world ("Do this, then do that" is time-sequenced). Even within the SOAP/WS-* world, while services are verb-oriented, metadata on services is noun-oriented. No wonder many recent registries for SOAP-based services are based on RESTful APIs (WSO2 Registry and Mule Galaxy).

It would be silly to disparage REST for being silent on "process", just as it would be silly to accuse SOAP of "dead-ending" services from a namespace perspective.

REST resources are composable in Namespace (to form content). SOAP services are composable in Time (to form process). Decide how you want to model your Universe, then take your pick.

Roy Fielding's Fundamental Omission

I think there is a fundamental omission at the heart of the argument that leads to the REST architectural style. It doesn't invalidate the style itself, but it does lead one to question REST's pretensions to being "the only way".

To understand this omission, it's important to study Roy Fielding's doctoral thesis, because that is the Bible of REST. The architectural principles of the Web were enunciated here for the first time, as an architecture. Before this, for many people, the Web just was. After Fielding's work, the Web began to be seen as something consciously designed and built according to well thought-out principles.

In Chapter 5, Fielding develops the REST model systematically and step-by-step, starting with a "null style" architecture with no constraints and then layering constraints upon it one by one in a controlled and logical manner. Indeed, this seems to be one of the core RESTian philosophies, that "constraints empower". With each set of restrictions that you place on an architecture, you actually get new benefits. When you're finished with all the constraints (capabilities) you need, the theory goes, you end up with the ideal architecture.

Indeed, on first walking through the analysis, the REST architecture seems to evolve naturally, inexorably and inevitably. With each new constraint (e.g., statelessness, caching, a uniform interface), the model moves closer to REST. The diagram towards the end of section 5.1 (which I'll call "All Roads Lead to REST" :-) reinforces this impression. No matter in what order we choose to layer our constraints (capabilities), we will always end up with REST.

Diagram: "All Roads Lead to REST"

At this point, I will invite my readers to pause and pop over to this page in Fielding's thesis to see for themselves if they're convinced, or if they think there's something fishy there. Don't read my post any further until you've formed your own opinion.

My fundamental issue with Fielding's argument is how he leaps straight from the null style architecture to the client-server model. When I first read this, I went, "Whoa, not so fast!" Because isn't there something between the null style and the client-server style? How about peer-to-peer?

I would argue that peer-to-peer is a more basic model than client-server, because we can always layer an additional constraint on the peer-to-peer model to convert it into client-server. The constraint is simply that only one peer can initiate an interaction and the other peer can only respond. In other words, request-response semantics.

Now here's my second objection to the REST philosophy. We don't have to glorify all constraints as unqualified positives for an architecture. Sometimes constraints are nothing more than just irritating, chafing restrictions. What does the request-response constraint buy us except the immediate loss of event notification capability (including callbacks)? (Using firewalls as a justification to ban unsolicited communications is circular reasoning. Firewall behaviour is informed by the web's request-response model.)

Besides, even if request-response wasn't a crippling limitation, client-server doesn't model the way things work in the real world. The real world is not client-server, it's peer-to-peer. Between organisations, there is a relationship of equals ("peers"). No organisation likes to see itself as controlled by another. All organisations are proudly autonomous. We need a peer-to-peer model to govern interactions between autonomous organisations, because that most closely models reality.

Even within a corporate setting, the various departments and product systems behave like autonomous units. My employer's HR system isn't sitting there passively, just waiting to answer queries and accept updates to employee data. It must actively remind me to fill in my weekly timesheet and remind my boss that it's time for my annual performance appraisal. But with a web architecture, that's needlessly hard to implement. With only browser-based interfaces to the HR system, we require users to log in to the system before they can receive any events. And that's just on login, or when they actually do something. They can't receive event notifications when they're just sitting there staring at the screen. So we resort to polling the system through page refreshes, or we go right around the limitations of the web architecture and get the HR system to send us e-mails instead. (Or we use AJAX clients, which hide the polling. AJAX is clever lipstick on the request-response pig.)

[I know that at this point, many REST advocates will point out that REST is not about one-way client-server. Each node can be both a client and a server. But two pairs of client-server nodes facing in opposite directions are not the same as a simple peer-to-peer system. That's trying to put Humpty-Dumpty together again.]

So now go back and read Roy Fielding's thesis again. Why, pray why, did Roy curse us with client-server with one perfunctory stroke of his pen, when he could have spent a few minutes exploring the peer-to-peer model? Mind you, there are enough constraints built into the peer-to-peer model when you assume autonomous peers. In fact, the autonomous peer-to-peer model itself can lead to the specification of important concepts like high cohesion (within a peer's domain) and loose coupling (between peers), the messaging paradigm, stateless interactions, discovery, logical addressing (which in turn leads to late binding, substitutability, routability, proxyability), metadata (generic policies as well as domain models), etc. Proxyability in turn leads to the notion of intermediaries (which can do caching, encryption, authentication and a whole heap of domain-aware functions as well). And then there's the whole hierarchy of capabilities that comes out of defining context and refining it:

Context -> Correlation -> Coordination -> Transactions

My, my, so many of the concepts we're wrestling with today, and we haven't had to move a step away from peer-to-peer! It appears that we can build a complete architectural style for SOA based on messaging between autonomous peers without taking on the needless additional constraints of client-server (which, as we argued before, buys us nothing but a loss of event notification capability). What was Roy thinking?

I can't help having mischievous thoughts at this stage. If Roy Fielding's analysis had been more rigorous, and he had properly explored the peer-to-peer model instead of jumping straight into the client-server model, would he have been known today as the father of SOAP messaging?

"The REST War has been Won!"

Thanks to Dilip Ranganathan for bringing these links to my notice:

http://www.markbaker.ca/blog/2008/01/17/rest-vs-soap-the-personal-cost/
http://steve.vinoski.net/blog/2008/01/19/many-thanks-mark/
http://lesscode.org/2006/03/19/rest-wins-noone-goes-home/

I'm really sorry to hear of Mark Baker's health being affected by all this "war" nonsense, and I can deeply empathise, because I've been doing similar evangelism for Linux and Open Source in the past, and suffered personally because people just didn't get it. (And now that Linux and Open Source have become mainstream, gone fashionable even, there's nary an acknowledgement of my early vision. "Oh, yes, Open Source has matured now (condescension, condescension)". I tell you, it hurts.)

Mark, if it's any consolation, there are individuals you may not even know about, who are grateful to you for your efforts. I've myself come to this debate rather late, but I'm happy that so much work has been done to bring the world a needed alternative.

But I wonder, what do Bill, Mark and Steve mean when they say "REST has won"?

Well, if they mean "REST has been accepted as a legitimate way to build distributed applications", then yes, I agree. Correction: I would say, it's well-known, but not yet universally accepted, but it deserves all the support it needs to get there. I'm the first to argue that REST is a beautifully elegant way to do simple things. Just look at the Blinksale REST API.

If they mean "REST has won over SOAP", I'm afraid I'm not seeing that at all. There are people in my own bank, SOA practitioners no less, who haven't even heard of REST. They're happily building systems using SOAP and WSDL and claiming great improvements in productivity! They're not lying either. So if the REST folk think SOAP is going to die quietly, they're very sadly mistaken.

(I feel a bit schizophrenic at times. With REST folk, I'm the skeptic, asking why it's a better model than SOAP messaging and raising uncomfortable questions around REST limitations. To people who are happily ensconced in their SOAP ivory towers, I'm the furtive revolutionary, going, "Psst! Have you heard of this really simple, elegant approach to SOA called REST? Have a look. You may find you could use it in many areas, and you may find it simpler and cheaper to implement.")

I like REST a lot, but I think it needs a bit more critical analysis around what it can't do well. I see a lot of religious fervour about REST, which is probably as bad as dogmatic opposition. And I think there needs to be a "SOAP Take 2", where some of the obviously bad ideas in WS-* get weeded out and the fundamental message-oriented paradigm is better implemented. I think we're moving towards a future where we have both, - a more capable REST and a more elegant SOAP.

I'm going to be blogging about some of these issues soon. Stay tuned.

Book Review: "Protocol" by Alexander Galloway, MIT Press

Stu Charlton recommended this book to me to read, as a way to explain the freedom of the Net that paradoxically stems from strict controls, i.e., protocols.

I ordered the book straightaway. It arrived yesterday, and I read it last night in a single all-night session. Not that it was unputdownable. Far from it (and more on that in a while). It's just that my sleep habits have become rather bizarre and all-nighters are now the norm.

The book makes a few interesting and important points, and the fundamental one is about "how control exists after decentralisation". That is in fact the book's subtitle. In contrast to earlier ages, when control was more direct, today's systems rely on protocols, which all players have to accept to even become part of the system. Protocols are highly formal. They are pervasive and must be internalised by all players (my words). Thus, even as systems become distributed with no central authority, the control systems do not disappear but manifest themselves as protocols. The contradiction with protocol is that it has to standardise in order to liberate. It has to be "fascistic and unilateral in order to be utopian".

There are interesting sections on the Internet's procotols, the hacker culture, cyberfeminism and "Net.art".

I like the idea that pro-protocol behaviour (giving out technology even to competitors) wins out over proprietary behaviour, which explains the success of Open Source. There's also the intriguing thought that protocol has a contradiction at its core, that "in order to be politically progressive, [it] must also be partially reactionary."

I learnt a few interesting bits of trivia, and also some important (?) subtleties, e.g., the difference between "decentralised" and "distributed" in the author's view. "Decentralised" means having many hubs as opposed to having just one hub, as in a centralised system. A decentralised system still differentiates between hubs and leaf nodes. Leaf nodes can only connect to hubs, not directly to one another (hubs are connected to one another). A distributed system, on the other hand, has no "hubs" at all. All nodes are equal and any node can connect to any other. It is the distributed system that Galloway argues requires (or manifests) protocol.

Another interesting insight the book offers about the web is how there are simultaneous and opposing forces at work. The web itself is distributed, but DNS is hierarchical. While content on the web cannot be deleted by a centralised authority, it can be rendered inaccessible by merely changing some DNS entries at the root level. An entire country can be made to "disappear" from the web through the mechanism of DNS.

Now for the criticism.

Alexander Galloway reminds me of the "literary intellectual" of C.P. Snow's "The Two Cultures", the kind who Snow says "took to referring to themselves as intellectuals when no one was looking". His style seems intended to impress rather than inform.

I believe his entire book (all 246 pages of it) can be expressed in about 10 pages or fewer if translated into English from pseudo-intellectual gobbledygook.

Some sections are almost verbatim reproductions from the publications of Karl Marx, Gilles Deleuze and Michel Foucault, with interminable sequences of quotations therefrom. It reminds me of the old joke that copying from one person is plagiarism, while copying from many is research. This book is full of research.

Alas, he seems not to understand the power of protocol himself! In the conclusion, he says, "The fact that Microsoft has not yet replaced TCP/IP with a commercial product of its own is one of the miracles of computer history. Chances are this will happen very soon". Really!? And this right after the section on speed bumps when he defines protocol as something that appeals not to the mind but to the "body", as something that one is inherently forced to do, not something that appeals to the intellect or is applied as overt force. Replacing TCP/IP is something Microsoft simply cannot do because it will run afoul of "network externalities". That will be virtually impossible for Microsoft to pull off, and not because someone is standing with a gun to their head either.

You will feel right at home reading this book if you understand the following words and phrases (Warning: even if you think you recognise a word in this list, rest assured the author uses it in a way you may not recognise):
semiotic, periodization, machinic, panopticon, dispositifs, immanent, au milieu, rhizomatic, territorializing, phenomenologically, radical dislocation, phoneme, hegemonic, protocological, affective, imbrication, anti-anthropological, autochthonic transformation, depresentify, plenitude, discursive, subjectless patterning, dividuals (no, that's not a typo), prefigure, reification, commodity fetishism, camera obscura, dialectical method, montage, vitalism, second-order system of signification, patina, vitalistic discourse, self-aestheticization, epistemological inversion, social hieroglyphic, meme, labile, anti-entropic, ontological rub, hyper-Platonic, hypertrophy, elided, juridical, autodidact, hyperlinguistic, protean, apotheosis, machinic phylum, valorizing, libratory, elliptical.

I notice that many of these words appear in quotes from other researchers. Perhaps that was the purpose of the quotes, to add to the book's impressive vocabulary. Thanks, Dr. Galloway, you've impressed me - NOT. An author can choose to either communicate, or to drown the reader in verbal garbage. You've done the latter, and wasted 6 hours of my time in the process.

Call me lowbrow, but I'm definitely of the "my 4 year old could do that" school of modern art appreciation. I'm impressed by Michelangelo and underwhelmed by Picasso. Someday we'll all learn about how Picasso laughed himself to death. And I suspect Alexander Galloway is similarly having a good laugh at his readers' expense.

Service-Orientation and Object-Orientation: An Analogy from Marketing

To continue my earlier train of thought, perhaps an analogy from Marketing will help to illustrate the difference between Object-Orientation and Service-Orientation.

Rookie marketing folk are often admonished to "sell benefits, not features."

It's no use telling a potential customer that your product is orange in colour, is made of plastic and consumes 1.5 volts of electricity. The customer is just going to think, "How does that benefit me?" and walk away without making the purchase. To be effective, the marketing person needs to cast those features of the product into benefits for the customer.

"It's bright orange and easily spottable even in dim light, so you'll never lose it or leave it behind in the park by mistake."

"It's made of plastic, so it won't rust, it's lightweight and non-toxic, safe for kids."

"It runs on 1.5 V batteries which are readily available and convenient to use."

When features are reframed as benefits to the consumer, a sale is more likely to happen.

Object-Orientation is like talking about features. It's all about (State and) Behaviour, i.e., what the object can do.

Service-Orientation is like talking about benefits. It's about what the consumer can do (with the object, or preferably leave the object out of the discussion entirely).

I suppress a smile when non-IT Indian friends remark about how "service-oriented" India has become these days. No, they don't mean that India has started using SOAP and REST. It's just that the old arrogant sales attitude of "Here's our product. Take it or leave it" has gone, replaced by a much more consumer-friendly attitude. There's far greater willingness to customise and to offer value-added services. That's because the focus has shifted from what a product can do to what the consumer wants to do. The product becomes incidental to the discussion.

This difference in focus is subtle but very important.

I suspect the reason for the popularity of aggregation services today is primarily because aggregation is something that objects themselves cannot do, because it's a "meta" capability. An airline system built according to Object-Oriented principles may be able to tell you all the details of a flight from city A to city B, but it can't reflect on itself and others like itself to tell you which is the cheapest and most convenient flight from A to B. That requires Service-Orientation. Lots of Web-based services are like that, about finding the "best", the "cheapest", the "most optimal".

[Aside: REST is based on the Web, and since SOAP-RPC is nothing but Object-Orientation-made-remote (unsuccessfully at that), REST obviously works better than SOAP-RPC. But SOAP Messaging is as Service-Oriented as REST, because it also focuses on what the service consumer can do (with specially tailored verbs that imply the service consumer as the subject).]

To my mind, aggregation services are low-hanging fruit for service-orientation. It's an obvious area where OO systems can't play. The attitude needs to become more pervasive, even for non-aggregation services.

To recollect our discussion about EJB, that's why Entity Beans require Service Façades, i.e., Session Beans that sit in front of them and change the focus from what the Entities can do to what the service consumer wants to do. Aggregation is one of the things that Session Beans do. It's the most obvious benefit, because making an interaction coarse-grained is more performant than having a highly chatty interaction with hundreds of Entities directly. But that's not the only benefit.

As I said before, REST is a great Service-Oriented solution even though it appears to be focused on entities just like Entity EJBs. The difference between REST and Entity EJBs (or between REST and OO for that matter) is that the focus is not on what the Resources can do, but on what the service consumer can do with those Resources. The four HTTP verbs of REST have the service consumer as the subject, not the resource. They are transitive verbs that have the resource as their object. You GET a Resource. You DELETE a Resource. Etc. It's easy for "you" to GET or DELETE a set of resources using the same idiom.

By contrast, OO systems have the "object" as the subject of the verb (if that doesn't confuse you), and the verbs tend to be intransitive (e.g., someObject.changeState()). (Where the verbs are transitive, another object (the "object" of the verb) is usually a parameter to the method verb, e.g., someObject.modify( anotherObject ).) It's not possible to extend an object's method to deal with a set of similar objects instead. That's because the focus is wrong from the start.

It's getting clearer in my mind now, but I realise the concept can take some getting used to. For now, a simple mantra to distinguish Service-Orientation from Object-Orientation:

Benefits, not features.

(I hope this doesn't spark off a new TLA fad - Benefit-Oriented Programming (BOP)). It's still just SOA, guys.

Macworld - Is This the Big One?

Well, Macworld is on again.

I blogged in March about what could happen. Will Steve Jobs do it? I can't wait.

Post-Macworld 2008 update: Well, he didn't. Another opportunity lost, Steve. You could have been the next Bill Gates (and I mean that entirely positively).

Resource-, Service- and Object-Orientation

I've had an epiphany. I can tell you the difference between systems that are Resource-Oriented (REST), Service-Oriented (REST and SOAP/WS-*) and Object-Oriented (Java, C#, etc.) And it's very simple, something that can be explained with basic English grammar.

No, I don't mean the old "noun-oriented" and "verb-oriented" classification. That's actually quite a meaningless classification. REST and Java are both noun-oriented, but they're opposites in a more important sense. REST and SOAP/WS-* seem to be opposites because one is noun-oriented and the other is verb-oriented, but they're both actually very similar.

No, the very term "noun-oriented" is meaningless because all these systems deal in verbs. The question is, what's the subject of the verb? In other words, who does the act represented by the verb?

If the "object" itself is the subject of the verb, i.e., it's an intransitive verb, then the system is Object-Oriented. If an external "service consumer" is the subject of the verb, then the system is Service-Oriented. In other words, Service-Oriented systems are Subject-Oriented as opposed to Object-Oriented.

Example: You start your car. The car starts.

It seems like the same verb in both sentences, but they're not actually the same. The verb "start" in the first sentence is transitive, i.e., it has an object. "You" are the subject and "your car" is the object. In the second sentence, "start" is an intransitive verb. "The car" is the subject and there is no object at all.

Service-Oriented (SOAP/WS-*):
verb/operation: start (specific verb)
parameter: car ID (i.e., which car to start)

Service-Oriented - Resource-Oriented subtype (REST):
verb: PUT (generic change of status)
resource: car URI (i.e., which car to start)
parameter: new status: "started"

Object-Oriented (Java):
car.start() (this is a different "start" from what the driver does to the car)

Note the difference between two "noun-oriented" systems here (REST and OO). An Object-Oriented system specifies what an Object can do (the car can start (itself)). A Resource-Oriented system specifies what can be done to a Resource by the service consumer (you can start it). So REST is Service-Oriented. The focus is on what the service consumer does to the resource (transitive verb), not what the resource does (intransitive verb). GET, PUT, POST and DELETE are all transitive verbs, and their object is always a URI.

If we want to take a less ambiguous verb than "start", how about beating a drum? A drum booms when it is beaten.

The Service-Oriented view of a drum is "You beat the drum". The Object-Oriented view is that "the drum booms". The concept of "beat" doesn't enter the Object-Oriented world unless you're modelling the drummer.

Service-Oriented (SOAP/WS-*):
verb/operation: beat (specific verb)
parameter: drumID (i.e., which drum to beat)

Service-Oriented - Resource-Oriented subtype (REST):
verb: PUT (generic change of status)
resource: drum URI (i.e., which drum to beat)
parameter: new status: "beaten"

Object-Oriented (Java):
drum.boom() (has nothing to do with "beating" because the viewpoint is different)

A few extra points:
SOAP/WS-* shares a trait with another much-maligned technology - EJB, specifically Session Beans. I've always felt that Session Beans were not true OO. What did they model? The answer now stares me in the face. Session Beans are Service-Oriented. They deal with what the service consumer can do, not what an (internal) object can do.

If you look at Session Beans from a purist OO perspective, you will see that the Session Bean's methods are the only important thing. The containing Session Bean class is only there because Java does not allow first-class verbs (freestanding methods). All methods must be wrapped inside classes. So the Session Bean class doesn't really model anything in the domain - it's not a Domain Object at all! It just satisfies a contractual requirement of the Java language to have classes wrapped around methods. That's why Session Beans tend to have such weak names as "Account Manager". They just act as catch-all containers for methods that seem to belong together. That's also why more than one Session Bean can often seem appropriate to house a certain method. Where does the changeAddress() method go? Is it CustomerManager.changeAddress( customerID, newAddress ) or AddressManager.changeAddress( customerID, newAddress )?

And why is SOAP/WS-* like Session Beans? What's a SOAP "service"? It's the SOAP "operation" that's important, - the verb. The SOAP service is a catch-all "noun" that holds a bunch of related operations together. In other words, SOAP (or WSDL to be precise) suffers from Object-Oriented thinking of the very kind that afflicts Session Beans. Why can't we have freestanding verbs in WSDL?

To round out the analogy, REST is like Entity Beans, but with one crucial difference. Both model resources. Both Entities and Resources have lifecycles. The methods on Entity Beans are lifecycle methods, not so much methods that make sense to the service consumer or which fit into the service consumer's "process". [This ties back to an earlier discussion on the difference between a process and a resource lifecycle. One deals with the viewpoint of the service consumer, the other deals with the viewpoint of the resource or object itself.] That's really why Entity Beans need Service façades (officially called Session façades in the literature), even though the immediate reason for Service façades is to avoid the performance overheads of remoting individual entities. I believe Service façades perform the important function of shifting the viewpoint of the system from Object to Subject and thereby make Object aggregation possible.

REST avoids the fine-grained remoting problem that plagues Entity Beans because it's Service-Oriented. The focus is always on what can be done to Resources. With Entity Beans, the focus is on what an Entity can do, and it requires a Session Bean to change the focus to what can be done with Entities. So it's easy to deal with a bunch of Resources in REST. Just adjust the URI and some parameters so that they refer to a set of Resources rather than just one. With Entity Beans, that is simply not possible. When the very paradigm focuses on an individual Entity, how does one deal with a set of them without enumerating them all? So the Session bean (Service façade) has to step in to do the expensive enumeration locally and provide a single remote interface that is less expensive.

So there you go. Object-Oriented systems tell you what Objects can do. The method verbs are largely intransitive, (although when an object's method takes another object as a parameter, it tends to be a transitive verb). Service-Oriented systems tell you what can be done, period. Resource-Oriented systems, which are a subset of Service-Oriented systems, tell you what can be done with Resources. The other Service-Oriented systems (the "rest", heh!) tell you what can be done in such specific terms that an object may not have to be specified, although it can. The object in such cases is the set of parameters to the verb, i.e., the XML input document defined for the SOAP operation.

Settling Some Accounts with REST Advocates

(Couldn't resist a cheeky title again, although in deference to those who find the term RESTafarian trivialising, I won't use it this time.)

Stu Charlton posted a lengthy "rebuttal" to my post on "Paying the RESTafarians Back in their Own Coin", and I've now answered his points. You may find that nothing he said really invalidated any of my points, and I think I may have unearthed a couple of his misconceptions around the application domain as well.

I must say that it appears people don't read what I have written carefully enough. They assume I'm saying something and then "refute" it. Others find such "refutations" convincing! Still others like to call me names without reading my post carefully and seeing that I haven't actually said a single negative thing about REST! (Now I'm being thin-skinned :-).

Anyway, I hope these points lay to rest (no pun intended) such arguments.

The Good in WS-* (Update)

See post below for my initial, entirely positive reactions to Stu Charlton's critique of the WS-* family of Web Services technologies.

Now for some qualifiers.

I wonder why Stu didn't include SOAP in the list of "good" technologies. I don't mean the overloaded meaning of SOAP (the RPC style of usage, for example), but quite simply the SOAP message, the envelope. All the WS-* stuff is useless without the primary vehicle of the SOAP message which carries them from endpoint to endpoint. One has to admit there is a certain elegance in its simplicity, analogous to the humble IP packet that is the workhorse of the Internet (read more here).

The envelope has just two parts - a header and a body. All the domain-neutral WS-* tags that Stu talks about go into the headers. The body contains domain-specific XML payloads that hopefully conform to some defined schema. Unlike with the TCP stack, where packets are recursively enveloped within other packets (e.g., HTTP inside TCP inside ESP (IPSec)), ultimately being packaged inside IP for transport over the "wire", SOAP has a single-level layering approach. All quality-of-service tags represented by the WS-* standards are placed on a co-equal basis within the SOAP header, which is OK because they largely represent orthogonal concerns. (There is still some interdependence, e.g., you can't just have WS-Security if you're using WS-ReliableMessaging; you need WS-SecureConversation as well, because the system now has to be cognisant of conversation state.)

The SOAP message represents, at once, a domain-neutral wire protocol that supports any application and a domain-specific application protocol that defines the verbs, nouns and conversation grammar of a particular application domain.

I see both SOAP and REST as ways to move domain-specific XML documents around. From that angle, they're both defining a core application wire protocol that can support any application domain (HTTP or SOAP-with-headers). Each application has to define the nouns, verbs and conversation grammar appropriate to it and exploit this "wire protocol" to move the XML noun payload in the context of an interaction verb (GET/POST/PUT/DELETE being pre-defined within the REST wire protocol or a bespoke one for SOAP/WS-*) and thereby build up a domain-meaningful conversation, what the REST folk call a state machine and what the SOAP/WS-* folk call a process.

I think we're all in violent agreement over the need to move XML around in a series of interactions that occur within the larger bounded context of a state machine or process. As has been said in many other contexts, our similarities are greater than our differences, so let's not read too much into the SOAP-REST debate.

William Vambenepe has more to add on this. It's an earlier post, but very topical. There's a lot more promise in SOAP than most of us realise, and the standards themselves sometimes belie that promise. Read William's post to see what needs to be fixed, hopefully in the next version of the WS-* standards.

The Good in WS-*

Stu Charlton has posted a fantastic and in-depth critique of SOAP/WS-* standards on his blog:

This is exactly the kind of analysis I was looking for. It's pretty humbling to realise the depth of knowledge that's out there. I guess I've lots of learning to do...

Too bad there doesn't seem to be any provision for comments on his blog. I wanted to say some nice things :-(.

Thanks, Stu!

The Domain Model, its Representation(s) and their Composition

Rajat, Vikrant (the latter's an outdated link) and I have been having these discussions for a while now. Our thinking is summarised in these two papers:

The first is an "astro-geological" meta-model that explains how the Domain Model relates to various internal and external Representations. It tries to unify the concepts of Domain-Driven Design (DDD) and Service-Oriented Architecture (SOA). A Domain Model is a system's own view of its Universe of Discourse. Representations are how the system allows external parties to view itself. SOA deals with representations that are decoupled from domain objects, and these representations could be noun-based (REST) or verb-based (SOAP/WS-*).

The second is a model for Representation Composition, which blends the concepts of Process Orchestration and Content Aggregation.

The analogy of the astro-geological meta-model is that the Domain Model is like the surface of the Earth (or any other planet). When these celestial orbs communicate with each other (the "astro" part of the meta-model), they exchange representations of their Domain Models rather than expose their Domain Models directly. This is classic architectural practice to avoid tight coupling between systems. The Domain Model includes State and Behaviour, so one could think of representations of State as well as Behaviour. We believe these are commonly known as REST and SOAP, though purists from both camps may take umbrage at the comparison ;-).

The Domain Model is implemented atop infrastructure (the "geo" part of the meta-model). Sometimes, it's in the form of data held in relational databases. Sometimes, it's functions coded in legacy systems. Again, these are representations of the Domain Model, some of State and some of Behaviour. O-R Mapping tools are commonly used to create representations of the State of a Domain Model built with an OO paradigm, and these representations are relational. Similarly, Behaviour in the Domain Model may be represented as procedures on a mainframe, with the appropriate loosely-coupled translation happening between the two.

The Domain Model itself does not have to be purely OO-based. Many different paradigms may be used to model a domain, - OO, AOP (aspects), Rules, Process definitions, etc. The key messages are that representations are always required whenever a Domain Model is to interface to any other entity, internal or external. These representations should be loosely coupled to the Domain Model.

Coming to Representation Composition, we believe this is where SOA really hits its stride. An enterprise consists of various Domains, each with its own model and its own external representations (SOAP or REST-based). The challenge is to weave greater meaning from these individual representations and thereby add value to the enterprise.

Process orchestration concentrates on the verb paradigm. While this is powerful and enables the composition of "sentences" that include compound verbs and conditional clauses, the relative neglect of primary nouns in this model is cause for concern. This is perhaps why content aggregation (and content management in general) occupy a separate niche. Mashups (and the older Portals) are stabs at content aggregation, but both suffer from limitations as described in the document. We don't see a rationale for this dichotomy between the composition of representations of State and the composition of representations of Behaviour. Both need to be managed using a common idiom, and true power and value-add will come from that composite model.

As mentioned before in this blog, the WSO2 Mashup Server comes close to delivering on this vision, so we don't think our model is vapourware. We're surprised that the WSO2 people are themselves rather understated about the capabilities of Mashup Server ("lightweight personal mashup server"). Hypothetically, if a JavaScript library is developed with classes corresponding to various components in a BPEL engine, and if an XSL transform is done from XPDL (the XML representation of a BPMN process diagram) to a set of JavaScript configuration statements, then we can run processes on the WSO2 Mashup Server just as we can do content aggregation today. One unified platform for "Representation Composition"! Consume SOAP or REST, compose in a noun-oriented or verb-oriented way, and expose in turn as SOAP or REST.

All of this is non-visual, and we already have a model for how the Presentation Tier should interface with a SOAP- or REST-based Service Tier. It's called SOFEA (Service-Oriented Front-End Architecture), which is described in this paper and discussed on this blog.

Comments as usual are welcome.