Sunday, October 21, 2012

The Promise And The Potential Of WSO2

(I write this piece with a bit of trepidation as I don't know how it will be taken, but I mean it with the very best of intentions and wishes for my ex-colleagues.)

Looking over the list of my FaceBook friends, I am struck by how many of those I connected with during my brief assignment with WSO2 (Aug-Dec 2011) have moved on to pursue graduate programs of study at US universities. Just like Macquarie Bank in Australia was once called "The Millionaire Factory" for making so many employees rich through bonuses, this little company is rapidly becoming an Intellectuals' Factory. If they succeed in staying in business over the next decade, they will have seeded some very powerful and influential links in academia and the higher levels of the technology world.

Which brings me to my main point - does WSO2 have what it takes to survive the next decade? I believe they do, and not just survive but also move up to the next higher level. But to do that requires a kind of thinking I have not seen from the company's leadership so far.

For at least the last five years, I have been searching for a "next-generation SOA company". I thought I had found it with WSO2, but was quickly disappointed when I realised that this was a company with traditional SOA thinking and only a next-generation business model (free software and paid-for support). That's not quite what I was looking for, and to me, it explains why WSO2 hasn't yet hit the big time in spite of a decent product suite and some very smart, sincere and hard-working people. They're just not revolutionary enough for the market to take notice.

OK, so what do I have in mind?

The view of SOA as being ultimately about SOAP-based Web Services in its implementation (OK, and some REST as well) is so last-decade. That view of SOA, I am convinced, is actually toxic to organisations. The impacts of that kind of SOA to agility and cost are entirely negative. (I have numbers to prove it but I could be sued by more than one ex-employer for breach of confidentiality, so you'll just have to take my word for it.)

If you sell traditional SOA with a different business model, you're not even half revolutionary. You're an interesting sideshow to the main game, and when the traditional SOA model gets discredited, as it largely has already, you get washed away in a way the IBMs and Oracles don't because of their size. (Which is a pity because I simply hate the IBMs and Oracles for what they have done to the industry - milked customers while providing them no benefit.)

My view of SOA is simple - it's "dependency-oriented thinking", and it applies to every layer of the organisation - Business, Applications, Information and Technology. Unfortunately, every SOA guru, after paying ritual obeisance to the notion that "SOA is not technology", proceeds to insult our collective intelligence by discussing Web Services technology (or lately, REST). It seems we just need to look away for a second after someone says SOA is not technology, and we find they're talking technology when we next look!

Similarly, I have a very jaundiced view of the term "SOA governance" as it is popularly applied. I suspect most SOA experts wouldn't be able to define the term in a readily comprehensible manner if they were tied to a chair and threatened with a copy of Lotus Notes. I also suspect most of them use the term "governance" for effect when all they mean is plain old "management".

My definitions of governance versus management are also simple - "doing the right thing" versus "doing things right" - in other words, the "what" versus the "how".

And so, "SOA Governance" is nothing but the process of determining the right dependencies that ought to exist at all layers (Business, Applications, Information and Technology) and identifying the dependencies that do exist. "SOA Management" is about using these as a starting point and eliminating the dependencies that should not exist, formalising the ones that should exist into contracts, and ensuring that fresh dependencies do not creep in. That's all there is to SOA governance and practice, but the benefits are significant and will not escape attention.

I talk about all this at some length in my InfoQ interview. [Update 29/11/2012: I've now also written a white paper on it, which interested folk can download from]

There is so much potential for a truly SOA-enlightened consultant to enter a client organisation and point out all the areas of tight coupling that are driving up their costs and risks and driving down their agility. Heck, if all that the consultant did was concentrate on tight coupling at the data layer, so much wasted potential could be unleashed. That's the power of SOA in the right hands.

What I want to see in a next-generation SOA company is an organisation that is not fixated on technology but on education and consultancy, because it's SOA thinking that is so badly in need of a reboot. WSO2 is full of smart people, but they're focused on the wrong thing. It's not technology they need to be tinkering with. They need to apply their screwdrivers to the minds of IT and business folk at all levels. They need to hire fewer nerds and more business-savvy consultant types trained to think about dependencies. People with backgrounds in risk management, project management and contract law are especially good at thinking about every kind of dependency or "fine print" that could trip up an undertaking, and these are the kinds of people needed for SOA consultancy.

Admittedly, that's a bit hard to do when you do technology too well. Perhaps WSO2 ought to start by setting up a consultancy arm (which isn't focused on selling their own technology support services), getting that division to drive SOA thinking from the business layer down, and then watching it outgrow its older sibling as customers start to see its dramatic impact.

Can they rise to the challenge?

Tuesday, October 16, 2012

A Simple and Nice Open Source Tool To Draw Sequence Diagrams

I was looking for a very simple tool to help me put together some sequence diagrams in a hurry, and a Google search turned up sdedit.

This is an Open Source Java-based tool. You download the jar file and run it with the command

java  -jar  sdedit-4.0-RC1.jar

It looks like a standard visual tool, but you don't actually interact with it visually. You type in statements in the bottom right window. Now that may sound like a retrograde feature, but it's actually very elegant, and you need to try it to realise just how simple it is.

It took me less than 5 minutes to create this PNG image file (generating a PDF or image file is just a couple of button clicks, although the PDF version does suffer some distortion):

The format of the text is really simple, and here are the rules.

It's like a simple program. You first declare all the entities that will be interacting, leave a blank line, then specify all the interactions.

There's one icon for the Actor type, and a standard rectangle for every other. The tool imposes a few simple rules. Undeclared entities are flagged. Only Actors are allowed to initiate interactions, and their interactions are always asynchronous.

It's also possible to mark off sections that are loops or that represent alternate flows. This is one tool that is really worth trying out.

You can use this code to get started:

#![eCommerce Example]
customer:Actor "Customer"
portal:Process "Sales Portal"
productdb:Database "Product Database"
paymentgw:Process "Payment Gateway"

portal:productdb.get product list
portal:customer.list of products
[c:loop for selected products]
  customer:portal.add to shopping cart
customer:portal.check out
portal:customer.redirect to payment gateway
customer:paymentgw.make payment
paymentgw:customer.paid - redirect to portal
customer:portal.back to portal
portal:customer.confirmation of order

Friday, October 12, 2012

My Interview at QCon, Where I Mostly Rant About SOA

I attended QCon New York in June this year and spoke about loosely-coupled Identity Management. I was also interviewed after my talk by Eric Newcomer, and the video of this interview has now been put up by InfoQ (the organisers of the conference).

The video is here. The transcript of the entire conversation is right below the video, so click on the "plus" sign to the right of each question to see the full text of the question and my answer.

I'm glad I got the opportunity to rant about SOA and how badly it has been misunderstood and misapplied by the industry.

I also managed to get a couple of sentences in about Identity Management, which I what I went there to speak on :-).

Tuesday, October 09, 2012

The Luxury of Instant Information Gratification

Have you ever thought about how easy it is to scratch an information itch these days?

I was looking at a presentation on a technical tool that is used in emergency warning systems. The system works by locating all mobile cell towers in an affected area, then sending SMS messages to subscribers within range of those cells.

One of the slides in the presentation illustrated the concept using this map:

Mobile cell tower coverage superimposed on a map - but which city is this?

I was distracted by an entirely irrelevant angle to the point being made. I noticed that all the street names on the map looked Spanish, and wondered which city it was from.

Well, Google and Google Maps were just a click away, so I did a quick search on "Calle de Alcala" (one of the street names I could see at the top of the map) and was immediately informed that this was in Madrid, Spain. [In fact, I wrongly spelt it "Alcaia" and Google helpfully corrected me.] I then switched to Google's Map view and zoomed in till I could see the "Sevila" metro station that was in the original, and positioned the map so I could compare it with the one in the presentation.

Yup, it's Madrid all right.

It took less than a minute for my idle curiosity (for it was nothing but that) to be satisfied.

And thinking about it, surely this is an amazing aspect of progress in the modern world! We didn't have this capability even 20 years ago. Trillions of trivial questions must have gone unanswered because there was no easy way to answer them. I don't know how I would have found the answer to my question in the pre-Google days.

It's nothing short of a luxury that we can afford to scratch our information itches almost without thinking. What wondrous innovations and advances will build on the shoulders of this capability, I wonder.

Wednesday, October 03, 2012

98% Compression For Diagrams - Unbelievable?

I didn't believe it was possible for compression technology to get any more efficient than it is today.

Take a PNG diagram like the one below:

How small do you think a file containing this diagram could be? The challenge is to get it under a kilobyte. Your choice of file formats and compression algorithms!

The uploaded version of the diagram on this blog may be smaller, but the one I have on disk is a PNG file that is 40,468 bytes in size. Compressing it using zip results in a file 38,394 bytes in size, - a 5% reduction.

That's not too surprising, because PNG is already a pretty efficient file format. All said, that's still way off our target size of a kilobyte. We're asking for an impossible 97.5% compression ratio.

Now, what if I told you there was a way to get there with no loss of information?


The above diagram is the standard example on the website. The website hosts the eponymous ditaa tool, which is Free Software under the GNU GPL. Ditaa converts ASCII art into image files. Aha!

The above diagram is generated from an ASCII art text file that weighs in at 1,569 bytes, which is still 50% above our target, but since ASCII text is very space-inefficient, it can be further zipped into a file that is just 713 bytes in size. Compared to the size of the original PNG image that it captures in a lossless representation, that's a compression ratio of 98%!

Here's what the ASCII art version looks like (this is a PNG screenshot, by the way ;-).

[How does one draw ASCII art in the first place? Isn't that a pain? Not really. There's another wonderful tool over at that provides a simple UI to create ASCII art diagrams (doesn't work with IE). Use the tool and follow the conventions specified by ditaa, and you can create really compact diagrams that can be expanded whenever required.]

To me, it's not the compression ratio that's so cool. It's the whole approach to representing technical diagrams. The efficiency is in being able to capture the basics of representation in such a minimalistic way, yet sacrifice nothing in terms of the output's visual appeal.

Full marks to asciiflow and ditaa!