Wednesday, May 26, 2010

Heavyweight Services Have Let IT Practitioners Down

Call it coincidence, but I had two meetings in the same day when different IT folk complained that a services approach was too cumbersome for them and argued for their applications to be able to connect directly to the databases of other applications to perform queries, joins, etc.

My involuntary shudder at this suggestion betrayed my SOA leanings, of course. I don't think these people realised how much future pain they would expose the organisation to by building implicit dependencies ("tight coupling") between their independent applications. But I could also empathise with their frustration.

The first generation of services in organisations has been well-meaning in intent, but expensive in practical terms. People ranted to me about the sheer effort overheads involved in trying to access data from elsewhere - setting up queues and queue clients, formalising XML messages to drop onto queues and pick from queues, parsing and generating XML, etc., etc., - all to do something that was essentially a SQL select or a join across two tables (theirs and another app's). I also heard a fair bit about the performance overheads of calling services to do simple things - operations that take seconds to do because they involve 2 or more separate service calls. "Decoupling at what cost?" was the refrain.

I'm forced to the realisation that our collective enthusiasm for SOA, while entirely correct and justified, has not provided a uniformly satisfactory solution to everyday IT practitioners. Bluntly put, we've let down the average developer by making their job unnecessarily harder, to the extent that even experienced designers who know the benefits of loose coupling are beginning to argue for a return to the Bad Old Ways.

A reader's comment on my previous post has made me think that "Data Services", especially of the RESTian CRUD variety, could be the answer. We still have a service interface that decouples applications from each other and their gory data structures and implementation details, but now we can set them up with minimal effort and call them with minimal overhead. Data Services could be the SOA SQL (That's a pretty apt pun, actually, if you pronounce SQL as "sequel").

More and more, I'm leaning to the view that most technical problems within an enterprise can be solved with "a web server, a database and some brains". This is a rich topic for a future post or even a whitepaper, but the unsurprising insight is that the third ingredient is probably the one in short supply at most IT shops ;-). (No insult intended, just that designers and architects tend not to step back and look for solutions from first principles, but rely on precedent and product-based approaches.)

1 comment:

The Unknown Programmer said...

It's probably worth looking at the problem why exactly is it so hard to build decoupled services.

There are so many ways of doing things too.

Maybe fire up 'events' to queue: X Happened, this is the resource URL for the Entity of concern. Go get it using REST if you wish, or not.

Maybe use intermediate stores like an Atom store where services publish documents to. You can watch for the changes using Atom feeds etc. You can do CRUD to documents there. Everything decoupled. A little bit like Java Spaces for documents.

REST and web servers are not a free lunch. Scale that bitch. Deal with badly behaving clients, location transparency gone - a different bag of problems. I tried things like reverse proxies to virtualize the URL space and cache transparently to get over some of the problems.

I am sure a lot could be done to remove grudgery from having to continuously translate from XML to objects to databases etc. Sometimes JSON comes to mind - at least it is a more object-like structure so it would take some of the pain away.

I am sure there is some happy medium for any given problem, but what that is ... well, that is the problem.