An Open Letter to the New Australian Prime Minister, Kevin Rudd

Congratulations on your election victory, Mr. Prime Minister! (I think I can safely address you as such even though you haven't been officially sworn in yet).

I would like to get a quick word in before you are deluged by work and a cacophony of voices representing the spectrum of Australian interests.

Your campaign promised Australia a world-class education system, with an emphasis on "digital schools" and a computer for every child in Years 9 to 12. I laud your vision and extend my support and best wishes towards its successful implementation.

I must however caution that to bureaucrats who haven't kept up to date with the latest developments in the computer industry, implementing your policy may seem to mean nothing more complicated than signing a multi-million dollar agreement with Microsoft, but there is a far more efficient use of taxpayers' money.

I would urge the Labor government to look closely at the Open Source movement not only to provide basic operating and application software for school computers, but also to provide access to courseware through an "educational commons." When you do the numbers, you will find that very significant savings can be obtained by substituting perfectly good Open Source equivalents for proprietary and expensive software such as Microsoft Windows Vista and Microsoft Office. I am talking about Linux, Firefox, OpenOffice and countless other cooperatively-developed software products. Not only are the Open Source products cheaper, they are more open and standards-compliant, which means they play nicer with software from other sources. Best of all, they run on less powerful hardware and require fewer hardware upgrades, reducing the amount of money that needs to be spent just to get the infrastructure up and running. Ongoing running costs are also likely to be cheaper. And that's just the hard (dollar) benefits.

An exposure to Open Source software and an "educational commons" will also help to create a new generation of technology- and community-savvy students who can more effectively participate in an increasingly collaborative world. Our educational investment is not in computers, after all, but in intellectual assets - our children. It's the Open Source way that our children need to learn in order to adapt and survive in tomorrow's world, and your government's policies can make that a reality.

I know that these revolutionary approaches and solutions will be bitterly resisted by the technology establishment, but if the election of your government is not a vote for change, what is?

As a voter, taxpayer and concerned citizen, I would like Australia to get the best for its investments in technology and education, and I believe the Open Source way is the best way forward for this country.

With my very best wishes for your term as Prime Minister,

Yours sincerely,
Ganesh Prasad

Why RPC is Evil, XML-RPC is Doubly Evil and SOAP-RPC is Triply Evil

After reading my own review of Yuli Vasiliev's book, I realised I had to better explain my "SOAP-RPC is evil" comment.

Actually, SOAP-RPC is not just simply evil but triply evil, and here's why.

The root of the whole evil was the original arrogant assumption behind the notion of distributed objects - that we can make remote objects appear as if they are local. It was only much later that Martin Fowler came out with his First Law of Distributed Objects ("Don't Distribute Your Objects").

Quick question: How can we make a remote object appear to be local?
Correct answer: We can't, period.
Naive answer: Serialise the remote object, transport it over the wire and deserialise it to create a local copy.

The reason the naive answer is so badly wrong is that copies behave very differently to the original objects. To be precise, when a copy is changed, the original is not changed. So "hiding" the remoteness of an object through some object serialisation sleight-of-hand doesn't work. The component or application that gets a reference to the copy must know that it is a copy, otherwise all kinds of application errors can occur.

So that's why RPC (Remote Procedure Call) is evil. It tries to make remote objects look local. If one is not careful, this can create serious errors in applications.

The other issue is the serialisation mechanism. Even assuming it's OK to serialise an object and recreate it elsewhere from the serialised representation, what is the mechanism used for serialisation? Java serialisation works because we have Java on both ends. In recent times, XML has become popular, and some bright spark must have thought of "XML serialisation". The concept is simple. Convert a Java object to XML format, transport it over the wire to a remote host, then convert the XML document back into a Java object to create a perfect copy, a "remote object". The problem with this assumption is that there is an impedance mismatch between Java and XML. We can do things with Java that we can't do with XML, and vice-versa.

So converting a Java object into XML isn't straightforward. Things get lost in the translation. Converting from XML back into Java isn't straightforward either. More things get lost in the translation. So this "serialisation/deserialisation" using XML as the transport format doesn't result in perfect copies at the other end. This is XML-RPC, and when it's used naively, it's easy to see why it's doubly evil.

As SOAP-RPC evolved beyond XML-RPC, the wire protocol itself began to be seen as a "contract" between systems. In other words, the SOAP message could be used to decouple implementations at either end. All of a sudden, the XML document going over the wire was not expected to be just a Java serialisation mechanism, it was a technology-neutral serialisation mechanism. So now it became fashionable to think of serialising a Java object into an XML document, wrapping it in a SOAP envelope and "exposing" this as a contract to another system. This other system could pull in the SOAP object, extract the XML document from within it, then deserialise it into...a C# object! Not only is the impedance mismatch alive and well here too, there is another subtle assumption made that is violated in the execution. Can you find it?

Let me sum up.

RPC is evil because it makes developers think they can make a remote object appear local.

XML-RPC is doubly evil because it makes developers think (i) they can make a remote object appear local and (ii) it is possible to convert a Java object to XML and back without error or ambiguity.

SOAP-RPC is triply evil because it makes developers think (i) they can make a remote object appear local, (ii) it is possible to convert a Java (or C#) object to XML and back without error or ambiguity and (iii) even though the SOAP message travelling between two systems is now a "contract" to be honoured regardless of changes to implementations, it can still be generated from implementation classes, and implementation classes can be generated from it.

What's the truth, then?

Nothing glamorous or counter-intuitive.

1. A copy is a copy, not the original.
2. XML is XML and Java is Java (and C# is C#).
3. Contracts are First Class Entities, just like Domain Objects in an OO system.

SOAP-based Web Services technology today is sadder and wiser because it has internalised these truths. Because of Truth 1, we don't pretend anymore that a SOAP message is a Remote Procedure Call. We know it's just a message. Because of Truths 2 and 3, we don't believe anymore that we can generate XML documents from Java/C# objects, or Java/C# objects from XML documents. We can only map data between these two forms. If you build your Web Services abiding by these principles, you can stay out of trouble, otherwise you will be left wondering why your systems are so brittle.

(Actually, there are still people who don't understand these Truths, and who still go on about SOAP-RPC. That's what makes me despair about SOAP's chances for success.)

Book Review: "SOA and WS-BPEL" by Yuli Vasiliev

I received a SOA book for review today. It’s called “SOA and WS-BPEL” by Yuli Vasiliev. The publisher is Packt Publishing.

I’ve only had time to skim through the book once, but I think I’ve understood enough to be able to post some initial thoughts on it. As I read it in greater detail, I may refine this post into a proper review.

What I like about the book:

The choice of PHP as an implementation language: This is a refreshing reminder that SOA is about hiding service implementation details from service consumers. The implementation language doesn’t always have to be Java or C#. What matters is what the service consumer sees (i.e., SOAP). There is however, a downside with PHP, which I'll cover in a moment.

The use of Open Source technologies: I believe that Open Source is the way of the future, and the use of Apache/PHP, Tomcat and ActiveBPEL to illustrate SOA concepts feels just right. Besides, readers can readily try out the examples in the book without having to buy expensive commercial software. (An exception is Oracle, which the author discusses in some detail in the context of data-centric services. Perhaps it’s the Oracle bias he has by virtue of having worked extensively with Oracle technology in the past ;-).

Copious examples: This is not a theoretical book, and there is plenty here for the reader to try out for themselves. Every concept that the author deals with is represented in code. [I can’t comment on how complete or correct the examples are and whether they work, because I haven’t tried them out yet.]

Lots of diagrams: Pictures are really worth thousands of words, and there are diagrams sprinkled liberally throughout the book to illustrate almost every concept discussed. I thought they were quite decent.

Emphasis on data: One of my pet peeves with many people’s approach to SOA is their relative neglect of the Data Interchange view of service interactions. SOAP-based web services is about exchanging XML documents that represent some structured data relating to the operation being performed. There is a lot of design that needs to go into these XML documents. Thankfully, the author spends a fair amount of time showing how to design the data payload of messages with XML schema and converting data into XML format (but a nit about that bit later!) I also liked the treatment of the data within the contract (importing the schema file into the WSDL file instead of defining the schema in place).

Introduction to WS-Security and to Qualities of Service (QoS): The book has a section on implementing secure messaging using WS-Security, and also makes the point that virtually all the WS-* specifications use SOAP headers to implement functionality. There’s not too much here, but enough to get the developer to understand the WS-* approach.

View of a process as a service in its turn: One of the value propositions of SOA is its ability to exhibit a “flat” landscape of services, regardless of how they were implemented. From a service consumer’s point of view, it doesn’t matter if a service was purpose-built in a programming language (e.g., PHP) or stitched together out of other services (using BPEL). Services of both types should look the same. The book shows how composite services can also be exposed as services in their turn, with the appropriate WSDL sections highlighted.

WS-BPEL treatment at the right level of detail: I thought the level of discussion and the examples of WS-BPEL were just right for a beginner. There is enough detail to be meaningful, but not so much as to overwhelm.

What I don’t understand or don’t like in the book:

The title: SOA and WS-BPEL are like fruit and oranges, not even apples and oranges. The first is an architectural approach; the second is a language used to implement processes. Considering that the book deals with building standalone web services in the first part, then composing them into processes in the second, perhaps it should have been called “SOAP and WS-BPEL” or “SOA: SOAP and WS-BPEL”.

The unquestioning acceptance of the RPC view of Web Services: I have religious feelings about RPC. It is the devil’s spawn. Many of the REST camp’s arguments about SOAP are actually directed against SOAP-RPC. The modern view of SOAP-based Web Services is based on messaging. Messaging, not RPC.

What’s the difference, and why is this important? RPC is architecturally dishonest. It is impossible to make a remote object behave like a local one, and I don’t mean the effect of network latency. A reference to a local object that is passed to an application carries with it the promise that any change made by the application using the reference will change the object. But with RPC, what is passed to the application is not a reference to the object, but a reference to a copy of the object. This is not an insignificant difference. When the application makes a modification using the reference, the local copy is changed, not the remote object. But the application thinks the actual object has been changed. That’s what is so dishonest about it.

Messaging turns this essentially hopeless exercise around. It makes local objects look remote, by always passing copies around, even if the actual object is accessible by a reference. The application is under no illusion. It knows that in order to make a change to the real object, it is not enough to make a change to the copy. Either the copy must be passed back to be synchronised in some sense with the original, or an independent operation to pass a Data Transfer Object is required. This is architecturally honest and clean. What it may lose in efficiency in some corner cases (local access), it more than regains in terms of robustness, flexibility and scalability.

That’s what SOAP messaging brings to the table. SOAP-RPC is evil and should have nothing to do with a book on modern Web Services. It’s a pity the author actually mentioned RPC by name when introducing SOAP messaging, because the actual examples do not assume RPC.

In this context, the use of PHP has a downside, as I indicated earlier. PHP is heavily tied to HTTP and by extension, to synchronous request/response semantics. SOAP-based Web Services technology does not inherently have this constraint and can work with asynchronous transports as well. The book could have illustrated this effectively using a message queue example. There are tools such as PHPMQ and Mantaray that make this possible, as this example shows.

Automatic data transformation to and from XML: This is another of my pet peeves. The service contract (of which the XML document forms a part) is a First Class Entity. So are the classes that make up the internal Domain Model. How can one ever be generated from the other using a tool? Code generation is an example of tight coupling, and if two First Class Entities are tightly coupled, then at least one of them is not a First Class Entity. QED. I’m not sure if there is an equivalent to TopLink or JiBX in the PHP world, but such a mapping tool is what is required to transform data between the PHP and XML worlds. To be fair, this book is not alone in propagating the code generation approach. Virtually the entire Java Web Services industry is consumed by JAXB disease.

Data-centric Web Services – Actually, I didn’t understand the point of this as a separate topic. It’s a special case of service implementation. In fact, I have a totally different view of Data-Centric Web Services. I call them REST.

No high-level view of process as an aspect of the business: For a book that purports to be on SOA, the treatment of composite services and processes is surprisingly low-level and technology-oriented. There should have been an introduction that focused on business processes and their decomposition into services. In fact, SOA best practice is all about business process modelling and re-engineering. That’s how architects and business analysts determine the services that are required and their granularity. Proceeding bottom-up from services and composing them into processes, as the book seems to suggest is the way to do SOA, is ingenuous.

Anaemic index: The index of the book doesn't list many of the things discussed inside. I tried going back a couple of times to look up something I had seen earlier, but the index was of no help.

Other comments:

ActiveBPEL Designer is not an Open Source product, merely free. This isn’t the fault of the author. It’s just something I’m personally sad about. I haven’t yet found a truly Open Source BPEL designer that is powerful and friendly, and generates full-featured BPEL.

Overall comments:

Actually, notwithstanding the negative comments I made (I'm a nitpicker, as my wife will attest), this is a pretty decent book on Web Services (SOAP and WS-BPEL). It’s got enough low-level detail to help developers get their hands dirty and understand the technology by actually building services and processes. The choice of PHP could turn out to be a masterstroke by reaching beyond Java or C# developers and appealing to the vastly more populous LAMP community. Time will tell. I thought the book was a bit light on architectural insight, but maybe it’s for the best. For a developer audience, such discussion might just cause eyes to glaze over.

SOFEA and SOUI - There is a Difference, After All

Great minds think alike, or so it would appear :-).

Just a month after we published our article "Life above the Service Tier" describing SOFEA (Service-Oriented Front-End Architecture), Nolan Wright and Jeff Haynie have proposed an architecture they call SOUI (Service-Oriented User Interface). (Independently, Roger Voss blogged that web frameworks are peaking towards obsolescence.)

It's worth repeating what we said in the Conclusion section of our paper:

"Although it seems presumptuous on our part to claim that we have “solved” the end-to-end integration problem, what is probably true is that recent paradigms and technology breakthroughs have brought the SOFEA model closer to conceptualisation, and someone or the other was bound to suggest it. It happened to be us."

Well, clearly not just us. Many others are saying the same thing.

Matt Raible wondered if there was any difference between the two frameworks (SOFEA and SOUI). SOUI proponents Nolan and Jeff haven't just proposed a theoretical architecture, they've actually created a set of tools to help developers build to this model, and what's more, this set of tools is available for Java, PHP, Ruby and .NET. It's called Appcelerator, and it's a pretty impressive piece of work.

Examining Appcelerator (and it would be a fair assumption that this is what Nolan and Jeff intend SOUI to look like), it appears to satisfy all the conditions that we proposed for SOFEA, except one.

SOFEA emphasises the use of XML for Data Interchange, because one of its guiding principles is to mesh seamlessly with the service tier. Services may be built using either the REST paradigm or the SOAP messaging paradigm, but XML plays a big role in either one. SOAP requires, and REST recommends, the use of XML for Data Interchange. XML has the essential characteristic of being able to enforce data integrity (i.e., conformance to specified data structures, data types and data constraints). That's why we make a big deal about XML support in SOFEA. We considered but rejected JSON because it's only slightly better than raw HTML-over-HTTP in these respects.

Appcelerator uses JSON, not XML. So that seems to be the big difference between the two models. SOFEA requires the use of XML for Data Interchange. SOUI prefers the more lightweight JSON.

I guess Nolan and Jeff have made a valid architectural choice favouring ease-of-use over data integrity, but better XML tooling may blunt that advantage in future.

Incidentally, Ye Zhang made a comment about SOFEA on Matt Raible's blog to the effect that we seemed to be chasing buzzwords and hence tacked on the "Service-Oriented" prefix to our model. Not true. Service-Orientation is at the heart of our model, because we approached the Presentation Tier from that angle. Our emphasis on XML as the Data Interchange mechanism for SOFEA was designed to enable the Presentation Tier to interoperate with the Service Tier with no impedance mismatch at all.

At any rate, the software industry seems to be at the start of a new era, and I'm happy we were able to make a contribution to the debate.

SOA Technology Best Practice

Let me formalise what's on my mind regarding SOA best practice.

1. Build your core application using the principles of Domain-Driven Design. This is your service implementation. Hint: If using Java, Spring/JPA is the way to go. EJBsaurus is dead.
   
2. Take the help of an off-the-shelf domain model like IFW for Banking and IAA for Insurance. This will let you build incrementally without experiencing refactoring pain as your model grows.
3. If the way your clients refer to your services emphasises verbs over nouns ("Jelly this eel"), you may want to look at SOAP-based Web Services, which are operation-oriented services. If your clients emphasise nouns over verbs ("Buy a ticket to the Spice Girls concert"), consider REST-based Web Services, which are resource-oriented services.
4. Model your interface contract using XML schema in either case. Layer SOAP or REST Data Interchange patterns over this basic XML document foundation.
   
5. Follow Contract-First Design. Do not generate your service interface from your implementation classes, even if your vendor gives you cool tools to do so painlessly (Hint: the pain comes later, when every change to your implementation breaks your service contract).
6. Use a meet-in-the-middle mapping approach to connect the XML documents in the contract to the domain model. Do not generate implementation classes from the service interface. The self-important JAXB library is a bad bug that's going around. Use JiBX or TopLink instead.
   
7. When building the Presentation Tier of an application that consumes services, use the SOFEA model.
   
8. If you want to do content aggregation using 2007 thinking, consider a mashup. If still stuck in 2003, use a portal server and jump through the associated hoops.
9. You don't need an ESB product, not now, not ever. Learn to see the "bus" in the way services are built and invoked. Remember that the web with its smart endpoints has comprehensively beaten the telco network with its smart network approach, even in voice communications. If you must use an ESB, deploy it in a decentralised, "smart endpoints" configuration. Obviously, a decentralised ESB is economically viable only with an Open Source product . A commercial one will force you to deploy it in a centralised architecture to save on licence costs (blech!).
   
10. You don't need an ESB product to do process orchestration either. Use BPEL engines at endpoints. If you need human workflow, use BPEL4People.
    
11. Open Source is the way of the future. Wake up and smell the coffee. For SOAP-based Web Services, consider Sun's Metro (formerly Tango) SOAP stack. For REST, consider Sun's Jersey (Java-based) or SnapLogic (Python-based). For an ESB (decentralised, of course), consider ServiceMix/CeltiXFire/FUSE. For process orchestration, including human workflow, check out Intalio.
    
12. Management of your services can be centralised even with decentralised services. That's not an argument for a centralised ESB either.
13. Take vendor education with a pinch of salt. Talk to fellow practitioners, especially the bold and the irreverent.
    

A Java-filled Day in Sydney

Wednesday the 17th October, 2007.

Sydney had a surfeit of Java today. First, there was the all-day Sun Developer Day at the Wesley Theatre on Pitt Street. And then after that, the Sydney Java Users Group had a meeting at the same place where Mike Keith (of Oracle and the EJB3 spec committee) spoke about JPA.

Where do I begin? I guess my takeaways were that we shouldn't write off NetBeans or Glassfish yet. There was a time, I admit, when I was one of those who thought Sun should just give up on building its own IDE and app server. I may have been wrong. The upcoming NetBeans 6.0 seems quite cool, not least because they seem to have shamelessly stolen a lot of IntelliJ IDEA's cool features.

Glassfish doesn't seem bad either. It's no longer a toy. It's clusterable, for one. And Sun's Project Tango (interoperability with Microsoft's implementation of Web Services in .NET 3) is now part of Glassfish. That means the Java world gets an advanced Web Services platform absolutely free. That should count for something. And Glassfish v3 is supposed to start up in under a second, so it should really wow people when it debuts.

There was a great talk on building RESTful Web Services by Sun's Lee Chuk Munn, which has inspired me to try and download Jersey and give it a spin. Should I look at Phobos as well - a JavaScript engine for a RESTful server? Decisions, decisions.

Angela Caicedo was another Sun evangelist who spoke very well about new features in Java 6 and 7, also about building mobile applications using Java, AJAX and SVG. I'm just a bit disappointed that a mechanism I thought of but didn't widely publicise is now being used for exposing Web Services. The Java 5 concepts of Responses and Futures as a way to turn nouns into verbs and get systems to turn what should be standalone "methods" into classes (which are naturally first-class objects) is at the heart of this.

Glassfish is built on top of the java.nio.* package, which makes it very fast, I believe.

In the evening, I attended Mike Keith's talk on JPA. I'm pleasantly surprised to see that I'm not the only one advocating a mapped-data binding approach. Mike told us about Eclipse MOXy, which provides a "meet-in-the-middle" approach. I wonder if they're aware of JiBX.

Services, Persistence and a Rich Domain Model

I see so many wrong-headed ideas about SOA and Web Services that I want to scream.

Here are just some:

1. Using the RPC style of SOAP Web Services (Wake up, Rip Van Winkle, it's 2007!)
2. Thinking that buying an Integration Broker or ESB will itself solve your integration problems (Sure, your shiny telephone lets you dial any international phone number, but does the person at the other end speak your language?)
3. Generating WSDL files and Web Service endpoints automatically from Java classes (So what happens to your service contract when you next change some aspect of your implementation?)

It's the last of these points that I want to talk about in this post.

On the face of it, the problem can be stated in a deceptively simple manner, but there are significant subtleties to it.

The application you are building uses (say) Java (it could be using C# and .NET for all we care). The way we have decided to interact with other systems is through Web Services. That means SOAP messages will be exchanged between our systems. Since the document/literal style of SOAP-based messaging requires an XML document to be embedded within the SOAP body, we can see that the service contract is basically a set of XML documents.

So our problem can be simplistically stated as follows:
How do we convert between Java objects used within our system and XML documents exchanged with other systems?

There are two "obvious" answers to this question - either generate the XML document(s) from your Java classes, or generate Java classes from the XML documents. Java IDEs tend to take the former approach. In fact, many of them go a step further by not only generating XML documents but SOAP endpoints as well, and WSDL definitions of the SOAP endpoints that you can conveniently distribute to your client systems. As I suggested earlier, this approach is wrong because it breaks the service contract (by generating a new WSDL file) whenever the Java classes or method signatures change.

There are also a bunch of tools that take the latter approach. Apache's ADB, XMLBeans and the ubiquitous JAXB library that comes bundled with your JDK all try and generate their versions of a set of Java classes that they believe represent your XML document. Looking at the resultant code can make you lose your appetite very quickly. I'm surprised how so many people gamely continue even after looking at the results of what is very patently a wrong approach.

(This is a blog. Bloggers are meant to be opinionated.)

I believe I know how the problem should really be tackled. Both answers above are wrong. Neither the XML document nor the Java classes must be generated from the other. The Service Contract (represented by the XML document) and the Domain Model (represented by the Java classes) are both First Class Entities. By this, I mean that both are concepts that stand independently. The Service Contract is something that is negotiated between service provider and consumer. It must have no implied dependencies upon how the service is implemented. The Domain Model, on the other hand, represents the designer's deep understanding of the business application and how it really works. This should be independent of how other systems may want to interact with it.

And so my answer to the problem is that we need a third entity, a mapping entity, that sits outside both the Service Contract and the Domain Model and maps them to each other in as flexible and bi-directional a manner as possible. We have such tools today. Their names are Castor and JiBX.

The tragedy of the Java-XML translation space today is not that most practitioners seem to think it is a solved problem. It is. The tragedy is that they think the solution is JAXB. The correct answer is JiBX, and they get no points for getting the letters a bit jumbled. (Think mapping, not code generation.)

Mapped data binding that respects the independence of the two entities it maps is the correct solution approach to Java-XML translation.

There is a second reason why mapped data binding works so well, and it has to do with what is called an impedance mismatch. Impedance mismatch is a term borrowed from Electrical Engineering, and refers to the difficulty of translating concepts from one paradigm to another. Classes in Object-Oriented systems and tables in Relational Databases have an impedance mismatch. Classes and XML schemas have an impedance mismatch.

The Object-Relational impedance mismatch has been satisfactorily overcome for a few years now by ORM (Object-Relational Mapping) tools. Hibernate, Toplink JPA and OpenJPA (formerly Kodo) come to mind. You will notice a similar approach with these tools to what I suggested was the solution to Java-XML translation. They respect both paradigms and can map existing instances of each using mapping files that sit outside both.

So overcoming the Java-XML impedence mismatch is a second good reason to use a mapped data binding approach like JiBX or Castor.

[I'm told that JiBX makes the use of XML as a data interchange mechanism as fast as native RMI. I just love it when the right way to do something is also made attractive.]

My third and final way of looking at the problem is from the traditional Java view of Domain Objects and Data Transfer Objects. Data Transfer Objects have never been considered true objects because they only encapsulate state, not behaviour. However, they serve a useful purpose as a data interchange mechanism. It's considered bad practice (except by the Hibernate crowd and their queer notion of detached entities) to directly expose your Domain Objects to the outside world. You should decouple your implementation from your service interface. This decoupling is done by using Data Transfer Objects (DTOs) as very specialised data structures visible to other systems through the service interface. DTOs are marshalled from Domain Objects and unmarshalled back to Domain Objects during service interactions. This is a paradigm well known to most J2EE developers. Looked at in this way, XML documents used in Web Services are just DTOs.

The best technology to marshal and unmarshal Java DTOs is Dozer, just as the best technology to marshal and unmarshal XML DTOs is JiBX.

I see a beautiful symmetry in the way all these tools and technologies inter-relate. At the centre is the rich Domain Model that implements your application. To persist the application's state, you use Hibernate or another ORM tool to map your Domain Objects to relational tables. To talk to remote Java clients, you map your Domain Objects to DTOs using Dozer. To expose services to clients that require a Web Services interface, you use JiBX or Castor to map your Domain Objects to XML documents.

This picture should be worth the thousand words above.

Who is the best Vendor of Collaboration Technology?

Collaboration is a hot topic in the industry today, and every organisation wants to set up an advanced collaboration capability for their employees and customers. My personal view is that collaboration has more to do with culture than technology. So if an organisation's culture is not inherently collaborative, merely buying collaboration technology will leave a hole in their budgets but not actually achieve very much.

But catch vendors telling customers that.

Listening to them tell it, the key to success in collaboration capability seems to be in implementing the most integrated "stack" of products you can find. The theory goes that if you find a vendor with a rich and integrated stack of "collaboration" products, buying and deploying that stack will lead to collaboration nirvana (canned laughter here).

It's funny enough seeing stodgy, hierarchical organisations trying to acquire collaboration capability without becoming any less stodgy or hierarchical, but just look at who the vendors are!

You shouldn't be surprised to find that the "leaders" in the collaboration space (self-styled or anointed as such by analyst firms with backward-facing crystal balls) are large, stodgy and hierarchical software vendors. Our usual suspects IBM and Microsoft lead the pack. Blind leading the blind is the expression that leaps to mind.

Face it, buying collaboration technology from large software vendors is like importing voting machines from dictatorships. What do these guys know about this stuff anyway? They wouldn't recognise it if it bit them in a sensitive area. I bet they wouldn't be prepared to face its implications in their own organisations/countries, but they don't seem to mind pushing it onto others. (OK, that's a bit unfair, because many technical people in companies like IBM and Microsoft do collaborate quite effectively, but I'm talking about a larger organisation-wide culture.)

I believe that organisations that want to make a success of collaboration must (1) ensure first that their cultures are collaborative and (2) source their collaboration technology from collaborative communities, e.g., the Open Source community.

Collaboration is what Open Source communities do all the time. Newsgroups and mailing lists, IRC, blogs and wikis, RSS and Atom, mashups, -- all invented by the necessity of collaboration across far-flung communities of peers. These products and technologies are truly collaborative because the people that built them use them for true collaboration, not the sanitised, managed and controlled collaboration that corporate bosses have in mind.

And the "stack" theory is my pet hate. In my evolved view (and I say this without vanity), I believe that the best integrated products are those that were not built as a "family of products" by a single entity but those that were built by independent groups that operated in a decoupled way but had no hidden agendas about locking out their competitors. That's why you can run a Linux desktop with a Firefox browser, a MySQL database and your own local Apache web server running PHP, and build applications using this suite of products without having to think about where they came from. The Linux community, the Mozilla Foundation, the Apache Software Foundation, the PHP community, the MySQL community and MySQL AB have all collaborated (yes, that's the word) to deliver to you, the developer, a seamlessly integrated set of products. Is it a monolithic "stack"? Nonsense. The myth that software products need a common brand to be interoperable is just that - a myth. Open Source software groups don't have ulterior objectives of locking each other out. That's why interoperability happens, - because users want it.

Will decision makers get this? I believe it's an ongoing process of learning and maturity. Hopefully the presence of Generation Y in the workforce will teach the rest of us how to really collaborate. And put away those chequebooks, because the best things in life really are free.

Life above the Service Tier

Or "How to Build Application Front-ends in a Service-Oriented World"

(This is a paper written by myself with two co-authors, Rajat Taneja and Vikrant Todankar, the same team that critiqued the EJB 3.0 spec in 2004.

You can download and read the paper and an accompanying presentation.

Update Sep 2008: How SOFEA fits into a SOA Ecosystem)

But let me talk a bit about it here.

When we look at the Business Logic Tier, although the industry hasn't reached nirvana yet, the story seems to be coming together rather well. The Service-Oriented Architecture (SOA) message is being heard, with its concepts of loose coupling between systems, removal of implicit dependencies and specification of all legitimate dependencies as explicit contracts. The SOA model yields flexibility and uniformity through the specification of a definite Service Interface, which hides domain models and implementation technologies.

Unfortunately, no such clear architectural blueprint exists for the Presentation Tier that lies just above the Service Interface. There are a dozen different technologies available here. There are also the thin-client and rich-client models to choose from. Which of these is the "right" one to use? What does industry best practice say about developing the front-end of an application? What is the best way to connect the front-end neatly into the Service Interface? Developers are confused and left without guidance.

And while on the subject of connecting the front-end to the Service tier, an additional source of confusion is the fact that the Service Tier may itself follow one of two different models - SOAP-based Web Services or REST. Which one should we target? We believe that both these models will continue to co-exist, and organisations will have legacy business services in both these models. Therefore a good Presentation Tier technology will be capable of interfacing with both SOAP and REST, not just one of them. It is important to note that both SOAP and REST use (or are capable of using) XML documents for data interchange.

When we put on our architects' hats and look at the Presentation Tier, we see that regardless of whether it is a thin client or a rich client, three essential processes always take place.

The first is what we call Application Download. In order to be able to run an application on a client device, it needs to be delivered to the device in some fashion.

When the application is in use, a sequence of screens is typically seen. We call this Presentation Flow.

Finally, any non-trivial application ultimately exchanges data with some back-end resource, a process we call Data Interchange. This is usually the crux of the application's functionality.

To reiterate, the three processes occurring in the Presentation Tier of any application are Application Download (AD), Presentation Flow (PF) and Data Interchange (DI).

In thin-client applications, the web server plays a part in all three processes. Application Download occurs piecemeal, in the form of individual web pages served up by the web server. The web server also drives the application's Presentation Flow, usually with the help of a web framework like Struts or Spring MVC. Finally, the web server acts as an intermediary for Data Interchange between browser and application server.

We believe that the thin-client model as described above suffers from at least three major architectural flaws. Unless these flaws are addressed, we cannot arrive at a clean architectural blueprint for the Presentation Tier.

The first flaw is that the thin-client model does not respect data. Look at the data that goes from the browser to the web server. There is no data structure. It's just a set of name-value pairs. There are no data types. Everything is a string. And there are no data constraints. We can put any values into those name-value pairs.

Look at the data that comes back from the web server to the browser. It's highly presentation-oriented, marked-up data. The thin-client model does not respect data as data. We believe we can do much better than this in this day and age.

The second flaw is that the thin-client model tightly couples the mutually orthogonal processes of Presentation Flow and Data Interchange. It is not possible to move through the steps of the Presentation Flow without initiating what amounts to Data Interchange operations. Web pages are displayed in response to GET and POST requests sent by the browser. Even worse, every Data Interchange operation initiated by the browser willy-nilly forces a Presentation Flow. An infamous result of this tight coupling is the "browser back-button problem". When a user tries to step backwards page-by-page through the Presentation Flow, the browser re-does the Data Interchange operations that resulted in each of those pages. If any of those operations is a POST, it spells trouble, because POST operations are not safe or "idempotent". They have side-effects if re-done.

True, there are patterns such as POST-Redirect-GET that are used to work around this problem, but the inescapable fact is that the fundamental architecture is broken because of this tight coupling.

The third flaw is that the web model is request/response. It does not support peer-to-peer interaction styles that are required for server event notification.

At this juncture, one would be tempted to conclude that AJAX is the answer to these problems. Unfortunately, AJAX itself is just a raw capability and not a prescriptive model. It is possible to use AJAX and still come up with a horrible hybrid model where the web server continues to drive Presentation Flow in response to Data Interchange operations, and the AJAX interaction just hangs off to one side, so to speak.

Clearly, what we need is a prescriptive architectural model that explicitly delineates the dos and don'ts of application design.

We call this model SOFEA (Service-Oriented Front-End Architecture).

In the SOFEA model, there is an entity called a Download Server. This role may be performed by a standard web server, but its role is very strictly restricted to enabling Application Download. It stays out of the loop thereafter and does not play a role in Presentation Flow or Data Interchange. This is the most significant difference between the SOFEA model and the traditional thin-client model.

The application is downloaded onto an Application Container, which could be a browser, a Java Virtual Machine, a Flash runtime or even the native operating system itself. The nature of the Application Container does not matter. What matters is that there is an environment within which the application can run.

The application is built using a proper MVC architecture, not the Front Controller pattern used by traditional thin-client applications. The Controller drives Presentation Flow.

The Controller also asynchronously initiates Data Interchange with the Service Interface as required.

We stipulate that a SOFEA-conforming application will support both SOAP and REST-based Data Interchange through XML documents. This is the seamless interface that we envisage between the Presentation Tier and the Service Tier. These XML documents, being specified through a schema definition language, will enforce data structures, data types and data constraints at the point of capture and ensure data integrity end-to-end. The XML documents of the service interface correspond to a representation of the domain, or (to a Java programmer) Data Transfer Objects (DTOs). They're ideal for transferring data marshalled from domain objects or unmarshalled into domain objects. Thus, while XML documents are a connection point between the Presentation and Service Tiers, they also decouple the client from the domain model, which is as it should be.

For maximum flexibility, Data Interchange should support Peer-to-Peer interaction. A strict client/server delineation as in the case of traditional thin-clients is likely to be limiting.

The SOFEA model unifies the "thin" and "rich" client models. Indeed, these labels are meaningless when systems conform to the SOFEA model. The differences between technologies form a spectrum based on the application's footprint and its startup time. There is no sharp demarcation between thin and rich clients anymore.

The SOFEA model also highlights why there is a plethora of web frameworks. SOFEA explicitly repudiates Front Controller as an anti-pattern. Driving Presentation Flow from the web server is tied to a fundamental architectural flaw (the coupling to Data Interchange). Therefore no web framework can ever be satisfactory. Continued innovation in search of a "better" web framework is, we believe, completely misguided.

The SOFEA model can be implemented through any of the following modern technologies:

I DHTML/AJAX frameworks for Current Browsers

1. Handcoded with third party JavaScript libraries
2. Google Web Toolkit (GWT, GWT-Ext)
3. TIBCO General Interface Builder

II XML Dialects for Advanced Browsers

4. XForms and XHTML 2.0
5. Mozilla XUL
6. Microsoft SilverLight/XAML

III Java frameworks

7. Java WebStart (with/without Spring Rich Client)
8. JavaFX

IV Adobe Flash-based frameworks

9. Adobe Flex
10. OpenLaszlo

Obviously, different technologies will support, out of the box, different subsets of the SOFEA model. It's up to designers to consciously apply SOFEA principles to their application design, making the appropriate decisions with regard to their technology of choice.

Our contribution through the SOFEA model is the following:

1. A renewed emphasis on data integrity, which traditional thin-client technology does not and cannot enforce.
2. A cleaner architectural model that decouples the orthogonal concerns of Presentation Flow and Data Interchange.
3. Affirmation of MVC rather than Front Controller as the natural pattern to control Presentation Flow.
4. Unification of the thin-client and rich-client models, now seen as an artificial distinction.
5. Support for SOAP- and REST-based business services, and a natural integration point between the Presentation and Service Tiers.
6. Positioning the web server as a Download Server alone. The evils of web server involvement in Presentation Flow and Data Interchange are avoided.

We believe we have provided an architectural blueprint for the Presentation Tier, with clear dos and don'ts, while still supporting considerable diversity in technologies. Life above the Service Tier now lives by similar rules as life within it.

Tell us what you think.

New Microsoft "Open Source" licenses reveal a company on the defensive

PJ of Groklaw believes Microsoft is (as usual) up to no good by pushing two licenses for OSI approval. I don't trust Microsoft any more than she does, but I don't share her seeming paranoia that every move the company makes is a deviously cunning ploy to destroy its enemies. Many of its recent moves, I believe, are death throes, nothing more than a pathetic and desperate attempt to simply survive.

What I think is behind Microsoft's current desperation is the release of the GNU General Public License version 3. Mind you, GPLv3 is the most discussed and collaboratively developed software license in history. It has an impeccable set of guiding principles, accommodation of the widest variety of views across the spectrum and (not least) watertight legalese that gives it teeth in a court of law.

Regrettably, Linus Torvalds isn't the sharpest knife in the Open Source drawer at the moment, but plenty of others see the tremendous advantages of GPLv3 and are releasing serious software under this license. Samba, OpenSolaris and SugarCRM are just the initial trickle. The flood is coming. Already, the GPL (version 2) is the most popular FOSS license, accounting for about 62% of all software on SourceForge. The GPLv3, being more modern and having carried along all interested parties in its drafting, will surely be more popular.

It will also put a stop to Microsoft's patent shell game, the last refuge of those unable to compete. Checkmate.

That's why these two licenses are being trotted out by the monopolist now. Any license at all will do, as long as it's not the GPLv3. I'm not a lawyer, but I'm betting that even if the licenses turn out to have impeccable Open Source credentials from OSI's point of view, they will turn out to be incompatible with GPLv3.

Because that's what these licenses are being rushed out to do - slow down the adoption of GPLv3.

Fat chance. The GPL version 3 is a juggernaut. And Microsoft is roadkill.

The awesome, benign influence of MIT

If that's what being a Massachusetts liberal means, let's have more of it!

MIT has always been respected as an institution of excellence in the teaching of technology. Gaining admission into its hallowed portals has never been easy. But the Massachusetts Institute of Technology can hardly be called elitist.

Three of the things MIT has done in recent times have been monumental gestures towards spreading knowledge beyond the privileged classes.

Exhibit 1: In 2002, MIT began to make its coveted course material freely available online, putting it within reach of anyone with an Internet connection and a willingness to learn. By the end of 2007, it is estimated that 1800 courses across 35 subject areas will be available. More importantly, MIT has created a trend in the OCW (OpenCourseWare) area, with more than a hundred other Universities around the world creating similar sites with online content to help independent learners.

Exhibit 2: Nicholas Negroponte of MIT's Media Lab has become world-famous for his sponsorship of the One Laptop Per Child program, an ambitious education scheme designed to bring computing power (in the form of $100 laptops) to millions of schoolchildren in developing countries. While the outcome of that project itself is far from certain today (thanks in part to the desire of parties not involved in it to sabotage it), I believe OLPC will (directly or indirectly) have far-reaching effects on the economics of computing, making computer hardware an order of magnitude more affordable and sparking off a fresh wave of IT innovation.

Exhibit 3: In May this year, MIT's Media Lab (yes the same outfit that Negroponte hails from) released Scratch, a simple, intuitive, visual programming language for kids. Learning computer programming has never been so easy or fun. Even adults will be hooked by its power and ease of learning. Nominally Open Source (but currently only accepting in-house contributions), Scratch promises to bring Computer Science concepts from University to primary school. As a programmer myself, I'm amazed to see object-oriented, event-driven systems presented so intuitively. It took me many painful months to learn OO concepts, but to teenage Scratch programmers, it's the most natural way to think about programming.

Hats off to MIT. And to conservative critics who look down on Massachusetts as a hotbed of liberalism, the guys at MIT aren't communists. Making education widely accessible isn't communism. It's how market capitalism is seeded.

Thank you, Microsoft

I don't need to add to the unanimous chorus that Microsoft's latest sabre-rattling (on Open Source's alleged infringements of 235 patents) isn't going to go anywhere. I agree that if anything, this campaign is going to sink far faster than the SCO affair.

However, I believe that Microsoft's FUD campaign is going to achieve a few things that would have taken far longer in the normal course, and I thank the company for these:

1. Accelerating the trend towards the GPL version 3 - Since it's now obvious to all (including Novell) that Microsoft's deal with Novell was just intended to set up an example of a "good" patent licensee to initiate the shakedown process that has now been inaugurated, there's a great deal more appreciation of the value of the GPLv3, which will prevent this kind of shady deal in future. If in the process, this license can kill DRM as well, that would be just what the doctor ordered.

2. Increasing the legitimacy/credibility of Open Source in the eyes of corporates - Even the dumbest PHB can see that Linux and other Open Source software must be hurting Microsoft badly to provoke such a high-profile attack. Ergo, there must be something there that's worth looking into.

3. Increasing the probability that software patents will vanish from the landscape, one way or the other - whether by convincing more voters (and their lawmakers) that software patents are a bad idea, or by sparking a "patent armageddon" through counter-strikes by IBM, the Open Invention Network (OIN) or others, or by galvanising a grassroots movement to seek out and invalidate every trivial, obvious or bogus patent, Microsoft must receive due credit for the coming end of software patents.

4. Hastening their own demise - I have maintained for many years now that Microsoft is a dead man walking. A company with only two main products (Windows Client and Office) that at one time contributed over 80% of the company's revenues and profit, with both products under threat from free rivals, has only so much time left before its growth slows and reverses. Once the Wall Street vultures catch on and start banging the drums about Microsoft's bleak prospects, the share price will plummet to the high single digits, shareholders and employees will head for the exits, and it'll be all over but the shouting. The company may go private and shrink to a black dwarf. The company's latest action may or may not have convinced anyone that Open Source violates their patents, but it sure has convinced a lot of people that Microsoft has run out of ways to compete with Open Source, and it's time to look elsewhere. Also, you can push spineless, risk-averse corporate types only so far before they turn and begin to offer resistance. I think the line has just been crossed in boardrooms, from "Open Source is too risky" to "Who do these guys think they're threatening?"

5. Increasing interest in Dell machines with pre-installed Linux - The publicity, negative sentiment towards Microsoft and the new-found cameraderie among technology users (who are being uniformly threatened with litigation) should increase interest in Dell's latest Linux-based offerings. Ubuntu is a worthy Linux distribution, and if it manages to hold up well under rough end-user handling, it should reach a tipping point very fast. Once again, thanks to Microsoft for the free publicity.

So it seems to be true that it's an ill wind that blows nobody any good. Microsoft may have done the world the biggest favour possible - hasten the advent of Open Source.

Twenty Years in the IT Industry

It's May 4, 2007.

I started my IT career on May 4, 1987 - twenty years ago to the day. That was the day I joined CMC Ltd as a programmer. (CMC is an Indian IT company, formerly government-owned, now with a large stake held by Tata Consultancy Services).

I'm still trying to relive the last 20 years. They seem to have gone by in a rush. It certainly doesn't seem like 20 years to me. When I get my thoughts together and can say something suitably philosophical, I'll add to this blog entry.

I dance on DRM's grave

It's happened, and sooner than I dared hope. The hated Digital Restrictions Management (DRM) technology that its backers barefacedly called Digital Rights Management, is dead on arrival.

Blatantly gouging customers requires a watertight cartel, and thankfully, the music industry isn't one. EMI, the smallest of the Big Four record labels, had the most to gain by breaking ranks. It had a willing partner in Apple. Together, they made history, and history will thank them for it. I know, they're rats, but they're our rats :-).

Now where does this leave Microsoft? The company spent years and megabytes encumbering its Vista operating system with DRM handcuffs to tie down users, and now the anti-consumer alliance is unravelling even before Vista has had a chance to gain traction. There must be chairs being thrown in Redmond at the news.

In the months to come, we will surely read detailed post-mortems about how a technology that had literally billions of dollars invested in it finally came a cropper. If the powerful backers of DRM are anything like the Colombian drug cartels, they'll target (Apple and EMI CEOs) Steve Jobs and Eric Nicoli for scoring a costly own goal, like poor Andrés Escobar did.

But for now, I'm celebrating. If the would-be-laughable-if-not-so-dangerous software patents are invalidated next, my cup of joy will truly run over...

We need a portal standard, not just a portlet standard

As I wrote to Punit Pandey (who runs this excellent blog on portals), I wonder at the fact that although so much work has been done on defining portlet standards, we do not have anything like a portal standard. A large part of a portal/portlet application has to do with menu design, page layout, portal themes, portlet skins, etc. These are not covered by the portlet spec and are left to individual portal servers to implement in their own (read: proprietary) ways.

Quite obviously, this impacts the portability of applications. I can trivially port a web application from one server product to another by just deploying a .war file, but I cannot do the same for a portal application. I would need to rebuild the portal pages from scratch and can only reuse the portlets in it. I can't see the portal vendors wringing their hands over this lack of portability. That's how they lock in their customers even while claiming to be standards-compliant.

I believe we need a portal standard in addition to portlet standards JSR 168, JSR 286 and WSRP. But we can't depend on vendors to kickstart the process. It's the users who need to set up a JSR to hammer out a portal standard, because we're the ones who lose from the absence of one.

The worst time in history to choose Microsoft

I've just heard of a company (which shall remain nameless) that has decided to move to Microsoft for their SharePoint collaboration stack, and that pretty much implies that they will be using Microsoft technology pervasively throughout their organisation.

Why, I asked my contact.

Because we've debated this for too long and we need to make a decision and move on, he said matter-of-factly.

I would normally accept this pragmatic wisdom, but I could not help raising an important point. Isn't this precisely the wrong time to be moving towards Microsoft, I asked, when the story in the non-Microsoft (read: Open Source) world has never looked better?

To pre-empt the usual "corporate" arguments against Open Source, I pointed out to him that much of the Open Source software seen today is in fact erstwhile commercial software that has been newly released as Open Source. Examples are OpenOffice.org (ex-StarOffice), Firefox (ex-Netscape), Fedora Directory Server (formerly Netscape Directory Server), Ingres, TerraCotta (clustering software), Eclipse, etc. What's more, the stream of goodies doesn't seem to be ending. Java itself will be Open Sourced this year. So will Sun's portal server. Why, so will Microsoft's FoxPro!

Third parties are rushing in to build integrated stacks from these excellent software products, and offering support at a price. Think Unisys and SpikeSource.

When so much proven software is "falling off the back of a truck", is this organisation's timing wrong in choosing this point in history to look elsewhere?

The software industry's very model seems to be evolving towards "free software, paid services". Shouldn't they be targetting this future instead of the old model of software license fees?

I don't know if my arguments will make any difference to this company, but articulating them has strengthened my conviction that this is the worst time in history to be choosing Microsoft, or any proprietary software vendor, for that matter.

It's like formulating business and/or IT strategy in 1994 without taking the Internet into account. Two years later, people would look at you and go, "What were you thinking?"

We can see the trend towards commercially-supported Open Source taking shape before our eyes today. If we lock ourselves into a multi-year licensing arrangement with Microsoft today, will we end up looking foolish tomorrow when the trend becomes the norm?

Sun should make a comeback this year

Jonathan Schwartz's elevation to CEO has been good for Sun so far, and I suspect it will prove to be even better in the days to come. In contrast to his predecessor Scott McNealy's on-again, off-again style (especially in relation to Open Source), Schwartz has brought a consistency and focus to Sun's actions. Finally, true to the words of a 1990s Sun ad, Sun once again has "all the wood behind one arrowhead."

The nature of the software industry is shifting, with Open Source gaining more legitimacy and services becoming the thing that customers pay for. Sun is moving to this model, too. In the process, they're beginning to give their competitors quite a few headaches.

You've read about Sun's Java and OpenSolaris plans. The latest news from Sun concerns their portal server. The portal space has been one where Open Source products have so far lagged behind their commercial rivals, mainly because of ease-of-use issues and imperfect integration with content management systems. It will take Open Source portals like Liferay and JBoss Portal more than a year to become "good enough". Sun has now weighed into this unequal battle, and will open-source their portal server. True, it hasn't happened yet, but the announcement has been made, and when the code is cleaned up, the product will be released. I'm hoping it will be under the GPL3.

Sun has a bit of housekeeping and tidying up to do before it can really boast of a full, consistent stack of Open Source products, but I'm confident it will get there.

On the client side, they really need to give OpenOffice.org a concerted push to get it over the line. OpenOffice.org has been at version 2.1 for ages now with no apparent progress. Its interface stills lags Microsoft Office and its performance and memory efficiency are features only a mother could love. No one but Sun can fix this, and they really need to do this quickly. There are many supporters of OpenOffice.org, but support could quickly evaporate unless the product performs.

On the server side, Sun should concentrate on two things:

1. Deprive their competitors of oxygen by providing full and credible stacks for most enterprise components, under a consistent licensing scheme (I'd recommend GPL3). That means GNU/OpenSolaris, Java, the GlassFish app server, Tango (the Microsoft-interoperable Web Services stack), Portal server and third-party Open Source software like Fedora Directory Server and Ingres. (My pet peeve: Forget the dalliance with PostgreSQL and get serious about a real database - Ingres. That's a GPL-ed product too, by the way.)

2. Build up a much stronger services arm or tie up with a dedicated services company (à la Accenture but with morals) and spend serious marketing dollars to get business.

Sun has a breathtaking opportunity to regain the initiative this year with bold moves that their competition cannot readily respond to. IBM, HP, Microsoft and Dell, all can be forced onto the back foot if Jonathan Schwartz plays his cards right.

Let me quote an old Irish blessing:

"May you always have work for your hands to do

May your pockets hold always a coin or two

May the Sun <grin> shine bright on your windowpane

May the rainbow be certain to follow each rain."

I'm waiting, and watching, and hoping.

Disclosure: I don't own Sun shares. I own SGI shares, and SGI has been delisted from the NYSE. Just my luck.

Desktop Predictions for 2007-2008

This has been stewing in my mind for awhile, and now the picture has crystallised. Here's what I predict will happen in the desktop market over the next two years:

1. Apple will license Mac OS X to OEMs this year (2007) while Vista sales are still sluggish.
2. Mac OS X sales will surge wildly and end up accounting for up to 25% of all new PC shipments by the end of 2007.
3. Emboldened by Apple's success, Sun will push GNU/OpenSolaris to corporations in 2008. The system will probably be called "The Liberty Desktop".
4. Linux will be seen as "ready for the desktop" once desktop diversity becomes the norm. Linux will dominate the low-cost desktop segment thereafter.

I don't believe Apple's management is living under a rock. I'm sure Steve Jobs is planning his next big smug announcement already. It'll be the news event of the year, but you read it here first. After years of tight control of its OS, Apple will finally realise the Zen paradox of gaining by letting go.

After all, if pre-installed Linux is top of the wish list for Dell's customers, what do you think the demand for pre-installed Mac OS X will be, once customers realise that's a real possibility? Midnight queues for Mac OS X will beat those seen for Windows 95, mark my words.

This is Apple's big chance. Windows as a desktop platform has plateaued in features, and Vista relies more on momentum and lack of alternatives than any compelling reasons to upgrade. There is a vacuum here, a hunger, a latent demand for something different and good. The OEMs know it too, and have been pushing Apple for this for years. Well, this year, they may actually get it. (If Apple is smart, the OEM price won't exceed $50. You need to seed the market, guys).

Once Mac OS X for PCs hits the shelves, the resulting media noise will drown out all other events for the year, including significant ones like the GPL releases of Java and OpenSolaris, the release of the next generation of Linux distributions (with Ubuntu-Linspire's upgrade advances) and further decreases in the price of hardware. But make no mistake about it, these events will impact the desktop market in 2008.

Sun is not sitting idly by. CEO Jonathan Schwartz has exceeded my personal expectations and is proving to be far more savvy than his predecessor, Scott "dinosaur" McNealy. I'm sure he sees the potential for Sun's resurgence, and Apple will open some doors for Sun.

You see, although Apple will take the consumer desktop market by storm in 2007 with freely licensed Mac OS X, the corporate market is deeply suspicious of Apple. Apple does not have a great reputation for reliability as a corporate supplier, having proved too fickle in the past and left enterprise customers in the lurch. The Apple desktop revolution will stop at the gates of the enterprise.

That's when I predict Sun will make its move. It will also coincide with the technical maturing of GNU/OpenSolaris as a desktop OS. Watch for Sun to exploit its links with Java, thin clients, SmartCard technology and the Liberty identity management system to push a compelling, low unit cost desktop solution to enterprises. GNU/OpenSolaris will ironically be running on virtualised servers, but visible to users through low-cost thin clients. SmartCards will provide roaming capability (among other things) and Liberty will provide a federated identity system. In my professional opinion, such a system would be architecturally elegant and also easy to justify from an infrastructure investment point of view. Compared to the hardware upgrades required to roll out Vista, this would be a far cheaper option.

For Sun, this solution would really be the Java Desktop System 2.0, but something tells me that "The Liberty Desktop" would go down better. I think the whiz kids in Marketing will finally call it that.

What does all this mean for our own dear Linux? (It's our own because we all own it, silly, even if you don't believe it. Think inclusive ownership.) Will the long-awaited Year of the Linux Desktop pass it by for a bunch of Johnny-come-latelies?

I believe Linux as a desktop OS has been asymptotically approaching readiness for years now, and its capability gaps have all but vanished. At this level of granularity, there are already market segments for which desktop Linux is a perfectly serious option. The major factor holding back mainstream Linux adoption is psychological. Nobody uses Linux on their desktop because nobody else does (Geeks don't count).

But when your PC salesman begins to ask you which OS you'd like with your new PC (Windows or Mac OS X), and you see the price of the OS as a separate line item on your bill, and compare that to the cost of the various hardware components, that's when you would start to ask about cheaper alternatives. The friendlier salesmen would point you in Linux's direction, even if their employers don't offer support for it.

And that's where I believe Linux will find its sweet spot -- the price-sensitive low end of the desktop market. That is also the volume market, so after 2008, the desktop market will have four major players:

1. Windows (still dominant in terms of installed base, but perceptibly losing share)
2. Mac OS X (the media darling with 25% market share of new shipments and the "coolest" OS to own)
3. GNU/OpenSolaris (whatever it's actually called, making strong inroads into corporates)
4. Linux (at the low end, steadily growing, and as always, with adoption impossible to estimate because it's all under the radar)

Diversity, I salute you.

The OS for which I'd ditch Linux

Yes, you heard that right. There is an OS out there on the horizon for which I would consider leaving Linux.

I've been a Linux loyalist since 1996. It was the answer to my prayers. Brought up on midrange minicomputers, I wanted Unix on my desktop as well, but I wanted it cheap, and I wanted it to make moral sense. To be precise, anything closed or proprietary has always been anathema to me, and that's why I dislike the Mac (closed software on closed hardware) even more than the Windows PC (closed software on open hardware).

Linux has been my darling for more than a decade now, but I now realise it's not Linux I love but the freedom that it guarantees and protects. And there's something around the corner that promises greater protection of my freedom than even Linux.

Version 3 of the GPL is undergoing its finishing touches as I write, and I believe it deals elegantly with two of the latest and greatest threats to user freedom - Digital Rights Management and Software Patents. One would think the next version of the Linux kernel would adopt GPL3, right?

Wrong, apparently. Linus Torvalds, who earned deserved acclaim as a software developer extraordinaire, has demonstrably reached his level of incompetence as a visionary by spurning and condemning this new version of the GPL. [For proof that Linus has this breathtaking blind spot when it comes to the importance of software freedom, one has only to look back at the Bitkeeper episode that left him with egg dripping from his face.]

Ah, but Linux isn't the only Free kernel in town anymore. There's Solaris, and Sun is reportedly looking very seriously at GPL3 for both OpenSolaris and Java. For the FSF, this is unexpectedly good news (if they can look beyond their egos, that is). A major software player (and two major pieces of software) are about to validate their latest license. And OpenSolaris can be the GNU kernel that the Hurd has never been able to become. But of course, as I said, they first have to be able to swallow their pride and quietly set aside the Hurd to adopt OpenSolaris.

Once that's done, we'll have a GPL3-fortified Free kernel running all the GNU and third-party FOSS software we know and love. Technically and legally, it'll be far less vulnerable to competitive attack.

I'm going to use this beast when it comes out. Even though it won't be called Linux.

Vista - The Last Microsoft OS?

OK, that title was deliberately provocative. But my point is very simple. Ubuntu and Fedora have convincingly demonstrated that users don't have to wait months and years to upgrade their operating systems. They can incrementally upgrade parts of their system every single day! My Ubuntu machine shows me a bright orange icon at the top of the screen every now and then to announce newly available versions of all the software I have installed. I have just to click on the icon, examine the list of new application versions, and choose the ones I want to install/upgrade. In minutes, my machine has been upgraded without even a reboot! A lucky few enjoy that luxury today, but with almost universal broadband in the very near future, that will rapidly become the preferred way to upgrade operating system software, - continuously, a little bit at a time. Linux users will always have the most up-to-date version of their distribution. Can users of any other OS say that?

Microsoft is notoriously quick to copy winning features from its competitors, so watch for the company to switch to a fully subscription-based model soon. Pay Microsoft an annual fee, and have a steady stream of incremental upgrades pushed to your computer. No more waiting for years and years for the next version. That move will be the end of monolithic, full versions of Windows, each with a unique name.

And that's why I think Vista will probably be the last Microsoft OS.