The Return of JavaScript - My Thoughts Crystallise

I wrote some months ago that JavaScript was back. I've been watching developments in that space with close interest, and a few glimmerings of "the right way forward" have begun to suggest themselves to me.

The first aspect pertains to the language itself. The JavaScript Renaissance has brought with it a desire to do things right the second time around. JavaScript doyen Douglas Crockford has lectured extensively (and even written a book) on the need to use only "the good parts" of JavaScript and avoid the bad ones. It's excellent advice, but to a new generation of JavaScript programmers, I suspect it may not be enough to post warning signs around bad features. We need a way to make it physically impossible to use JavaScript's "bad parts".

Enter CoffeeScript. While I haven't played with it too much, I like what I see here. [Update 24/09/2011 - I *have* played with it for a few hours now, and I really like!] There has been some skepticism about CoffeeScript from a section of the JavaScript old guard, but that's to be expected. The best answer to such skepticism is from ThoughtWorks's latest Technology Radar document, which is well worth a read in itself.

First, even if it seems a trifle off-topic, read what Thoughtworks thinks about GWT, which is a way to write visual components in Java and have JavaScript-based widgets automatically generated for a web application that runs within a browser:

GWT is a reasonable implementation of a poor architectural choice. GWT attempts to hide many of the details of the web as a platform by creating desktop metaphors in Java and generating JavaScript code to implement them. First, in many ways, JavaScript is more powerful and expressive than Java, so we suspect that the generation is going in the wrong direction. Secondly, it is impossible to hide a complex abstraction difference like that from event-driven desktop to stateless-web without leaky abstraction headaches eventually popping up. Third, it suffers from the same shortcomings of many elaborate frameworks, where building simple, aligned applications is quick and easy, building more sophisticated but not supported functionality is possible but difficult, and building the level of sophistication required by any non-trivial application becomes either impossible or so difficult it isn’t reasonable.

Now read what Thoughtworks says about CoffeeScript:

Some readers may be confused by our advocacy of Coffeescript given our general dislike for GWT, because on the surface they seem similar: tools that generate JavaScript. However, it is the level of abstraction that differs. GWT has an elaborate component model, which tries to hide details about the underlying language (JavaScript) and platform (the web). Coffeescript tries to make it easier to write proper JavaScript, avoiding pathological but default “features” of JavaScript, and does not build a layer that tries to insulate you from the platform.

I think they've nailed it. I have been ambivalent about GWT for a while, but couldn't quite put my finger on why (i.e., it's a "reasonable implementation of a poor architectural choice"). They've also very cogently argued why CoffeeScript doesn't fall into that category.

So my first takeway from the JavaScript Renaissance is that today's developers should code in CoffeeScript and not in JavaScript itself. If Java and Scala are two languages that compile to Java bytecode and run on the JVM, then think of JavaScript and CoffeeScript as two languages that "compile to JavaScript" and run within a JavaScript engine. The runtime behaviour is unchanged, but you use the language that works better for you as a developer.

CoffeeScript is JavaScript with the sharp edges removed. I'm sold. [There is an issue with E4X, which I'll cover when I talk about Nodejs, but it applies equally to CoffeeScript. In short, there's no support for E4X, and I want it!]

Beyond the choice of language, the direction is not so straightforwardly clear. It's tempting to say that the future of JavaScript on the client is CoffeeScript plus jQuery, and the future of JavaScript on the server is CoffeeScript plus Nodejs. But there are some issues.

To paraphrase Thoughtworks's critique of GWT, jQuery is an elegant way of working with a horrible object model (the DOM). I don't know if there is any way to replace the DOM with a better client-side abstraction, but until then, the best of client-side goodness will remain somewhat unsatisfactory.

Nodejs is the server-side angel in white from all accounts, but I've heard some valid grumbling about Node as well. The Spanish outfit ASPL has a BEEP implementation called jsVortex. This JavaScript-based server pre-dates Nodejs, and ASPL was reportedly keen to explore Node as the underlying platform for the next version of jsVortex. However, they quickly realised that Node had no support for TLS at its core. This is not the same as supporting HTTPS, because HTTPS support obviously exists in Node. Think of TLS as HTTPS-like capability for any protocol, and since BEEP is a framework to create protocols à la carte, and since security is a ubiquitous and pretty strong requirement for most protocols, the lack of TLS support within Node is a serious deficiency for them and for anyone who wants BEEP on the Nodejs platform.

[Most people might go "BEEP who?", but in my opinion, this is a technology that was sadly delayed by over a decade, perhaps because of personality politics caused by creator Marshall Rose's less-than-smooth interactions with powerful members of standards bodies, who then blocked BEEP from becoming a standard until very recently. If we'd had BEEP 15 years ago, I suspect SOAP and WS-* would have looked very different, and perhaps REST would have looked different too. That's how powerful it is, and I still think it will change our lives one day.]

Well, so there's a piece of work to be done to enable TLS within the core of Nodejs, but that's not all.

WSO2's Mashup Server was another under-appreciated piece of server-side JavaScript from many years ago, and one of its best features was its support for E4X. It's understandable why WSO2 would be keen on E4X, because their enterprise middleware suite is based on SOAP/XML, and a JSON-only approach would hinder interoperability between Mashup Server and the rest of their product suite. Like everyone else in the server-side JavaScript space, WSO2 is looking at Nodejs as a possible underlying platform for the next generation of Mashup Server, but the leading lights of Node are reportedly not keen on supporting E4X at all. I wonder what the problem is. Surely supporting XML (through E4X) in Node wouldn't come at the cost of supporting JSON! At least the CoffeeScript guys have an excuse for not supporting E4X (not that I'm letting them off the hook for that reason!) because not all browsers support E4X and they don't want to be browser-specific. But Nodejs? I think the throat to choke is V8. If V8 supports E4X, then so will Node, and then there'll be pressure on CoffeeScript to support it too, since both Mozilla and V8 do. I asked them for this 3 years ago, but no dice. Ahem, Google...?

Then there are the various flavours of web frameworks for Nodejs (Backbone, Express, Geddy, etc.), but where is SpringFramework.js? Where is the Grails, Roo or PlayFramework equivalent? I certainly hope server-side JavaScript revs up to an equivalent level of maturity as server-side Java double quick. [Server-side Java meandered a fair bit and had many mis-steps along its journey, so server-side JavaScript could get there far faster by learning from Java's mistakes.] Judging by the level of activity in the Nodejs ecosystem, we need not fret on this account. It's just a matter of time before the current effervescence and flux give way to a stable yet rich multi-layered platform.

So that's my current view in a nutshell:

* Use CoffeeScript and not raw JavaScript from now on

* jQuery is great on the client side, but will no one rid me of this turbulent DOM?

* Nodejs is The One on the server side, but lacks some critical features (e.g., TLS and E4X). I don't know if they know about broken TLS, but they don't seem to think E4X even belongs in the to-do list, which is a pity if true. I'd like to be proved wrong.

* We need better server-side frameworks, and these are already on the way. We just need to wait till the dust settles to pick a winner.

I'll post updates as my view evolves.

Advice To Organisations Embarking On SOA Today

I have been involved with SOA in various roles over the last two to three years, and my thinking has evolved a fair bit over this period. If I was asked to advise an organisation embarking on a major SOA initiative today, I would probably say this to them:

1. The End Goal: Remember that SOA is not about integration but about inherent interoperability. Think health, not medication. SOA is about raising the capability of your systems such that they can easily and inexpensively integrate with others, not about introducing a new, slick technology that will connect your systems together more easily. Simplifying the components of your enterprise and making them easy to understand and connect to will give you SOA. So keep simplicity at the back of your mind all the time, and don't confuse it with expediency, which is the path of least resistance. Simplification could take some effort.

2. Domain Models: Don't waste too much time searching for "the" canonical data model. Most off-the-shelf ones are too high-level and abstract to be useful. And building your own comprehensive dictionary is wasteful and time-consuming. Instead, identify logical owners of different elements of data and let them own the data dictionary for those elements. All services exposed out of these logical domains should use these definitions and it is the responsibility of service consumers from other domains to understand them. Processes that combine services crossing such domains should perform their own mapping between similar data elements. It's necessarily messy and plenty of out-of-band communication will be required at design time. After all, even similarly-named elements may suffer from subtle interpretation errors, so manual discussion and clarification will always be part of a service or process design.

This is not as bad as it sounds because only a subset of data elements managed by a domain is exposed through its service interface, and it's only these that may need translation to the context of their consumers. Don't look to do away with this manual effort through a single canonical data model. That's a wild goose chase, so don't even start.

3. Infrastructure and Connectivity: Try and avoid using message queues unless you're looking at low latency situations like a trading floor, or if there's simply no other way to interface to a legacy system. The queuing paradigm introduces various coordination issues into application design, and implementing message queues requires establishing more complex patterns to solve these gratuitous problems. [I have a larger, philosophical argument about the need to innovate an application protocol on top of an asynchronous, peer-to-peer transport, but let's not confuse the current set of recommendations with that idea.]

In today's world, HTTP-based communication patterns backed up by databases will often do the trick more simply than expensive message queues. Look beyond the apparent need for reliable message delivery. Often, an idempotent operation will suffice to meet the real requirement, and this is quite a standard pattern to implement. Often, queues are used in synchronous (blocking) patterns anyway (to avoid the coordination problems I talked about), so nothing is being gained in an architectural sense by the use of queues. And even asynchronous communications, where required, can be implemented in standard ways over HTTP, so HTTP is quite a universal protocol to use as the logical infrastructural element for your SOA.

ESBs, Service Directories and other "governance" components are often only required to manage the complexity that they themselves introduce. It's amazing what you can achieve with a farm of simple web servers and a database, and still keep things simple and understandable.

4. Service Enablement: Try and avoid the entire SOAP/WS-* stack of technologies if you can. There is a significant complexity overhead associated with this set of technologies, and you will need an expensive toolset to (partially) simplify its use. Look seriously at REST instead. Even though REST advocates don't make the case strongly enough (and sometimes see SOA as an antithetical philosophy), REST is in fact a valid way to do SOA and can usually help to deliver solutions at much lower cost and complexity. The hard part about doing REST is finding good people who can think that way. REST is subtly different from the SOAP/WS-* approach, even though they may just look like different kinds of plumbing to move XML documents around (and I confess that's the way I initially sell REST to corporate skeptics brought up on a diet of vendor-provided Web Services propaganda).

5. Data Contract: Consider alternatives to XML for the data contract. Though this sounds like heresy, XML is heavyweight and cumbersome, and XML manipulation tools in high-level languages (with the possible exception of E4X in JavaScript) are clumsy to use and suffer major impedance mismatches. You will spend more time wrestling with XML than on the service itself. Although many in the web world will immediately recommend JSON, raw JSON is not sufficient to ensure data integrity, because it has hitherto lacked a strong schema definition capability. Maintain a watching brief on the JSON Schema proposal, submitted for approval as an IETF standard. Already, there are JSON Schema libraries in many high-level languages such as Java. It should be possible to define data contracts with as much rigour as with XML, but at a much lower level of complexity. A newer and more compact JSON Schema representation called Orderly is also maturing, which makes this approach simple as well as easy.

So instead of going down the XML rabbit-hole, start with JSON anyway, and incorporate JSON Schema/Orderly as it matures. You will find this works especially well in combination with REST. A quick Proof-of-Concept may convince the skeptics in your organisation (although the opposite result may also occur, with many going away convinced by the speed of this approach that it's either simplistic or too good to be true!)

6. Web Service Implementation: If you're trapped by circumstances into an XML-and-SOAP/WS-* approach, look at the WSO2 suite of commercially-supported Open Source products. Especially look at the WSO2 Mashup Server. Don't be fooled by the name. It's more than just a mashup server. It's a service orchestration engine that (curiously) uses server-side JavaScript as its programming language. The major advantage of JavaScript is the ability to use the E4X library to perform extremely straightforward XML manipulation. Once you use E4X, you will never go back to JAXB or any other XML-processing library. WSO2 Mashup Server allows SOAP or REST services to be consumed, combined and orchestrated, and in turn exposes SOAP or REST services. It's a good way to hedge your bets if you're only half-convinced about REST. The WSO2 suite is also much less expensive than its proprietary rivals, although the real expense is in the heavyweight approach that it unfortunately shares with them.

7. The Paradox: SOA is really all about simplicity, but it's hard to find SOA architects who seek to simplify systems. Conventional SOA practice seems to be about making integration complex through heavyweight approaches, then introducing tools to manage that complexity, tools that require specialist skills to use properly. If done the conventional way as most SOA consultants seem to agree, your SOA initiative will only leave you with additional complexity to manage.

Of course, if you're politically inclined, you will bask in the prestige of a hefty budget and a large team, and can declare victory anyway on the basis of the number of services and processes you have delivered. But if you want to be really successful at delivering SOA (i.e., making your business more agile and able to operate on a sustainably lower cost basis) while keeping your burn rate low along that journey, you would do well to look at boring, unimpressive and even anticlimactic approaches and technologies such as the ones I've listed above. Give the big vendors a wide berth. You don't need to buy technology (beyond the web servers and databases you already have). You certainly don't need to buy complex technology, which is what the vendors all want to sell you.

And don't let the lack of grandeur in this approach worry you. Complexity impresses the novice, but results are what ultimately impress all.

Postscript: Vendor REST is coming. Beware.

Google Chrome - The Promise

I'm excited about Google Chrome, the newest browser on the block, even though I haven't used it yet.

How come? Well, I use Linux at home, and Google hasn't seen fit to release a Linux version of Chrome yet...

Still, I am excited.

But do we really need yet another browser? Well yes, I think Chrome is important for at least a couple of reasons.

1. It's high time Netscape's original vision of the browser as an application platform was realised. Our current generation of browsers is architecturally hobbled. Chrome implements the behind-the-scenes improvements required for a browser that has ambitions of being a true application platform. See this tech-friendly explanation of what these improvements are.

2. It's high time Microsoft's browser market share received another blow. For those who think Microsoft has begun to play nice in the web standards space, think again. They're up to their old dirty tricks once more. Anyone whose employer has deployed Sharepoint will know what I mean. There are heaps of Sharepoint features that don't work in Firefox. You're forced to use IE if you have Sharepoint. We learn once again that it's dangerous to give Microsoft control of both ends of the Web. Apache is slipping. And Firefox isn't enough. I hope Chrome provides enough buzz to draw users away from IE.

For all the coolness of Chrome, there are still important features missing. The obvious one for me is E4X support. How can an application platform not have native support for XML manipulation? We live in a SOA world, don't we? We have to deal with business logic in the form of services, right? So what's with the lack of XML support? I certainly hope this is a temporary issue, because I believe the application platform of the future must support SOFEA, and without XML support, it's just not going to cut it.

Of course, these are early days, so the glass is really half-full. Good on ya, Google!

No, Really. What is SOAP? What is BPEL?

Sometimes the answers to simple questions are profound.

I have been asking myself a couple of very simple questions for some time now, and although I initially didn't want to accept some of the answers I got, I guess I can't avoid their implications.

What is SOAP? I'm deliberately not looking at any of the SOAP spec documents for the answer. I want to know the value proposition of SOAP.

Similarly, what is BPEL (or WS-BPEL, if you prefer)?

I can think of two quick answers:

1. The value proposition of SOAP is interoperability.
2. The value proposition of BPEL is portability.

To my mind, interoperability is at least an order of magnitude more important than portability.

SOAP is the "wire protocol" in one view of SOA. Within the "cloud" or "SOA Fabric", the only things one sees are SOAP messages flying past. Nobody sees BPEL within the SOA cloud. BPEL is an implementation language for the execution of processes, just like Java is an implementation language for general-purpose business logic. It runs on a node. All interactions between that node and others are through SOAP messages. A BPEL process consumes SOAP-based services and in turn exposes itself as a SOAP service.

So is BPEL really all that important in the scheme of things? Wouldn't any other portable language do? I think I've stumbled upon a candidate language, and it's not Java.

Lately, I've been playing around with the WSO2 Mashup Server, and I'm increasingly getting the excited feeling of a kid who's somehow come into possession of a machine-gun. This is not a toy, folks. Are the adults aware of what this thing can do?

Most people seem to think mashups are a cute visual gimmick. The WSO2 guys themselves don't have the air of people handing out machine-guns. The examples they bundle with the server are classic mashup fare - TomatoTube, in which you take the top-rated movies from Rotten Tomatoes and mash the list up with trailers from YouTube, enabling you to see the trailers of top-rated movies. Very cute and harmless.

But now for the machine-gun bit. The development language offered by Mashup Server is JavaScript (think general-purpose programming language). JavaScript augmented by E4X (think really easy XML manipulation). Mashup Server hides SOAP very effectively, although its interfaces to the outside world are SOAP (SOAP 1.1 and 1.2, also REST/HTTP, but more about that later). SOAP is out there in the cloud. But here, within this processing node, it's just XML representations of data and JavaScript code to process it with, thanks to the why-didn't-I-think-of-it simplicity of E4X. Surely we can do more than mashups with that kind of power...

I've found myself thinking a disturbing thought: If no one sees BPEL in the forest, then does it really exist? What if BPMN-based process modelling tools spat out E4X-enhanced JavaScript code instead of BPEL? Would anyone know or care? Take the output of the modelling tool and drop the file onto a server. The process is ready to run. All external interfaces are SOAP-based, just like with a BPEL implementation. Got any problems with that?

There's more revolutionary potential here than in the Communist Manifesto. Another of the cutesy examples bundled with Mashup Server is a REST service. You can do HTTP GETs, PUTs, POSTs and DELETEs on a simple city resource to manipulate its weather data. Very harmless, but again, the developer has very simple JavaScript-based access to REST services.

So is the WSO2 Mashup Server the one that will bring balance to the Force? A powerful programming language. Laughably easy XML manipulation. Simple access to SOAP services and REST resources. Transparent publication of itself as a service or resource in turn. Isn't this the holy grail of service composition?

Content management, process orchestration, what's in a name? I'm beginning to think BPEL is dead. Its value proposition doesn't stack up anymore. SOAP still makes sense, but not BPEL.

WSO2 Mashup Server seems to be the industry's best-kept secret for now. Keep the safety catch on, and watch where you point that thing.