Targeting A Better Life Above The Service Tier

In 2007, Rajat Taneja, Vikrant Todankar and I co-wrote a white paper called Life above the Service Tier. This was back when web applications were built using server-side frameworks. Our paper decried these frameworks as representing anti-patterns, and recommended that web applications ought to run independently within client devices (such as mobile phones and desktops), and only make service calls to the back-end.

We called this architecture SOFEA, for Service-Oriented Front-End Architecture.

The core principle of SOFEA was to decouple the three orthogonal concerns of Application Download, Presentation Flow, and Data Interchange, which were tightly coupled in server-based frameworks.

This recommendation seemed revolutionary at the time, as witnessed by the many blog posts and comments that it received. However, when we fast-forward 16 years to today, SOFEA is in fact the way the vast majority of web applications are being built. All the popular web frameworks, such as Angular, React, Vue, and Svelte, are SOFEA-based.

Everyone should have lived happily thereafter, but the main problem being faced by development shops today is that these web frameworks are too complex and heavyweight. It takes months or years to acquire true expertise in any of them. Full Stack developers are hard to come by, because both the back-end and the front-end require in-depth skills. This has made it costly and time-consuming not only to build applications, but also to maintain them over their lifetime.

Rajat, Vikrant and I discussed this once again, and we were joined by another friend, Subbu Balakrishnan.

The time seemed ripe to release an update to the SOFEA paper, this time recommending a simpler architecture for the front-end. We have been tracking developments in the industry over the past 16 years, and have detected some paradigm shifts taking place, which the most popular web frameworks have not taken advantage of. Other frameworks have leapfrogged them, but aren't as well-known, or as comprehensive.

Here's our high-level view of industry trends.

What we have come up with is a new architectural style, a successor to SOFEA that we call BUCAREST (for Browser-based Unified Client Architecture (using) REST). This is not just a theoretical concept, but practically implementable using frameworks that are available today. While no single framework today can implement BUCAREST in its entirety, a combination of frameworks used in alignment with the patterns we describe will do the job just fine.

The two frameworks we have identified are Alpine.js and HTMX.

The diagram describing this architecture is here:

This new white paper, which describes BUCAREST in detail and explains how to use it to build web applications in a much simpler way, is called Targeting a Better Life above the Service Tier.

We hope the BUCAREST style will greatly simplify web development in the coming years. Thanks to its simplicity, it should also be possible for those who have considered themselves back-end developers to venture into front-end developent too. We daresay there will be more "Full Stack" developers in the years to come.

Download our white paper, have a read and tell us what you think!

Google's Aimless Darting

Search Engine vendors should be in the clarification business, not the muddying business. All the more frustrating to read the news about Dart, Google's new web programming language to replace JavaScript.

My only reaction is - WHY???

Yes, JavaScript is not exactly a perfect language. It has warts, huge ones. That's been known for a long time. Is that sufficient reason to throw the whole language overboard and try and popularise a new one? Especially after CoffeeScript has already done the job? JavaScript has good parts as well as bad, and I'm not talking about just its technical aspects. Its ubiquity is a major strength. Replacing it is an exercise with very doubtful prospects. I don't think even Google can pull it off.

CoffeeScript has taken the right approach, in my opinion. CoffeeScript is JavaScript with lead shielding over the reactor core. And that was all that was required. CoffeeScript on the server side uses Nodejs to run scripts anyway, so one can write server-side code in CoffeeScript and run it with "coffee" instead of with "node", and the ugliness and danger of JavaScript can be neatly sidestepped. Even on the client side, a single line such as

<script type="text/JavaScript" src="coffeescript.js"></script>

will allow you to write the rest of your client-side code in CoffeeScript, because coffeescript.js is a minimised library that will let your browser interpret CoffeeScript natively. Your application code will look like this:

<script type="text/CoffeeScript">
# CoffeeScript code
</script>


What is Dart going to do for us beyond that, functionality-wise? Technically, Dart won't even replace JavaScript, because it will compile to JavaScript. Does that sound familiar? Because that's just what CoffeeScript does as well! Was it so hard for Google to get behind CoffeeScript? Some NIH at work, I think.

Now the frustrating thing is that Dart will waste at least some developer mindshare and bandwidth when the world should have been just getting on with the job - using CoffeeScript plus jQuery on the client side, CoffeeScript plus Nodejs on the server side.

What a waste!

The Return of JavaScript - My Thoughts Crystallise

I wrote some months ago that JavaScript was back. I've been watching developments in that space with close interest, and a few glimmerings of "the right way forward" have begun to suggest themselves to me.

The first aspect pertains to the language itself. The JavaScript Renaissance has brought with it a desire to do things right the second time around. JavaScript doyen Douglas Crockford has lectured extensively (and even written a book) on the need to use only "the good parts" of JavaScript and avoid the bad ones. It's excellent advice, but to a new generation of JavaScript programmers, I suspect it may not be enough to post warning signs around bad features. We need a way to make it physically impossible to use JavaScript's "bad parts".

Enter CoffeeScript. While I haven't played with it too much, I like what I see here. [Update 24/09/2011 - I *have* played with it for a few hours now, and I really like!] There has been some skepticism about CoffeeScript from a section of the JavaScript old guard, but that's to be expected. The best answer to such skepticism is from ThoughtWorks's latest Technology Radar document, which is well worth a read in itself.

First, even if it seems a trifle off-topic, read what Thoughtworks thinks about GWT, which is a way to write visual components in Java and have JavaScript-based widgets automatically generated for a web application that runs within a browser:

GWT is a reasonable implementation of a poor architectural choice. GWT attempts to hide many of the details of the web as a platform by creating desktop metaphors in Java and generating JavaScript code to implement them. First, in many ways, JavaScript is more powerful and expressive than Java, so we suspect that the generation is going in the wrong direction. Secondly, it is impossible to hide a complex abstraction difference like that from event-driven desktop to stateless-web without leaky abstraction headaches eventually popping up. Third, it suffers from the same shortcomings of many elaborate frameworks, where building simple, aligned applications is quick and easy, building more sophisticated but not supported functionality is possible but difficult, and building the level of sophistication required by any non-trivial application becomes either impossible or so difficult it isn’t reasonable.

Now read what Thoughtworks says about CoffeeScript:

Some readers may be confused by our advocacy of Coffeescript given our general dislike for GWT, because on the surface they seem similar: tools that generate JavaScript. However, it is the level of abstraction that differs. GWT has an elaborate component model, which tries to hide details about the underlying language (JavaScript) and platform (the web). Coffeescript tries to make it easier to write proper JavaScript, avoiding pathological but default “features” of JavaScript, and does not build a layer that tries to insulate you from the platform.

I think they've nailed it. I have been ambivalent about GWT for a while, but couldn't quite put my finger on why (i.e., it's a "reasonable implementation of a poor architectural choice"). They've also very cogently argued why CoffeeScript doesn't fall into that category.

So my first takeway from the JavaScript Renaissance is that today's developers should code in CoffeeScript and not in JavaScript itself. If Java and Scala are two languages that compile to Java bytecode and run on the JVM, then think of JavaScript and CoffeeScript as two languages that "compile to JavaScript" and run within a JavaScript engine. The runtime behaviour is unchanged, but you use the language that works better for you as a developer.

CoffeeScript is JavaScript with the sharp edges removed. I'm sold. [There is an issue with E4X, which I'll cover when I talk about Nodejs, but it applies equally to CoffeeScript. In short, there's no support for E4X, and I want it!]

Beyond the choice of language, the direction is not so straightforwardly clear. It's tempting to say that the future of JavaScript on the client is CoffeeScript plus jQuery, and the future of JavaScript on the server is CoffeeScript plus Nodejs. But there are some issues.

To paraphrase Thoughtworks's critique of GWT, jQuery is an elegant way of working with a horrible object model (the DOM). I don't know if there is any way to replace the DOM with a better client-side abstraction, but until then, the best of client-side goodness will remain somewhat unsatisfactory.

Nodejs is the server-side angel in white from all accounts, but I've heard some valid grumbling about Node as well. The Spanish outfit ASPL has a BEEP implementation called jsVortex. This JavaScript-based server pre-dates Nodejs, and ASPL was reportedly keen to explore Node as the underlying platform for the next version of jsVortex. However, they quickly realised that Node had no support for TLS at its core. This is not the same as supporting HTTPS, because HTTPS support obviously exists in Node. Think of TLS as HTTPS-like capability for any protocol, and since BEEP is a framework to create protocols à la carte, and since security is a ubiquitous and pretty strong requirement for most protocols, the lack of TLS support within Node is a serious deficiency for them and for anyone who wants BEEP on the Nodejs platform.

[Most people might go "BEEP who?", but in my opinion, this is a technology that was sadly delayed by over a decade, perhaps because of personality politics caused by creator Marshall Rose's less-than-smooth interactions with powerful members of standards bodies, who then blocked BEEP from becoming a standard until very recently. If we'd had BEEP 15 years ago, I suspect SOAP and WS-* would have looked very different, and perhaps REST would have looked different too. That's how powerful it is, and I still think it will change our lives one day.]

Well, so there's a piece of work to be done to enable TLS within the core of Nodejs, but that's not all.

WSO2's Mashup Server was another under-appreciated piece of server-side JavaScript from many years ago, and one of its best features was its support for E4X. It's understandable why WSO2 would be keen on E4X, because their enterprise middleware suite is based on SOAP/XML, and a JSON-only approach would hinder interoperability between Mashup Server and the rest of their product suite. Like everyone else in the server-side JavaScript space, WSO2 is looking at Nodejs as a possible underlying platform for the next generation of Mashup Server, but the leading lights of Node are reportedly not keen on supporting E4X at all. I wonder what the problem is. Surely supporting XML (through E4X) in Node wouldn't come at the cost of supporting JSON! At least the CoffeeScript guys have an excuse for not supporting E4X (not that I'm letting them off the hook for that reason!) because not all browsers support E4X and they don't want to be browser-specific. But Nodejs? I think the throat to choke is V8. If V8 supports E4X, then so will Node, and then there'll be pressure on CoffeeScript to support it too, since both Mozilla and V8 do. I asked them for this 3 years ago, but no dice. Ahem, Google...?

Then there are the various flavours of web frameworks for Nodejs (Backbone, Express, Geddy, etc.), but where is SpringFramework.js? Where is the Grails, Roo or PlayFramework equivalent? I certainly hope server-side JavaScript revs up to an equivalent level of maturity as server-side Java double quick. [Server-side Java meandered a fair bit and had many mis-steps along its journey, so server-side JavaScript could get there far faster by learning from Java's mistakes.] Judging by the level of activity in the Nodejs ecosystem, we need not fret on this account. It's just a matter of time before the current effervescence and flux give way to a stable yet rich multi-layered platform.

So that's my current view in a nutshell:

* Use CoffeeScript and not raw JavaScript from now on

* jQuery is great on the client side, but will no one rid me of this turbulent DOM?

* Nodejs is The One on the server side, but lacks some critical features (e.g., TLS and E4X). I don't know if they know about broken TLS, but they don't seem to think E4X even belongs in the to-do list, which is a pity if true. I'd like to be proved wrong.

* We need better server-side frameworks, and these are already on the way. We just need to wait till the dust settles to pick a winner.

I'll post updates as my view evolves.

The Return of JavaScript

Hardly had I begun to play with Node.js when a friend pointed me to this news item about JavaScript being used to run Linux.

Déjà vu all over again.

Some months ago, after the perceptible pall that descended on Java with the Oracle takeover of Sun, a group of tech enthusiast friends of mine opined that the time was ripe for a new technology. Java was now legacy. It would continue to grow and influence, of course, but its glory days were over. Java had been a shooting star under Sun. Under Oracle, it was now a cash cow. The next paradigm shift would therefore have to come from a new technology altogether. Much as we might pride ourselves as technology futurists, we only struggled to pierce the veil. We spoke about Android, cloud computing, NVIDIA processors and Web 3.0, but none of us imagined that the revolution would come from an established and very familiar technology.

JavaScript is as old as the web, and born close on the heels of Java, which explains its confusingly similar name. (Well-meaning managers at leading technology companies were known to discourage programmers from writing JavaScript because they heard that the corporate firewalls were configured to block Java applets. Microsoft evangelist Stephen Jeffries even ruined a perfectly credible sales pitch for Microsoft technology in 1999 by claiming to have written "some Java scripts", which were inferior to his company's offerings, of course.)

Naming confusion aside, not many will remember that in the early days of the web (circa 1995), Netscape not only had a browser (the legendary Navigator), but also a web server, a directory server, a certificate server and many other components of a web-based ecosystem. JavaScript was a Netscape invention, and the Netscape web server could run Java servlets as well as JavaScript code. There was an attribute that could be used with the "script" tag in HTML, which was "runAt = 'server' ". JavaScript running on the server - imagine that!

But that initial innovation died out shortly after. JavaScript became a purely client-side phenomenon. Other languages, notably Java and PHP (and ASP in the Microsoft world) took over the server side.

The next time I heard of JavaScript running on the server side, it was in the context of WSO2's highly innovative Mashup Server.

Server-side JavaScript was still a rare phenomenon, and had the faint aroma of a toy system to boot. After all, serious software was written in Java. JavaScript was at best useful to demonstrate clever ideas. It wasn't production strength.

Now along comes Node.js with its innovative thesis about how to do I/O right. Ryan Dahl explains the philosophy in a way that appears obvious in hindsight. When you see the numbers he puts up comparing the CPU cycles for different levels of I/O, it becomes extremely clear that it's criminally wasteful of computing resources to wait on disk and network I/O.

L1 cache: 3 cycles
L2 cache: 14 cycles
RAM: 250 cycles
Disk: 41,000,000 cycles
Network: 240,000,000 cycles

Node.js boasts a single event loop which, combined with its absolute refusal to block on any external I/O operation, allows it to scale up to ridiculous levels of concurrency. Add to that the simplicity, flexibility and familiarity of the JavaScript language, and the runaway popularity of Node.js becomes understandable. I'm personally blown away.

Even Ryan Dahl, its inventor, struggles to explain what Node.js is (beyond the obvious fact that it's a bunch of libraries on top of Google's V8 JavaScript virtual machine). That's because Node.js is nothing less than a new platform for innovation. It's clay in the hands of a new generation of developers, and fascinating new products (Node.js modules and frameworks) are emerging.

I think one of these will be a rich GUI library to rival the offerings from Adobe. It will be possible to build extremely flexible and lightweight client apps with rich user interfaces very soon. Web forms will soon seem clunky, because we can soon have web applications without web forms.

Another phenomenon that Node.js will enable is peer-to-peer systems. I believe we've outgrown the synchronous request-response behaviour of the traditional web. Something new and big is coming soon.

So move over, Java. Here comes JavaScript.

No, Really. What is SOAP? What is BPEL?

Sometimes the answers to simple questions are profound.

I have been asking myself a couple of very simple questions for some time now, and although I initially didn't want to accept some of the answers I got, I guess I can't avoid their implications.

What is SOAP? I'm deliberately not looking at any of the SOAP spec documents for the answer. I want to know the value proposition of SOAP.

Similarly, what is BPEL (or WS-BPEL, if you prefer)?

I can think of two quick answers:

1. The value proposition of SOAP is interoperability.
2. The value proposition of BPEL is portability.

To my mind, interoperability is at least an order of magnitude more important than portability.

SOAP is the "wire protocol" in one view of SOA. Within the "cloud" or "SOA Fabric", the only things one sees are SOAP messages flying past. Nobody sees BPEL within the SOA cloud. BPEL is an implementation language for the execution of processes, just like Java is an implementation language for general-purpose business logic. It runs on a node. All interactions between that node and others are through SOAP messages. A BPEL process consumes SOAP-based services and in turn exposes itself as a SOAP service.

So is BPEL really all that important in the scheme of things? Wouldn't any other portable language do? I think I've stumbled upon a candidate language, and it's not Java.

Lately, I've been playing around with the WSO2 Mashup Server, and I'm increasingly getting the excited feeling of a kid who's somehow come into possession of a machine-gun. This is not a toy, folks. Are the adults aware of what this thing can do?

Most people seem to think mashups are a cute visual gimmick. The WSO2 guys themselves don't have the air of people handing out machine-guns. The examples they bundle with the server are classic mashup fare - TomatoTube, in which you take the top-rated movies from Rotten Tomatoes and mash the list up with trailers from YouTube, enabling you to see the trailers of top-rated movies. Very cute and harmless.

But now for the machine-gun bit. The development language offered by Mashup Server is JavaScript (think general-purpose programming language). JavaScript augmented by E4X (think really easy XML manipulation). Mashup Server hides SOAP very effectively, although its interfaces to the outside world are SOAP (SOAP 1.1 and 1.2, also REST/HTTP, but more about that later). SOAP is out there in the cloud. But here, within this processing node, it's just XML representations of data and JavaScript code to process it with, thanks to the why-didn't-I-think-of-it simplicity of E4X. Surely we can do more than mashups with that kind of power...

I've found myself thinking a disturbing thought: If no one sees BPEL in the forest, then does it really exist? What if BPMN-based process modelling tools spat out E4X-enhanced JavaScript code instead of BPEL? Would anyone know or care? Take the output of the modelling tool and drop the file onto a server. The process is ready to run. All external interfaces are SOAP-based, just like with a BPEL implementation. Got any problems with that?

There's more revolutionary potential here than in the Communist Manifesto. Another of the cutesy examples bundled with Mashup Server is a REST service. You can do HTTP GETs, PUTs, POSTs and DELETEs on a simple city resource to manipulate its weather data. Very harmless, but again, the developer has very simple JavaScript-based access to REST services.

So is the WSO2 Mashup Server the one that will bring balance to the Force? A powerful programming language. Laughably easy XML manipulation. Simple access to SOAP services and REST resources. Transparent publication of itself as a service or resource in turn. Isn't this the holy grail of service composition?

Content management, process orchestration, what's in a name? I'm beginning to think BPEL is dead. Its value proposition doesn't stack up anymore. SOAP still makes sense, but not BPEL.

WSO2 Mashup Server seems to be the industry's best-kept secret for now. Keep the safety catch on, and watch where you point that thing.