JavaOne 2009 Day Two (02/06/2009)

The first day of JavaOne proper (or the second day if you include CommunityOne on Monday) started with a two hour General Session.

A brief documentary showing the parallels between a 14 year old boy named Justin and 14 year old Java set the tone for the session. Each year from 1996, a different facet of Java has come to the fore. And in 2009, just as Java turned 14, the 14 year old boy was introduced as a gamer and Java developer. Sun is appealing to a new generation of developers now, and the emphasis on JavaFX reflects a generational shift in more ways than one.

Jonathan Schwartz, Sun's CEO, came up on stage next, and was the MC for almost the rest of the session. I have always been impressed by Schwartz for the direction he has given Sun as CEO, and I was impressed by his poise, confidence and easy manner as he conducted the proceedings. He did goof up a bit, though. He announced the release of Java 7 as of today, but it turned out later that it was just a milestone release. The final release of Java 7 isn't due till early 2010.

The theme for the morning was the power of a simple idea - to wit, "Write Once, Run Anywhere (WORA)".

An impressive array of customers and business partners trooped up to join Schwartz one after the other, all testifying to the wonderful goodies that Java technology had delivered to their organisations.

First up was James Baresse (VP Architecture, Platforms and Systems, eBay). eBay uses the integrated Java stack to run their business - the application framework, content management, batch systems and SOA.

Next in the witness box was Alan Brenner (Senior VP for the BlackBerry platform at RIM). For those who don't know already, the BlackBerry is a Java phone. RIM uses Java end-to-end - the core apps, the development platform, the works. Brenner showed off a third party app called Zavnee that integrates with the BlackBerry's email and phonebook APIs as well as community sites to provide a more insightful address book. According to Brenner, the open APIs of BlackBerry allow ISVs to build applications that integrate tightly with the device.

Don Eklund (Executive VP of Advanced Technologies at Sony Pictures Home Entertainment) came up to crow about the victory of Blu-Ray over its competition, or so it seemed to me. I'm not a great fan of Sony, which represents evil media. Eklund sang the praises of Java's openness. I couldn't help mentally completing his statement. Every company loves it when its building blocks are open and free, and when their own products are closed and proprietary. How about opening up the Blu-Ray format to everyone, Sony?

Lowell McAdam (President and CEO, Verizon Wireless) spoke about Verizon's Open Development Initiative last year that dealt with hardware, and their Open Development for Applications this year that deals with software. Verizon is opening up its APIs to enable third party developers to exploit functions such as presence, location, friends and family, etc.

Diane Bryant (Executive VP and CIO at Intel) talked about the collaboration between Intel and Sun to deliver high Java performance on the Intel platforms (Atom, Core and Xeon). They claim to have achieved an 8x raw performance improvement since 2006.

The final partner to come up on stage was Paul Ciciore (Chief Technologist at the Chicago Board Options Exchange). The CBOE is the world's largest options exchange, but it's a relatively new company. The Java-based implementation was conceived in 1998 and launched in 2001. From 5000 transactions per second in 2001, CBOE has grown to 300,000 transactions per second in 2009. They're also driving latency down lower and lower.

The next phase of the General Session talked about what I would characterise as an attempt by Sun to shift gears and start to provide the technology tools for applications that deal with media content in addition to those for the traditional enterprise applications that have been its mainstay so far. It's a risky gambit for Sun, and I'm not very optimistic about their chances of success. As an example, if Adobe trivially reengineers Flex to generate JVM bytecode, it's bye-bye, JavaFX.

There were some nifty demos of JavaFX applications, including one that ran on an HDTV set. There was another demo by Sun's Director of Engineering for the JavaFX platform, Nandini Ramani (wrongly pronounced (what else?) Ramaani). She showed off a nifty development environment for JavaFX that allowed content to be edited and targeted simultaneously for different devices. There's no compile-build cycle for JavaFX, this being a scripting language.

James Gosling, the inventor of Java, then came up on stage. This is one person I put in the same category as Bob Metcalfe, the inventor of Ethernet. Both are technology geniuses who have an astonishing blind spot about Open Source. [Metcalfe once famously referred to it as "Open Sores".] Gosling made a snide remark about how the Linux platform doesn't let developers build applications for it unless they're willing to make it a labour of love. His self-styled mission is to help developers convert a labour of love into a day job that puts food on the table. That's all very well, but someone should tell him that Open Source works very well as it is, thank you very much. It's a benign pyramid scheme, where a new generation of programmers comes along to build the next layer of an open platform on top of the last. The commoditisation continues up the stack. Too bad for people wanting to make money along the way. But that's not a problem for Open Source, nor is it a problem for the world. It's only in Gosling's (and Metcalfe's) world that a failure to monetise something equates to failure, period.

Gosling presented the Duke's Choice award to Mark Gerhard (CEO of Jagex, maker of the popular RuneScape game). About 20% of Jagex's users are paying customers. RuneScape will be available on the Java Store that I wrote about earlier. The Java Store provides tools to "ingest" and "distribute" applications, and Sun is still working on a suitable cash register implementation to enable effective commercialisation. Suggestions from the community are welcome...

Randy Bryant (Dean of Computer Science at Carnegie Mellon University) received another Duke's Choice award from Gosling on behalf of Randy Pausch, the inventor of the Alice Project. Alice, like MIT's Scratch, is a means of teaching computer programming to kids. Alice 3 goes a step further by introducing kids to Java programming, which Scratch doesn't do. [I'm planning to introduce Alice to my twelve year old.]

Somewhere along the way, Scott McNealy, Sun's chairman and former CEO, came up on stage. I have previously referred to McNealy as a dinosaur, and I must say that he literally looked and sounded old. Gone was the youthful "puckish humour" that magazines used to refer to. He seemed resigned to an imminent retirement. Reading between the lines, though he made some condescending remarks to Jonathan Schwartz about the wonderful way he had led the company even though he was a relative newcomer, I sensed some resentment and disapproval. I guess it's an open secret that Schwartz preferred an IBM takeover of Sun, and McNealy won out with the Oracle deal. I wonder how long Schwartz will last with Oracle CEO Larry Ellison as his boss. Open sourcing Java was the best thing Schwartz has done for the world, and must have turned Ellison's dreams of monetisation to ashes.

The piece de resistance of the morning's session was then the surprise introduction of Ellison himself.

Ellison said many nice things about Java (e.g., "Except for the database, everything at Oracle is Java-based"). Again, I was tempted to complete his sentence that "Java was attractive to us because it was open" with the clincher "and because it has allowed us to close everything we build above it". I hope the emerging Open Source enterprise stack eats Oracle's lunch in the coming recession.

Ellison said a few things that I found very significant. He wanted to see OpenOffice being redeveloped using JavaFX. He's now in a position to drive that initiative through investment, and something tells me he'll do it. It's more than just his traditional hatred of Microsoft. There's probably a big support subscription-based revenue stream that will come from the corporate market when OpenOffice supplants MS-Office.

Ellison also pledged not to make changes to the Java model but to "expand investment" in it, a commitment that drew relieved applause from the mostly geek audience. I guess there's indirect benefit to Oracle from Java's success, largely from its ability to prevent competitors from succeeding with their proprietary equivalents.

In another very significant statement, Ellison also hinted that Sun and Oracle would jointly introduce a mobile device to compete with devices running Google's Android. I think JavaFX is the weapon that Oracle-Sun are betting on.

Someone (I forget whether it was Gosling or someone else) also made a snide remark about Ajax that made me prick up my ears. I have previously wondered on this blog why people even bother with RIA tools like Flash, Silverlight and JavaFX when Ajax/DHTML is getting to be so much more capable. I realise now that I've been looking at it from the viewpoint of the community at large. From the viewpoint of Sun (and now Oracle), there is a silent and desperate struggle for survival going on. If Ajax succeeds, it will further reduce the relevance of these vendors. JavaFX is critically important to Sun. It's irrelevant to the world.

The second session I attended was "Deploying Java Technology to the Masses: How Sun deploys the JavaFX runtime" by Thomas Ng from Sun.

My major grouse against JavaFX is that it still isn't available for the Linux platform. Nigel Eke commented on my earlier blog post that Sun was going to announce Linux support for JavaFX at JavaOne, but it hasn't happened yet.

Very briefly, Ng's talk covered the following points - the deployment mechanism is JNLP, pioneered by Java Web Start almost a decade ago. There's a tool called the JavaFX Packager that is bundled with the JavaFX SDK. This helps to automate the creation of JNLP files, which is otherwise an error-prone undertaking. In future, the same tool will be used to create JNLP-based launchers for applets and Java Web Start applications, but as of today, that's still on the wishlist.

A potentially very useful feature (albeit a potential security headache) is the ability of the JavaFX runtime to permit cross-domain XML traffic. This removes the crippling constraint that prevents current-generation browser applications from reaching back to any servers but their own to make service calls.

A future version of JNLP will also remove the current requirement for an absolute "codebase" URI in the jnlp tag. This relaxation will make applications more readily portable between servers.

The next session was another General Technical Session hosted by Robert Brewin (Distinguished Engineer, VP and CTO for Application Platform Software at Sun). It was called "Intelligent Design: The Pervasive Java Platform".

[My "Press/Analyst" badge entitled me to special seating at the front of the hall, but since the perk didn't come with a table for my laptop, I declined the honour and chose to sit in the bleachers with a mortal friend.]



Items in brief:

Project Kenai allows developers to collaborate, share and engage. It seems to be a richer version of CVS or Subversion as a repository, because it has some of the characteristics of a social networking site. It will soon include continuous integration capabilities through its incorporation of the Hudson continuous build tool.

JDK 7 (prematurely announced by Jonathan Schwartz) will feature 3 things when it debuts - a modular platform, a multilingual VM and productivity enhancements for developers.

A new dependency information file format for classes means that "the classpath is dead", a statement that drew cheers. This diagram shows the new packages in Java 7, and their dependencies.



It will also make it easier to create deployment packages for various target platforms (such as .deb files for Ubuntu Linux).



There is a project called the "Da Vinci Machine" to enable the Java VM to be a true multi-language VM. For the first time in a decade, there will be new bytecode that will effectively upgrade the JVM architecture.

Then there's the cheekily-named Project Coin, which deals with small change(s) to Java.

Many minutes were wasted on a description of JEE 6. I have never understood the rationale for some of the components of JEE, and why Sun insists on throwing good money after bad. There's JSF 2.0 and EJB 3.1. The EJB cancer has reached Java's lymph nodes and is now threatening to infect healthy web server tissue through "Lite EJB", which can be bundled inside .war files and does not require .ear files or a heavyweight container. Why, why, why??? I thought Spring framework chemotherapy had forced EJB into remission, but this seems much more virulent than I thought.

Bean validation (JSR 303) seems to be a way to ensure consistent data validation across tiers (through JSF in the presentation tier and JPA in the persistence tier). I can't help thinking this is just the Java version of XForms. XForms carries an XML document payload that is defined through a schema that can then be used to validate this instance data in any tier.

There's a lot of emphasis on the Glassfish server that I just can't understand. Again, it's probably critical for Sun to own an application server that isn't just Tomcat+Spring. It just isn't that relevant to the rest of the world. Glassfish has ambitions of being scalable, "from embedded to carrier-grade".

One corollary of a more modular Java that I deduced and that Sun acknowledged is that in the future, there will be just one Java. No more ME, SE and EE variants.

I next attended a rather boring and depressing session that was optimistically called "Tips and Tricks for Ajax Push and Comet Apps". This was conducted by Jean-François Arcand of Sun and Ted Goddard of ICEsoft Technologies. It was depressing because at the end of the day, there seems to be no satisfactory method to implement server push that will scale well and work on all browsers. The lecture was a tour of various unsatisfactory compromises. The three standard techniques (Polling, Ajax push (long polling) and HTTP Streaming) all have their drawbacks. I personally like HTTP Streaming with its chunked response (multiple as-and-when partial responses to a single request). However, this doesn't seem to work very well in practice because of the lack of a client-side API for reading such an input stream, and the tendency for proxies to cache responses instead of passing them on, incomplete though they may be.

My conclusion is that push is impossible with HTTP. We need to implement AMQP over TCP for true bidirectional messaging. It will take time, but it will happen within a decade. And then these problems (and their kludgy solutions) will seem laughable in retrospect.

The last session that I attended was "Spring Framework 3.0: New and Notable" by Rod Johnson himself. The hall was huge but absolutely packed.

Points in brief:

Spring 3.0 will use Java 5+. Anyone wishing to use Java 1.4 must remain with Spring 2.5. Three cheers for courage. Backward-compatibility is a two-edged sword, and I'm glad Spring has made the leap (no pun intended).

There is a new expression language (EL) that allows developers to navigate bean properties, invoke methods and allow construction of value objects. It also allows us to parameterise annotated code:

@Value("#{systemProperties.favoriteColor}")
private String favoriteColor;

where the term within "#{}" is in the EL.

There is an elegant implementation of REST based on Spring MVC (and what was unsaid was that it was not based on JAX-RS). The Spring MVC ViewResolver is a natural point to implement the various resource representations that REST provides a common interface to.

There are several MVC enhancements. @RequestHeader provides access to HTTP request headers. @CookieValue provides access to cookies. It's also possible to have application-specific annotations.

There is a new Java configuration mechanism. Annotations can now be placed in dedicated "configuration classes" rather than in POJOs themselves. In other words, these are XML files all over again, but in Java 5 syntax rather than using angle brackets. I like this, because I was uneasy about annotations. One of the major benefits of externalising configuration is the ability to decouple wiring from the application's own components and make it possible to substitute implementations without recompiling code. Annotations in code are a step backwards, from this viewpoint. Annotations in dedicated configuration classes are three-quarters of a step forward again. We still need a recompile, but only of the config classes.

Rod Johnson pointed out that previous versions of Spring had a rather ad hoc approach to the web side of an application, but that Spring 3.0 now features a consistent stack of components. The base is Spring MVC. Above this layer are Spring Web Flow, Spring Blaze DS (for Flex) and Spring JavaScript. At the top layer is Spring Faces (ugh!). Spring Blaze DS will make it very easy to build Flex apps with Spring. The diagram made no mention of Spring Portlet MVC, which is perhaps just as well. [The portlet specification is another monstrosity that continues to give organisations grief. As an "integration" technology, it belongs more in the problem space than in the solution space.]

The Spring Java config mechanism is type-safe and does not depend on string names. It therefore has more robust bean-to-bean dependencies. It allows inheritance of configuration. And it allows object creation using arbitrary method calls.

Johnson also introduced Spring Roo, the Domain-Driven Design tool and Ben Alex's brainchild. There is a major functional overlap between Roo and Grails, which Johnson did point out, but his logical flowchart to decide which one to use was a bit contrived, in my opinion. Roo and Grails are competitors, and the fact that they are uncomfortable co-components in the Spring portfolio does not make them part of a seamless product line with an implied "horses for courses" decision-making logic. To put it bluntly, if Roo had appeared a couple of years earlier, Grails would probably have been stillborn. The delay in Roo's release gave Grails its break, and now they're competing for developer mindshare. Well, may the best framework win.

JavaOne 2009 Day One (01/06/2009)

I got an invite to the 2009 edition of JavaOne as a blogger, with a Press Pass and everything ;-).


The conference is at the Moscone Center in SanFrancisco.



June 1 saw the CommunityOne prelude to the actual JavaOne conference that would follow from the 2nd to the 4th.

The General Session opened with Dave Douglas (Senior VP, Cloud Computing and Chief Sustainability Officer at Sun) anchoring the event and introducing other speakers. The major focus of the morning's General Session was Cloud Computing, with a secondary emphasis on OpenSolaris.

Lew Tucker, Sun's Cloud Computing CTO (yes Virginia, there is such a title), came up on stage with Dave and the two had a conversation in which they described the "Sun Cloud" (a meteorological contradiction in terms, but never mind).

I was glad to see that they actually called out an important point that usually tends to be glossed over at events like this - Open Source is not enough, it's open APIs and interoperability that are really important. Hear, hear! Obviously, we're a way away from having interoperable clouds, because most first movers seem to want to lock in users even as they tout the Open Source underpinnings of their solutions.

Lew Tucker demonstrated what I thought was a very cool tool - the Sun Cloud Compute Service, or at least the GUI tool that's used to configure it. Using a drag-and-drop interface, it's possible to set up a complete network, with firewalls, switches and all manner of servers, then have that configuration created in the cloud.



The virtual servers can even have the MAC addresses of their VNICs set during this configuration.


I was glad to see that besides OpenSolaris, some of the other server platforms available were Fedora, Ubuntu and CentOS. I've observed that the Linux support subscriptions from Red Hat and Novell/Suse are generally priced in such a way as to make mainframes look cheap. Since I don't believe that mainframes can be cheap, I'm happy about the CentOS, Fedora and Ubuntu server options. Depending upon the Sun Cloud pricing, we may actually see lower costs associated with cloud computing. [But with Oracle's acquisition of Sun, I'm not holding my breath.]

The Sun Cloud Compute Service doesn't seem to be in General Availability mode yet. Perhaps it's just released to a few partners right now. Some of those partners came up on stage to briefly discuss their wares. These were companies like Moonwalk, Vertica and WebAppVM. I didn't have time to digest what these people were doing with the Sun Cloud, but they did seem to derive some benefit from smooth virtualisation.

Someone mentioned an emerging standard for specifying virtual configuration topologies called OVF or some such. I'll need to keep an eye out for it.

Sun is also reportedly working with the Center for Internet Security to develop standards for securing virtual machines. The servers that Sun Cloud will provide on its drag-and drop palette will all be hardened by default.

Some of the tools being offered for cloud computing are Netbeans (for development), Hudson (for continuous integration) and Kenai (for repositories).

There was a panel hosted by Jon Fowler that delved into what's new and cool in OpenSolaris. There does seem to be a fair bit there. Sun Open Storage is based on ZFS. DTrace is another component that's built into the OS core, so everything that runs on OpenSolaris can be dynamically instrumented with very low performance impact. Project Crossbow is another initiative that deals with network virtualisation. Compared to Linux, OpenSolaris appears to have a few advantages, such as networking efficiency (OpenSolaris can continue to process network traffic in polling mode even when the CPU is maxed out, for example). There was also some discussion about the ground-up support for flash memory. Flash storage ranks between DRAM and disk in both speed and cost and represents a good compromise for certain functions (caching of certain types of data). ZFS can integrate DRAM, flash storage and disk.

My question remains the same as after Sun Developer Day in Sydney - why can't they port these great features to Linux and dump Solaris? We need to move ahead with a single great OS, and Linux is it. Petty sibling rivalry doesn't help.

I then attended two very similar sessions one after the other, that dealt with Metro-based Web Services. The first was by Jonathan Scudder of Identitas, and dealt with securing Web Services using Metro and OpenSSO. The second talk was by Harold Carr of Sun that covered a few other topics such as connecting to Amazon's web services. My takeaway from Scudder's talk was that there are still a number of non-intuitive steps that need to be followed to make Web Services work using Metro and OpenSSO. There were some cool aspects to NetBeans, though. If the developer drags a Web Service operation into the code of a service consumer, it actually expands into a try-catch block with the actual call inside. OpenSSO is configured as an STS (Secure Token Service). I wonder if CAS can be similarly configured. I think I remember hearing that CAS 4 has a lot of such goodies. Also from Scudder's demo, I realise that WireShark is a neat tool to analyse network traffic. Harold Carr's talk convinced me that the time is not yet ripe to try and integrate Amazon's Web Services into one's application. It appears that Amazon does not follow the WS-* standards with respect to specifying WS-Security. There's no use of WS-SecurityPolicy in Amazon's WSDL, and that sucks in my opinion. I'll wait for Amazon to fall in line with industry best practice before I go anywhere near their services.

I then attended a talk on "Developing RESTful Web Services with JAX-RS and Jersey" by Marc Hadley and Paul Sandoz of Sun. I had attended a similar talk before, so there wasn't much that was new. I was just struck by how glacial the pace of development has been. I have been following JSR 311 for the past two years and I'm disappointed that we aren't much further ahead.

A few things Hadley and Sandoz said did strike me as significant, though.

One is that the venerable "web.xml" file will be obsoleted in the Servlet 3.0 spec.

Another is that content negotiation, which is a big part of REST, is coded for very simply with @Produces and @Consumes annotations on methods. Jersey (i.e., the JAX-RS implementation) takes care of the gory details.

There's a difference between "JAX-RS aware" and "non JAX-RS aware" containers. With the latter, the URI must point to a servlet, and an init parameter must point to an Application class. In the former, the URI can directly point to an Application class.

No new technology is complete without its share of horror, and with JAX-RS, the horror seems to be Web Beans (JSR 299). This is of course the hideous combination of JSF and EJB (two monstrosities that have no right to exist independently, and certainly no right to exist as a combination). I'm not quite sure where exactly in JAX-RS these Web Beans are going to raise their ugly heads, but I'm already shuddering.

There are apparently many implementations of JAX-RS in the works, and they are Jersey, RESTlets, JBoss RESTEasy, Apache CXF, Triaxrs and Apache Wink.

In JAX-RS 2.0, which is currently underway, the roadmap includes a client API, "quality of source" (some content formats such as XML are superior to others and should be used if the client is indifferent between them), form data and MIME multipart, declarative hyperlinking and representation templating.

The final session I attended on the first day was a hands-on workshop that dealt with the use of an Eclipse toolset for Glassfish. This was a fairly good session with well thought-out materials (a DVD with all required software and examples, and a printout of the assignments) and helpful instructors. It was a "Bring Your Own Laptop" workshop, and fortunately the lab had power outlets and Ethernet cables for all. I had whinged about the lack of Eclipse tools for Glassfish on some earlier occasion, and my prayers were evidently heard. I didn't find the Eclipse toolkit for Glassfish the most intuitive possible, but it's perhaps a function of learning. I shouldn't criticise it before I learn to use it well.

I look forward to tomorrow, when JavaOne proper begins.

A Cool Tool (kwout)

There's a new tool called "kwout" that makes it extremely easy to cut out parts of web pages as images and use them to "quote" the sites. Installing it on Firefox is as easy as dragging a hyperlink to the toolbar. I used Kwout to cut its own logo out of its web page (above). Pasting it into Blogger took only seconds.

Check it out.

Advice to Sun from Michelangelo

Sun is a great company with some great software, yet something holds it back from achieving its true potential. What is it that's missing? I've been thinking about this ever since I got back from Sun Developer Day earlier this week.

And then I got it. Nothing's missing. On the contrary, there's too much.

Michelangelo defined sculpture as the art of "taking away", not "adding on". The sculpture already exists inside the block of marble. The sculptor only has to take away the excess to reveal the work of art within.

Sun has a full line-up of extremely useful products. But there's some superfluous marble covering the work of art within. That's what prevents us from seeing the masterpiece. It's an unfinished work, and the sculptor's unwarranted sentimentality towards superfluous marble is what prevents him from completing the statue.

The biggest piece that Sun needs to sculpt away is OpenSolaris. No, really, the world doesn't need another Open Source Unix. Now that we have Linux, that's more than enough. Let's not get into bureaucratic arguments over which is the more senior Unix. Put bluntly, OpenSolaris is a dog-in-the-manger that just muddies the waters. It will limp along, delaying Linux's inevitable triumph, yet never quite succeeding, and in the process, never letting Sun achieve greatness. To quote from one of Sun's own marketing campaigns, they need to put "all the wood behind one arrowhead." An arrowhead called Linux. Sun should help port ZFS and DTrace to Linux, and quietly bury OpenSolaris.

But OpenSolaris isn't the only offending piece. There's the GlassFish app server. Mind you, there's a lot that's cool in GlassFish, such as Jersey (for REST services) and Metro (for .NET-compatible SOAP/WS-* Web Services). But these are components that should simply be contributed to Tomcat as libraries. We don't need an app server when we have Spring and Tomcat. By clinging to a heavyweight EJB container, Sun is refusing to chip away at one of the ugliest pieces of marble obscuring Enterprise Java. (There's no such thing as a lightweight EJB container, by the way. The EJB spec is heavyweight, even with the figleaf of annotations.)

The last bit is NetBeans. NetBeans is all right by itself as an IDE, but the world already has one - Eclipse. Why doesn't Sun throw in the towel already? If they have something to contribute in IDE land, let it be in the form of Eclipse plugins! That's what the rest of the world does.

There's altogether too much "invented here" sentiment that prevents Sun from ruthlessly pruning its inventory and letting elegance shine.

So much for taking away. Moving on from sculpture to painting, which is all about "adding on", there's some stuff Sun is neglecting, too.

I think they're wasting time with MySQL when they should just leave it alone. MySQL is doing just fine by itself. It's an OK database for a lot of people, but Sun's energies are better spent improving another one. Let me beat the drum once more for my favourite Open Source database, which seems to me to be the world's best-kept secret - Ingres.

Sun should contribute to Ingres. I believe Ingres is near-perfect, with just one fatal flaw. It requires the installer to understand sophisticated database concepts before even installing it. If you don't understand why the database snapshot and journal files should be on a separate disk from the database itself, then you've missed the whole point about database recoverability and also messed up your install. Sun should contribute to the usability of Ingres, just as they poured millions of dollars into Star Division GmbH's software to make it a friendly office suite known as OpenOffice 3. My mouth waters at the thought of a rejuvenated Ingres, more beginner-friendly and riding as a package on board the best distribution channel in the world - a Linux installation CD.

But fat chance of that happening now that Oracle has its hooks into Sun :-(.

On second thoughts, forget painting. If Sun can just be a good sculptor and chip away at extra marble, they can still contribute to that masterpiece - a lean and clean Enterprise Java ecosystem that is also Open Source.

Will Windows Become a Drain on Microsoft?

A friend just pointed me to the latest blog posting of Sun's CEO Jonathan Schwartz. Titled "Will the Java Platform Create the World's Largest App Store?", the post reveals a side of the Java platform I hadn't much thought about. I suspect not many people are aware of the revenue model that the Java runtime has created for Sun. Google and Yahoo! (I reckon they're the companies Schwartz refers to) obviously find it worthwhile to pay Sun for the opportunity to reach out to the billion users of the Java runtime.

Three observations I can make immediately:

1. Microsoft ironically did Sun a favour by trying to corrupt Java in the mid-nineties. This caused Sun to bypass Microsoft and go direct to the user's computer. That's what has now resulted in the happy situation of Sun being able to negotiate effectively with the search giants without having to cut Microsoft in on the deal.

2. Oracle probably has a financial reason to buy Sun after all :-).

3. I can now understand another motivation for Google to innovate Android. With Android, Google can be in the position that Sun now occupies as gatekeeper to a billion users' eyeballs. Like Sun cut Microsoft out of the negotiation, Google can cut Sun out with a simpler licensing deal (for the Java VM) and lock onto a growing revenue stream instead. But they may have to share their profits with the owners of the hardware platform.

This line of thinking leads me to a conclusion that is very bad news for Microsoft. If I was an executive at Nokia, I would be talking to Google about getting a share of the ad revenues that Google will surely get through widespread penetration of Android. Armed with a likely deal of that nature, I would then approach Microsoft to work out something similar. Microsoft will probably be in for a shock. Rather than be able to charge hardware vendors for the privilege of licensing Windows, they would be asked to pay rents to those vendors (and their telco partners) for the privilege of reaching millions of customer eyeballs. If Microsoft doesn't play ball, the phone vendors can simply switch to Android. It's not like the PC platform where users have been dog-trained to demand Windows. Even a zero-licence fee Windows won't be good enough in the mobile device market.

I read an article recently that speculated Microsoft was cutting Windows licence fees to the bone to make it viable on Netbooks, and the article then went on to say it was no wonder Microsoft was shedding staff. Now, if OEMs start expecting Microsoft to pay for them to use Windows, the job losses at Redmond will only mount.

All because of a little operating system called Linux, and an open platform called Java (that together go to make up the base platform for Android).

It's wonderful what a bit of competition will do.

Sun Developer Day Sydney (19 May 2009)

I attended Sun Developer Day in Sydney today. Compared to the three-day extravaganza of last year, this was a much simpler affair, with just eight short sessions over a single day.

The sessions were as follows:

1. Keynote (Reginald Hutcherson)
2. JavaFX (Angela Caicedo)
3. Java, Dynamic Languages and Domain-Specific Languages (Lee Chuk Munn)
4. MySQL (Peter Karlsson)
5. OpenESB (Lee Chuk Munn)
6. Virtualization (Peter Karlsson)
7. Developing and Deploying apps with Java SE 6 Update 10 (Angela Caicedo)
8. DTrace (Peter Karlsson)

I'll organise my feedback grouping the topics by speaker, since they specialise in related fields.

Reginald Hutcherson is "Director, Technology Outreach" at Sun and seems to have been with the company forever. I remember hearing him speak at a Sun event more than 10 years ago. He provided an overview of the topics that would be covered over the rest of the day. The key takeaway from his speech was the central role of the Java Virtual Machine. On one side, there are multiple languages (including the increasingly popular scripting languages) that are capable of running on the JVM. On the other side, the JVM has been ported to "all the screens of our life" - computers, TVs, mobile phones, GPS devices, etc. Java the platform, as has always been emphasised by Sun, is more important than Java the language.

Angela Caicedo spoke about JavaFX and later about new options to develop and deploy applications after the innovations in Java 6 update 10. I didn't get the impression that anything very major had occurred in the JavaFX world since last year. My reaction to all these RIA tools (Flash/Flex, Silverlight and JavaFX) is, why bother? JavaScript-based tools are becoming so much more powerful these days that we can build astoundingly rich applications using nothing more heavyweight than JavaScript and HTML. So I don't know where JavaFX will go. I commented last year after Jim Weaver's demo that the declarative JavaFX code quickly ends up looking like the dog's breakfast. I haven't seen anything this year to change my opinion of JavaFX.

[Out of curiosity, I tried downloading NetBeans 6.5 later in the evening and was more than a little disappointed to see that it had no JavaFX support on Linux, only on Windows. Why not? As an Ubuntu user, I'm offended that this excellent platform isn't deemed a first-class development environment.]

Angela's second talk (on the latest options for developing and deploying applications since Java 6 update 10) was, in my opinion, the best session of the day.

Minor revisions (called updates) don't break APIs, but update 10 is still quite revolutionary.

Where do I begin? First, my personal background - unlike most Java developers in the industry today (who build web applications), I have actually built Swing, Applet and Java WebStart-based applications for a number of organisations. And I've been personally disappointed in the fact that these technologies did not become more popular.

With Java 6u10, perhaps some of the reasons for their relative unpopularity have gone away (but it may be too late to get the developers back).

In a nutshell, Sun has now unified the development and deployment of applets and Java WebStart applications, cleaned up the browser-JVM architecture and improved the bidirectional interoperability between Java and JavaScript. I can only ask Sun, where have you been these ten long years?

JVMs no longer have to share process space with the browser, and different applets can run in different JVMs if required. This reduces the impact of an applet crash on other applets and on the browser. Note this: the crash of an applet no longer results in the browser crashing, but more importantly, one can close the browser and have an applet continue to run. This is big stuff. When one then tries to close the applet, one is prompted with the choice to save the applet as a desktop icon. In other words, it's no different from Java WebStart. This is really powerful and cool.

There are a number of other improvements and optimisations. The JRE is now broken into a number of smaller components, and a more lazy download strategy is employed under the covers to reduce the startup time of applications. There is a more modern look and feel to replace the dated Metal appearance. Components are now rendered using scalable vector graphics rather than bitmaps.

I'm most excited about the three major improvements I listed earlier. I can see a very elegant way to implement SOFEA using these innovations. The MVC controller on the client should be an applet, running in a persistent JVM outside the browser process. The Application Download will start the applet and configure it with the wiring logic for the application's Presentation Flow. The controller will handle not only Presentation Flow but also Data Interchange with services hosted by remote servers. Perhaps a variant of Spring Web Flow suitably modified for the client side will be a good model for Presentation Flow. Server-side UI resources in the form of Freemarker or other templates could be pulled by the controller and populated using appropriate models. With the seamless interaction between Java and JavaScript, many things are now possible.

Lee Chuk Munn spoke about Java, Dynamic Languages and Domain-Specific Languages and in a later session, about OpenESB.

There wasn't anything very new in the session on scripting languages, but it may be worth repeating the benefits of scripting.

Scripting helps build an ecosystem for an application. Applications become platforms. Witness plugins for Firefox. Firefox is no longer just an application. It is a platform by itself, because of the JavaScript-based plugins that it supports. Scripting also enables domain-specific applications, such as Mathematica.

[I hadn't heard of Lua before. This is a scripting language for gaming.]

In his second session, Lee worked through an example to demonstrate the features of OpenESB. I must confess I'm a bit of an ESB skeptic. To be fair, the ESB feature that was demonstrated was plain BPEL functionality, so it wasn't so much an ESB as a BPEL engine. I have no objections to BPEL engines as long as they don't pretend to be "SOA fabric in a box", which is my grouse against ESB products.

The main feature of OpenESB as a BPEL engine seems to be its support for JBI (Java Business Integration). Put bluntly, JBI seems to mean a WSDL-defined interface on both the service consumption side and the service provision side. Lee demonstrated how a process could be graphically mapped using NetBeans. The WSDL on the service provision side made the entire process look like a standard SOAP service. On the other side, the actual implementation of business logic was encapsulated in a Stateless Session EJB (ugh!) The interface to the EJB was once again a WSDL. The demo worked quite well, showing not only how logic could be placed in components but also in the process definition itself. However, I fear that powerful tools that simplify service and process development, when placed in the hands of developers without a sufficiently "SOAphisticated" understanding of concepts, will lead to horribly designed applications. (And why are we still wasting our time with EJBs?)

Peter Karlsson gave three talks, one on MySQL, one on virtualisation and one on DTrace. Unfortunately, his areas of specialisation (other than databases) are not of great interest to me personally, so I'll just give a brief run-down of what he covered.

My biggest takeaway from the MySQL talk was that MySQL supports multiple "storage engines", each optimised for a different task. In the same database, one can allocate different storage engines to different tables depending on usage. Examples:

MyISAM is useful for tables that don't require transactional integrity. This is the default, and is most useful for web and data warehouse applications.
InnoDB should be used for tables that do require transactional integrity, but because of the extra performance overhead, should not be used for other tables. Slashdot, Google, Yahoo! and Facebook are InnoDB users.
Archive should be used for log and audit trail type tables, because of its native compression and high insert performance.
In-memory should be used for scratch tables (working storage).

I was a bit confused during the virtualisation talk because there were some generic virtualisation techniques being described, but also some Sun products, and I wasn't always able to tell them apart.

There are four broad categories of virtualisation:
1. Hard partitions (e.g., Dynamic System Domains).
2. Virtual Machines (e.g., Logical Domains, Solaris xVM x86 hypervisors, VirtualBox).
3. OS Virtualisation (Solaris Containers, Solaris Trusted Extensions, Brand Z Solaris Containers, Trusted VirtualBox).
4. Resource Management (Solaris Resource Manager).

My takeaway from this was that VirtualBox was something worth looking into as a developer. This is a Sun product that is Open Source.

Peter Karlsson's last talk was on DTrace, in which I have not the slightest interest. This is a Solaris-based system tool, I understand, and provides relatively low-overhead diagnostic capability. It is reportedly safe for use in production environments because it is supposedly zero-overhead (though Peter did warn about some specific settings that had high performance impact), because it cannot change data, because it cannot cause crashes and because it eliminates the need for post-processing of data.

There are Probes and Providers and a scripting language called D. Traces can be set on activities using a predicate-based expression language. The approach reminded me of AOP (Aspect-Oriented Programming).

Overall, I was satisfied with the coverage of the day's topics, but I have a lingering feeling that the action in the Java world has moved away from Sun.

NetBeans => Eclipse
GlassFish => Tomcat
EJB => Spring POJOs
ESB, SOAP => REST
JavaFX => JavaScript/Ajax libraries (AJAX is now so common it's now spelt Ajax)

It'd be good if Sun could drag itself closer to where the action is, instead of seeming to play in its own little sandpit.

I'll be at JavaOne in two weeks, and I'll report on my impressions there.

Intel's Turn at Monopoly

It's about time. I've been wondering where all the competition watchdogs have been the last few years. Surely the recent dominance of Intel-based computers over AMD wasn't due to product superiority!

At last, the European Union has woken up and taken Intel to task for abusing its monopoly. And how! 1 billion Euros should hurt even a company raking in monopoly profits.

I hope that with increasing regulatory oversight, prices of hardware and software will fall even lower.

Microsoft had better take note, because there's surely a similar level of monopoly arm-twisting going on in the Netbook market. At price points like $300, there's no way a Windows Netbook can be cheaper than a Linux-based one when taking into account OS licence fees. In a competitive market, OEMs would be scrambling to shave off every possible dollar from the price of their offerings. Yet, it's very rare to find a Linux-based Netbook even today. Surely it's not the lack of user-friendliness of Linux. Check out Ubuntu today. Surely it's not the lack of applications for Linux. Take the most popular application, the office suite. Linux comes bundled with OpenOffice. You have to pay extra for MS-Office. How do Windows-based Netbooks even sell? How? How??

I'm waiting for the new US administration to follow the EU in returning to a tough stand against monopolies.

When the playing field is level and fair, I believe we will soon see $50 Netbooks running either Intel or AMD chips. And Microsoft will go bankrupt.

The Next Target for SpringSource?

I'm convinced there is a new powerhouse emerging in the Java enterprise domain, now that the Sun has virtually set. SpringSource is rapidly entering the radar of industry watchers with its recent acquisitions. These acquisitions extend its reach outwards from its core strength (application frameworks), both downwards (the Apache products through the Covalent acquisition) and sideways (the acquisition of G2One and the Groovy/Grails toolset, Hyperic and its enterprise monitoring tools). While important, these acquisitions have all been rather technology-focused and limited in mainstream commercial potential. Now logic demands that SpringSource move up the stack into userland.

Up the stack, there are plenty of soft, juicy targets ripe for the plucking.

Among the softest targets is Enterprise Content Management. Gartner's Magic Quadrant has the usual suspects in it: IBM, Oracle and Microsoft, plus EMC and OpenText. None of them is particularly known for affordable pricing, except perhaps Microsoft (which is known for predatory pricing - killing off competitors before turning on the customer.)

I believe there is pent-up demand (especially in this economic climate) for an affordable ECM product that provides Document Management, Records Management, Image Management, Web Content Management and Collaboration.

There are many Open Source products in this category - Joomla!, Drupal, Plone and Alfresco. If SpringSource acquires (or otherwise takes over the stewardship of) one of these products and gears itself up to offer comprehensive support, training and consultancy around it, they could enter the big league.

The SharePoint phenomenon is rich in lessons to be learnt. By all accounts, SharePoint is a bear to install and maintain. Yet it's growing like a weed in thousands of organisations worldwide. A reasonably-priced alternative with strong support can carve a good chunk out of this market. Plus, my technical suggestion (putting on my architect's hat) is for this alternative to ensure adherence to the CMIS standard. CMIS is still only a proposal, but Alfresco is already implementing support for it, I understand. (It'll be a great coup to be able to tout AtomPub as the interface to content :-). This isn't just a nerdy bragging right. Integration costs are probably the biggest component in any rollout, and integration should be a breeze with the RESTian AtomPub interface.

My only caution, as before, is that SpringSource should be careful not to get too greedy. The customer is the goose that lays the golden eggs. If SpringSource scrupulously respects the freedom of its customers (including the freedom to shop elsewhere for the same products), they'll definitely see more business flowing their way.

The Unsung Wonder Called DHCP

Some of the best features of the Internet are unglamorous. DNS, for example. We would be typing in raw IP addresses into our browsers and keeping personal directories of our favourite websites' IP addresses without DNS to derive them from more human-friendly domain names.

I realised the worth of another such humble Internet protocol last week, when I helped a friend install Ubuntu onto his laptop. His motive was fairly standard and increasingly common - growing exasperation with the number of viruses and malware that regularly infected his Windows laptop.

At any rate, what amazed me was not the ease with which I was able to install Linux on a machine. Ubuntu's friendliness and speed of installation (under 30 minutes) no longer surprise me, although they're still pretty impressive.

He brought his laptop to my house and I hooked it up to my network when I installed Ubuntu 9.04. Ubuntu quietly detected the network, reached out into the Internet and downloaded all the updates that had been released after the release of 9.04, then announced that it was all done. So far, nothing extraordinary (although it's a poke in the eye to those who claim Linux still isn't friendly enough or "there yet" - try installing Windows with as much ease.)

The real wonder was when he went home with his laptop, plugged it into his network and booted it up. Everything worked straightaway. He was online immediately.

It was then that the power of DHCP (Dynamic Host Configuration Protocol) struck me with full force. I didn't have to configure his laptop's IP address. DHCP allowed the laptop to pull its IP address from a server on the network. In the case of both his home network and mine, the DHCP server was the cable modem/router.

This episode reinforced two important lessons for me:

1. Open standards are very powerful. DHCP is an open protocol owned by no proprietary vendor. The manufacturer of his router and mine just don't matter. Mine's NetGear. I know it because I happened to look at the name on the box. I didn't have to know it in order to set up anything. I still don't know what make his router is. And everything still works.

2. If it wasn't already blindingly obvious, loose coupling is perhaps the greatest factor that facilitates ease of integration. DHCP makes it unnecessary to configure IP addresses statically. What if his private IP network is 192.168.0.x and mine is 10.x.x.x? It just doesn't matter. The D in DHCP stands for "Dynamic". The "late binding" that it allows makes all the difference between a smooth installation and one with frustrating setup and numerous support calls.

Though I'm a tech-head who would normally be expected to favour "cool" technology, I'm beginning to think technology actually needs to be boring in order to work well. Boring is the new cool.

Open Source's Assault on the Enterprise Data Centre

This week's announcement of SpringSource's acquisition of Hyperic, a maker of Open Source enterprise monitoring software, made me think about a number of things.

1. What does this mean for the prospects of Open Source within enterprises, especially in data centres?
2. What are the prospects for SpringSource as a company? Crucially, what are SpringSource's strategic intentions?
3. What do these trends mean for customers, users and the community as a whole?

I keep reading about how Open Source's prospects have paradoxically brightened thanks to the deepening economic gloom. For all the wrong reasons, enterprises are now looking to Open Source. I say their reasons are wrong because short-term cost savings in the form of lower licence fees are not the major advantage of using Open Source. The opportunity to control their own destiny, free from coercion by vendor agendas, is the greatest benefit, and this can deliver savings far in excess of the measly amounts that licence fees alone would suggest.

I believe that the increasing profile of Open Source in the enterprise software stack is a good thing for user organisations, with some qualifications that I will come to later. I have believed for some time, based on my experiences in large commercial organisations, that proprietary software (and hardware) companies manage to earn a hefty premium from corporate customers on the basis of benefits that are more mythical than real. There is a certain type of corporate decision-making animal that wants to be convinced and comforted by the brand shorthand of large vendors that spell safety, reliability, and a general "enterprise classness". Thus a WebSphere "has to be" better than a Tomcat, an Enterprise Service Bus "has to be" better than simple REST, and an AIX, HP-UX or Solaris "has to be" better than Linux. They cost so much more, so they've got to be better. Cheaper alternatives damn themselves by being cheaper.

How can this mentality be overcome?

It's instructive to consider the hardware market. The most dangerous stealth assault weapon in hardware is the Intel platform. I made the mistake of underestimating the potential of Intel in the early nineties. Those who worked in the industry in the nineties would remember a product category called the Unix workstation. Unix workstations were desktop machines used by the elite in the IT industry. Compared to Unix workstations, Intel PCs were woefully underpowered apologies for computing platforms. Lowly, second-rate developers wrote applications on PCs using pathetic tools like FoxPro. Real developers (like me) built high-end applications using real tools like Oracle Forms on workstations.

Well, the bottomline is that today's generation of developers has probably never heard of a Unix workstation. The puny, pathetic PC platform, like an imperceptibly rising tide, completely swamped its high-end cousin. The tables turned swiftly. "Underpowered" no longer was. That change swiftly repositioned the alternative as "overpriced". End of story.

Today, the Intel assault continues, this time against big iron in the data centre. I'm frankly skeptical about the value proposition of big iron. Intel power and capability are constantly rising, and its price-performance trend continues unabated. The ability of Intel to virtualise is changing the game rather rapidly. The argument in the data centre, like on the desktop, will shift from "underpowered" to "overpriced" in short order, and big iron will be out. Big Unix will go first, followed by mainframes when organisations can wean themselves off their legacy mainframe applications.

But as on the desktop, the Intel platform cannot prevail against the incumbent without the help of software. And that's where the SpringSource acquisition drops a piece of the jigsaw puzzle into place. Hyperic now challenges enterprise monitoring tools like IBM Tivoli, HP OpenView and BMC Patrol, providing similar capability at a fraction of the price. The pattern is familiar.

For a while, corporate decision-makers will hesitate, reluctant to abandon the comfort blanket of familiar brand names from the big end of town. But I suspect the growing pressure to find cost reduction opportunities in the current climate will force many of them out of their comfort zone. Many of them will sample Open Source. Most, I suspect, will like what they find, once they overcome their own mental roadblocks. [I have never ceased to be amazed at the reluctance of customers of software to choose options that will give them control of their own destiny. No wonder Marx had to exhort workers to unite with the reminder that they had nothing to lose but their chains. Humans seem to prefer the comfort of familiar bondage to unfamiliar freedom. It's the Matrix's choice of red pill (harsh reality) and blue pill (comfortable illusion). Many actually prefer the blue pill. It's only when the blue pill is no longer an option will they try the red pill.]

But that brings me to the qualification I expressed earlier. I support the notion of Commercial Open Source as long as certain principles are respected, the main one being the ability of users to control their own destiny. SpringSource and its increasingly powerful stack of products (Tomcat or tc Server, the Spring framework and now Hyperic) are most definitely liberators when compared to the gouging data centre vendors that we all know and (don't) love. But will SpringSource stay faithful to the Open Source social contract?

I believe that software must be collaboratively developed to amortise the costs of development among a number of developers and thereby reduce the requirement to show a return on investment on the core product. That is the fundamental difference between the Open Source development model and the proprietary one. Proprietary software is forced to show a return in the form of per-copy licence fees because it is funded by an investment model. Open Source is not, because its funding can follow an expense model with development costs written off, or amortised. Thus Open Source appears "fairer" because it doesn't charge "rents" on copies of software whose marginal cost of production is actually zero. Commercial Open Source carries this model to its logical conclusion by only charging for non-repeatable services that cannot be stored and replicated, such as training, consultancy and support.

So far, SpringSource as a company has followed the book on Commercial Open Source. Their products tend to be freely downloadable (with some exceptions such as their "enterprise" versions), and organisations can run them without having to pay either licence fees or support fees if they can also maintain them in-house. There is sufficient documentation available to give users control of their own destiny, which is the crucial benefit of Open Source.

My fear is about the future. Will SpringSource continue to maintain a benign pay-for-services model on top of genuinely Open Source products, or will they go the way of so many others before them (like Compiere), and be tempted to convert their stewardship of a powerful stack of enterprise software into a model with tighter control and less user freedom? I sincerely hope they can avoid the temptation of greed, because the long-term benefits to them as well as the rest of the industry are much higher if they retain a light touch. A tighter grip will make their users want to wriggle free. Witness the challenges of Adempiere to Compiere, MariaDB to MySQL and CentOS to Red Hat Linux. If a commercial entity seems to wield an unhealthy degree of influence on an Open Source product, the tendency to fork the project will increase, to the detriment of all (the company more than the community, ultimately). It's a bit worrisome that only SpringSource employees have commit rights to their software. A more inclusive and meritocratic development community around products under SpringSource's stewardship would allay user apprehensions and pre-empt such forking.

This month's Indian elections underline my point. How could an incredibly diverse country of over a billion people have stayed united and largely peaceful for over sixty years? Only by giving those people genuine control over their own destiny through democracy. Much smaller and more ethnically homogeneous countries have been less successful at this.

In future, SpringSource could attain sufficient gravitational mass to pull the stewardship of the Java platform itself into their orbit. If they can avoid the temptation to tighten their grip, "monetise" their influence or otherwise break faith with the community, they can look forward to a (gradual) rise to a position of pre-eminence in the enterprise software market.

The current big names in the enterprise data centre have the most to fear from this development. If SpringSource is wise, the enterprise market is theirs. Customers will gladly give them their business in return for guaranteed control of their destiny.

Ubuntu 9.04 - First Impressions

I just downloaded Ubuntu 9.04 and installed it on an old laptop (Mobile Pentium 4, 2.2 GHz CPU, 1 GB RAM, 35 GB disk). The first thing that amazed me was the sheer speed of the install - all of 21 minutes! Of course, the machine wasn't connected to the network, so maybe a couple of extra steps got skipped, but still, this was awesome. I've never seen any distribution (not even previous versions of Ubuntu) install so quickly. This could be a huge selling point.

I can't find much to report on, in terms of the differences between this version and previous ones. It's as good as I've come to expect from Ubuntu.

There was just one minor disappointment, though. I found a new tool under System -> Administration called "Computer Janitor", but when I tried to run it, I got an error message to the effect that "dash" (whatever that may be) was not installed, and the application terminated. If I'd been connected to the Internet, I guess I could have done an 'apt-get install dash' or used Synaptic to install the missing component, but in standalone mode, there wasn't much I could do.

[Update: I've done two more installs since then, both while connected to the Internet, and Computer Janitor works fine on both. Maybe there was a required update that only gets installed when the machine is online.]

Other than that little hiccup, everything seemed fine - a fairly standard Ubuntu system. I'm planning to get a new laptop before I go off to attend JavaOne in June, and it's sure to be nippier than this one which is 5 years old, so Ubuntu 9.04 should be even more impressive. I will be relying on its wireless connectivity, and using it as a workstation-cum-server running lots of heavy-duty Java apps, so stay tuned for further reports next month. I wonder if it will install in less than 21 minutes on a faster machine...

Adventures with Android - 1

Since I'm so convinced that Android is going to take over the world ;-), I decided to walk the talk and try some Android development.

My desktop machine runs Ubuntu 8.10 (Intrepid Ibex) and I have OpenJDK 1.6.0_0-b12 installed. Ubuntu 8.10 doesn't give you the latest version of Eclipse (3.4 - Ganymede), so I downloaded and installed it separately.

Then I pointed Eclipse to the Android SDK plugin site and installed both the Android Development Tools and Android Editors. I have no idea what Android Editors are required for, but bandwidth is cheap, so what the hell...

I have no idea why, but I also downloaded the Linux version of the Android SDK separately. (Maybe I downloaded this before the Eclipse plugin, I don't remember.) My standard policy is to install all software under /usr/local with the exploded directory having the full version number, then to create a symbolic link without a version number, which points to the latest directory.

i.e., /usr/local/android -> /usr/local/android-1.1r1

[This way, I can keep multiple versions of software installed simultaneously, while pointing to the latest version through the symbolic link. If required, I'll use the symbolic link in PATH and CLASSPATH variables, so I don't have to change them when I upgrade or downgrade versions. The power of symbolic links...
Unix was pretty cool even twenty years ago :-).]

If I now open a terminal window and go to /usr/local/android/tools, I can execute the Android emulator by typing "./emulator". Be warned that even though the emulator comes up right away, it takes a few minutes for it to fully initialise and display its beautiful mobile screen.


I played around with the emulator for a while. This is pretty cool. The apps actually work. I checked out the browser. I could see my own blog. Then I went to Google Maps and saw my own house. It's pretty eerie, sitting inside my house at a computer running an emulator running a browser that shows a street view of the same house from the outside. MC Escher would have loved that.

Well then, the next step was to follow the tutorial and build my first Hello, World application on Android. The tutorial is designed for Eclipse with the Android plugin. I followed it fairly faithfully and got my first application running.


It was only when I tried to deviate from the script that I ran into trouble. The tutorial advised me to edit the "strings.xml" file in "res/values", but I couldn't see the file in the Java perspective. So I thought I'd create one, but I couldn't because Eclipse told me a file with that name already existed! No problem, I thought naively, let me at least try and create a "strings2.xml". After that, the application simply stopped working. I'm sure it was because of the empty XML file that I had created, but the SDK didn't allow me to even see the file so I could remove it. I don't remember what sequence of steps I followed to finally see the two XML files and remove the offending "strings2.xml". The app started working again, to my relief.

If anyone at Google is reading this, please note that even non-newbie Java developers can find the Android SDK confusing in places.

Anyway, that was my first baby step into the world of Android. I'll continue to blog about this as and when I get time to progress through the tutorials and build more stuff.

Sun in Oracle's Orbit - It Could Have Been Worse

So Oracle has bought Sun.

I read the news with some misgivings. Am I sorrier to hear this news than I was to hear about the earlier rumour of IBM's takeover? It's hard to say.

I would have been happiest if Sun had continued on its own long enough to set Java free (both as a completely defined spec (Java 7) and as a completely free (i.e., GPL-ed) implementation). After that, I really wouldn't have cared what happened to Sun.

Unfortunately, the takeover of Sun before the emancipation of Java doesn't give me a good feeling.

A chill goes through me whenever I think of Microsoft, IBM or Oracle. "Solutions" from these three vendors rarely benefit the customer as much as they benefit Microsoft, IBM and Oracle. Java from IBM? Java from Oracle? Visualise great big dollar signs in your sales rep's eyes. Java is unlikely to be free (as in speech or beer) with either of these rapacious corporations controlling its destiny.

Still, I think the world has escaped with the lesser of two bleak futures. Sun in Oracle's clutches is probably better for all of us than Sun in IBM's.

In software, there are now three powerhouses (Microsoft, IBM and Oracle/Sun) instead of two (Microsoft and IBM/Sun).

In hardware, there are now three powerhouses (IBM, HP and Oracle/Sun) instead of two (IBM/Sun and HP).

As a devotee of a competitive market, I think things could have been worse than what has come to pass, and so I'm relatively grateful.

The takeover of Sun (by either party) is good news for Linux. Oracle was never an OS company, and they have been the corporate world's strongest Linux backers. IBM does support Linux, but only up to the point where it believes AIX should rule. Solaris would have lost out under IBM. I cannot imagine IBM tolerating the threat to AIX. Oracle will probably be opportunistic about both Linux and Solaris, although it will have no particular religion towards Solaris. The market will push Oracle to favour Linux over Solaris in a way that Sun would have been reluctant to do. So I think Solaris is the big loser in the bargain. But it's no real loss. The best parts of Solaris, like ZFS, have already been cannibalised and are used with Linux, so nothing of value has disappeared.

I think the Oracle takeover is paradoxically good for MySQL (the product, not the company). Either Oracle will push MySQL aggressively to block Microsoft SQLServer at the lower end of the market, or Oracle will view MySQL as a competitor to its own flagship product and discourage its use. Either way, MySQL will gain energy. In the latter case, its Open Source version will attract more developers and become more popular. (Thank goodness for the GPL, which prevents Oracle from killing it off!)

In any case, my favourite Open Source databases are Ingres, followed by PostgreSQL. MySQL, to my mind, is a distraction.

Oracle must be suffering J2EE app server indigestion. GlassFish would be their fifth app server, I think. (The previous four being AS, iAS, Orion and WebLogic). Good for all of us that the J2EE/JEE app server market is consolidating (read shrinking) with only a few big players left now (IBM with WebSphere, Oracle with whatever they choose as their flagship from their distended line-up and JBoss, whose market seems to have disappeared at about the same time as Marc Fleury disappeared with his millions). I'm a strong proponent of Spring/Tomcat, so the sooner these dinosaurs die out, the happier I'll be.

I'm not sure what this means for OpenOffice. Both IBM and Oracle would dearly love to poke Microsoft in the eye, though I can't tell who would have been a more effective seller of OpenOffice into the corporate market. With either, I don't believe corporate customers would find OpenOffice any cheaper than MS-Office. Microsoft can actually breathe easier now. Having Oracle push Open Source makes Microsoft sound truthful when it talks about Open Source being more expensive than its own products.

What about Web Services? Will Oracle want to pursue interoperability with Microsoft the way Sun did with Project Tango? One could argue that it really doesn't matter. The increasing popularity of REST is making SOAP-based Web Services less relevant with each passing day. It is only the blindest SOA practitioners, with their heads buried deepest inside corporate caves, who remain ignorant of REST.

Sun has recently developed a strong line of Identity Management products (OpenDS, OpenSSO, etc.) I wonder what Oracle will do with them.

I sometimes wonder about Oracle's strategy. They've bought so many disparate products and forced them into that clumsy box called Oracle Fusion that it's beginning to look like Oracle Confusion.

But the bottomline after all the analysis is, I believe, still Java. Thankfully, most of Java is now GPL-ed, so if Oracle doesn't behave, there should be a strong community push (with IBM's backing, no doubt) to "unencumber" the last few Java libraries and truly set it free.

It's going to be an interesting year.

AMQP Lurches Towards Completion

I really mean the word "lurches" that I used in the title of this post. I've been following the AMQP specification since the 0.8 version, and there have been more dramatic twists and turns in this than in an old Perry Mason novel. (In the latest twist, they seem to have dropped the very concept of an Exchange, a mainstay of previous versions).

I don't believe the latest version of the spec is available (I would dearly love to have a read), but the working group met recently in San Diego, and the materials presented are here.

My overall reactions are these:

1. We really do need an industry-standard messaging protocol to match the capabilities of proprietary products like IBM's MQ and TIBCO's EMS. These two vendors have built their hugely profitable businesses at the expense of customers who have nowhere else to go. The commoditisation that has lowered prices in other areas of distributed computing (TCP/IP for networking, HTTP for web, SMTP for email, etc.) has simply not occurred in enterprise messaging. AMQP aims to rectify that. All I can say is, "It's about time".

2. I note with amusement that Microsoft has joined the AMQP bandwagon. Nothing like a lack of market share to elicit good behaviour from Microsoft ;-)

3. As I said, I don't have access to the latest spec, but I know that an intermediate version had dropped support for streaming communications and file transfer. If these are still not part of the spec, I believe that's a huge mistake. An enterprise messaging protocol must natively support these significant aspects of distributed computing, otherwise the protocol as a whole will fail to pass muster in spite of its excellence in other areas.

4. I don't know what the AMQP Working Group is thinking, but I believe that the main competition for AMQP is not IBM and TIBCO but HTTP and REST. Whenever I talk to REST afficionados about AMQP, the response is a big ho-hum. "What can AMQP do that we can't already do with HTTP?" is the response. Asynchronous notification? Just use XMPP for that. Security? SSL is good enough. And so on. The AMQP Working Group should co-opt a particular breed of REST expert - one who understands the value of resource-orientation without being wedded to HTTP. I believe that if we can define an application protocol on top of the transport protocol that AMQP is (by usefully constraining it with special verbs, headers and status codes), we will have a more capable architectural style than REST, one which includes native support for event notification, end-to-end security, reliable message delivery, transactions, file transfer, streaming communications and process coordination. Rohit Khare's ARRESTED style can become a reality.

Is anyone listening?

The Computing Platform of the Future

What will the computing platform of the future be like? (The consumer-side platform, that is)

It'll be a super-slim, compact and lightweight laptop with some innovative folding smarts that will let people carry it around in their pockets, yet open it up to a regular laptop form factor for ease of use. It will feature several important innovations, some of which are already available, or tending towards a tipping point:

1. Really cheap hardware - $50 for the device that you can pick up at the supermarket and toss into your trolley without a second thought (impulse purchase territory)
2. Absolutely free software either on the device (like OpenOffice) or in the "cloud" (like Gmail)
3. Ubiquitous wireless connectivity (some cities do have 'em)
4. High network bandwidth (Australia is investing in this right now)
5. Breakthrough battery technology that lets you work a regular day without power supply

Needless to say, this device will double as a mobile phone, portable music player, portable video player, SatNav, game console and much else. This "computing platform" will be the enabling innovation that will let many others ride on its back. And it won't be a real innovation so much as a perfect storm of many tipping points being reached at the same time.

The key technology to watch is Android. By standardising and commoditising the core software layer, Android will enable innovation both below it (i.e., hardware) and above it (i.e.,applications) in the stack.

Death of the Directory

The LDAP directory is a strange beast. It's a data store, but it's hierarchical in nature, rather like the old hierarchical databases most of us only read about in database textbooks. Hierarchical models, as we know, have inferior expressive power when compared to network and relational models. So shouldn't LDAP directories suffer from similar limitations?

It turns out that they do have similar limitations, and the industry as a whole has been going along with the convenient illusion that the emperor has clothes.

Talking to some old industry hands, I became gradually convinced of something that I've suspected for a while - that the LDAP directory was an invention that solved a temporary problem. The problem has gone away, but the awkward solution remains with us.

Twenty years ago, relational databases were relatively slow and cumbersome. Even reads were slow. Relational databases were therefore not a good fit for read-mostly use cases.

Enter the directory server. Directory servers were designed to be very fast at lookups. They were abysmally slow at updates, but that didn't matter. They were designed mainly for lookups, not for updates. Systems requiring on-line transaction processing needed relational databases.

Somewhere along the way, relational databases got faster. Much faster. Faster at reads as well. And somewhere there, LDAP directories lost their raison d'etre. Only the world never noticed.

Today, LDAP directories do some things well - a very small number of things. They perform authentication very well. Checking a user's credentials (an encrypted password, for example) is a directory server's bread-and-butter. Plus it can enforce password policies, lock accounts on a certain number of failed attempts, etc. These are all built-in features, whereas relational databases are too general-purpose to be able to do these out of the box.

But relational databases are excellent at modelling complex relationships. Directories suck at this. If we try and shoehorn complex structures into a hierarchical model, we will end up with highly unsatisfactory results. Not only will it be inelegant and difficult to understand, the resulting directory bloat will also slow down the system, negating the performance advantage that prompted the choice of the directory in the first place.

The bottomline is, anyone looking to implement a "directory" today would be better off with a hybrid model. They should use an LDAP directory for the limited tasks which it is good at (authentication), and should use a relational database to model and store all other information. They then gain the best of both worlds. The directory server will hold just a handful of attributes needed for authentication.

Hybrids are all very fine, but how do we bridge between a relational database and an LDAP directory? In other words, what do we use to link a record in a database with the corresponding entry in the directory? One candidate would suggest itself to anyone who has been following this blog lately - the Universally Unique ID (UUID).

Of course, most relational databases don't support a native UUID datatype. And many DBAs are used to having identity columns for tables, which are autogenerated by the DBMS. In such cases, the UUID could be another attribute of the same table with a unique constraint defined on it. In other words, the UUID is another candidate key, and this is the one that is used to link an entity within the database to corresponding attributes in other systems. It is truly universal and unique.

How do we provision databases and directories so that their records are consistent? There may be no theoretically satisfactory answer (such as a two-phase commit transaction across both), but there are a number of real-world solutions that are adequate. Organisations may find that the hybrid data store is a pragmatic approach after all.

UUIDs and 2D Barcodes

I wrote about UUIDs (Universally Unique IDs) earlier and speculated about some of their possible uses. One of them, I thought, would be the ability to link them to real-world entities through corresponding 2D barcodes, which are a cool innovation in their own right.

So this weekend, I spent some time trying to combine both technologies.

Apparently, there is a standard called PDF 417 that defines 2D barcodes. There are Open Source encoders and decoders that implement this standard.

I downloaded the Pdf417lib encoder software from SourceForge. It's just a simple Java class that you compile and run, so it's not particularly complex to use. Of course, it's not the best-documented piece of software around, but then again, I can't look an Open Source horse in the mouth. I just wish I knew what the various parameters were so I could tune them. As it was, I tried fiddling around with some of them and settled for something that sort of worked. The main() method of the class is written to generate a PostScript file corresponding to any text string that is fed to it, so all I had to do was pipe in a fresh UUID as that text string, using the java.util.UUID class and its randomUUID() method. I didn't have the appetite to do much more than prove the concept.

I wrote the following bit of code based on the main() method of the Pdf417lib class itself. This is what my test class looks like:



The critical lines here are:

Pdf417lib pd = new Pdf417lib();
pd.setText( UUID.randomUUID().toString() );

I compiled and ran the class quite simply:

$ javac TestUUIDBarcode.java
$ java TestUUIDBarcode test1.ps
$ ps2pdf test1.ps test1.pdf

The "ps2pdf" command, as can be guessed, converts PostScript files to PDF.

I ran this once more just to check that I wasn't getting the same barcode again. The UUID generation is meant to be random, so I should get a new barcode every time.

Here are two of them. I hope they look different enough.






So that's what UUIDs look like when converted into 2D barcodes. All courtesy a standard JDK function and an Open Source library. During the dot-com era, someone could have started a company based on this idea ;-).

Java's Centre of Gravity Has Shifted

I was amused to read this slant on the rumours of IBM's takeover of Sun.

No, it wasn't industry analysts who were being asked their views on the merger. It was Rod Johnson, original developer of the Spring framework and CEO of SpringSource.

SpringSource isn't an IBM or an Oracle in size. Outside of the most tech-savvy segment of the Java development community, SpringSource and Rod Johnson are virtual unknowns. Yet the press was beating a path to this man's door. That in itself was somewhat surprising. Even more surprising was that Rod's reaction to what should have been major industry news was rather ho-hum. His reasoning was that in recent times, Java middleware has been influenced more by Open Source and independent developers than by large corporations, so it really doesn't matter either way if the merger goes ahead or not.

I tend to agree, with one caveat. The roadmap for an Open Source Java is still murky, and I'd like this to be clarified before I'm willing to yawn at the irrelevant plans of corporate titans. There are those like Stephen Colebourne who have almost a conspiracy theory about Sun's plans for Java. While I hope they're wrong, I too would like to see an open Java 7 specification that can be implemented by anyone and be certified as such.

If I can't have an open Java specification, I'm willing to settle for at least one safely Open Source (i.e., GPL-ed) implementation of Java, but I'm disappointed even there. Java is still not 100% Open Source. It still suffers from "encumbered code". There are a few libraries that have not been released under the GPL, and that is a lingering source of worry.

But in the main, Rod Johnson is right. A few years ago, JBoss began to shift the centre of gravity of Java away from the Weblogics of the J2EE world towards a more lightweight, Open Source implementation of J2EE. Today, that wrenching movement has continued with the marginalisation of J2EE itself.

I believe that Spring and Tomcat are ringing the death knell for heavyweight Java. If Java becomes a 100% open platform with a completely open specification and at least one completely GPL-ed implementation, then it won't matter which industry heavyweight buys which. It will be a mating of dinosaurs in the age of mammals.

The GUIDness of UUID

Microsoft calls it the GUID. The rest of the world calls it the UUID. But regardless of its name, the Universally Unique ID or Globally Unique ID is truly unique.

On the face of it, what's so great about a 128-bit number apart from its length?

Well, plenty. The length, humble as it may seem, crosses a tipping point of sorts, and imparts to the UUID a number of extremely useful properties.

340,282,366,920,938,463,463,374,607,431,768,211,456

That's what 2 to the power 128 is.

What's the probability of two randomly generated 128 bit numbers being the same?

2 to the power -128, or 0.0000000000000000000000000000000000000029.

That's 38 zeroes before the first significant digit. In contrast, the much-vaunted "five nines" reliability of the very best computer systems corresponds to 0.00001.

Now think about it. In the past, if we ever wanted to ensure that two entities were given IDs that had to be unique, the only way to ensure that uniqueness was through a sequence number generator, and one that was maintained at a single point.

Suddenly, the UUID provides a new way. We can randomly generate IDs of this length from any number of independent sources., and they're virtually guaranteed never to conflict! Of course, our pseudorandom number generator had better be cryptographically secure, or all bets are off...

Many languages provide libraries to generate random UUIDs. Java has the utility class java.util.UUID.

One generates a UUID in Java through the static method call UUID.randomUUID(). A toString() will generate a string representation of the UUID, which has 36 characters (32 hex characters plus 4 hyphens at defined places).

E.g., cd6b31ee-a877-41fe-a8f1-87d35d2045a6

This is more compact than the numeric representation and possibly more portable between systems.

There are any number of possible uses for UUIDs, especially when combined with other ingenious innovations like 2D barcodes. Complete identities and properties can be output in barcode format and affixed to real-world entities, providing a ready link to corresponding data in databases.

I've known of the existence of UUIDs for a while, but I'm only now waking up to their potential.

Update 10/09/2009: Here's the regular expression for a UUID:
[0-9A-Fa-f]{8}(-[0-9A-Fa-f]{4}){3}-[0-9A-Fa-f]{12}