Ubuntu 9.04 - First Impressions

I just downloaded Ubuntu 9.04 and installed it on an old laptop (Mobile Pentium 4, 2.2 GHz CPU, 1 GB RAM, 35 GB disk). The first thing that amazed me was the sheer speed of the install - all of 21 minutes! Of course, the machine wasn't connected to the network, so maybe a couple of extra steps got skipped, but still, this was awesome. I've never seen any distribution (not even previous versions of Ubuntu) install so quickly. This could be a huge selling point.

I can't find much to report on, in terms of the differences between this version and previous ones. It's as good as I've come to expect from Ubuntu.

There was just one minor disappointment, though. I found a new tool under System -> Administration called "Computer Janitor", but when I tried to run it, I got an error message to the effect that "dash" (whatever that may be) was not installed, and the application terminated. If I'd been connected to the Internet, I guess I could have done an 'apt-get install dash' or used Synaptic to install the missing component, but in standalone mode, there wasn't much I could do.

[Update: I've done two more installs since then, both while connected to the Internet, and Computer Janitor works fine on both. Maybe there was a required update that only gets installed when the machine is online.]

Other than that little hiccup, everything seemed fine - a fairly standard Ubuntu system. I'm planning to get a new laptop before I go off to attend JavaOne in June, and it's sure to be nippier than this one which is 5 years old, so Ubuntu 9.04 should be even more impressive. I will be relying on its wireless connectivity, and using it as a workstation-cum-server running lots of heavy-duty Java apps, so stay tuned for further reports next month. I wonder if it will install in less than 21 minutes on a faster machine...

Adventures with Android - 1

Since I'm so convinced that Android is going to take over the world ;-), I decided to walk the talk and try some Android development.

My desktop machine runs Ubuntu 8.10 (Intrepid Ibex) and I have OpenJDK 1.6.0_0-b12 installed. Ubuntu 8.10 doesn't give you the latest version of Eclipse (3.4 - Ganymede), so I downloaded and installed it separately.

Then I pointed Eclipse to the Android SDK plugin site and installed both the Android Development Tools and Android Editors. I have no idea what Android Editors are required for, but bandwidth is cheap, so what the hell...

I have no idea why, but I also downloaded the Linux version of the Android SDK separately. (Maybe I downloaded this before the Eclipse plugin, I don't remember.) My standard policy is to install all software under /usr/local with the exploded directory having the full version number, then to create a symbolic link without a version number, which points to the latest directory.

i.e., /usr/local/android -> /usr/local/android-1.1r1

[This way, I can keep multiple versions of software installed simultaneously, while pointing to the latest version through the symbolic link. If required, I'll use the symbolic link in PATH and CLASSPATH variables, so I don't have to change them when I upgrade or downgrade versions. The power of symbolic links...
Unix was pretty cool even twenty years ago :-).]

If I now open a terminal window and go to /usr/local/android/tools, I can execute the Android emulator by typing "./emulator". Be warned that even though the emulator comes up right away, it takes a few minutes for it to fully initialise and display its beautiful mobile screen.


I played around with the emulator for a while. This is pretty cool. The apps actually work. I checked out the browser. I could see my own blog. Then I went to Google Maps and saw my own house. It's pretty eerie, sitting inside my house at a computer running an emulator running a browser that shows a street view of the same house from the outside. MC Escher would have loved that.

Well then, the next step was to follow the tutorial and build my first Hello, World application on Android. The tutorial is designed for Eclipse with the Android plugin. I followed it fairly faithfully and got my first application running.


It was only when I tried to deviate from the script that I ran into trouble. The tutorial advised me to edit the "strings.xml" file in "res/values", but I couldn't see the file in the Java perspective. So I thought I'd create one, but I couldn't because Eclipse told me a file with that name already existed! No problem, I thought naively, let me at least try and create a "strings2.xml". After that, the application simply stopped working. I'm sure it was because of the empty XML file that I had created, but the SDK didn't allow me to even see the file so I could remove it. I don't remember what sequence of steps I followed to finally see the two XML files and remove the offending "strings2.xml". The app started working again, to my relief.

If anyone at Google is reading this, please note that even non-newbie Java developers can find the Android SDK confusing in places.

Anyway, that was my first baby step into the world of Android. I'll continue to blog about this as and when I get time to progress through the tutorials and build more stuff.

Sun in Oracle's Orbit - It Could Have Been Worse

So Oracle has bought Sun.

I read the news with some misgivings. Am I sorrier to hear this news than I was to hear about the earlier rumour of IBM's takeover? It's hard to say.

I would have been happiest if Sun had continued on its own long enough to set Java free (both as a completely defined spec (Java 7) and as a completely free (i.e., GPL-ed) implementation). After that, I really wouldn't have cared what happened to Sun.

Unfortunately, the takeover of Sun before the emancipation of Java doesn't give me a good feeling.

A chill goes through me whenever I think of Microsoft, IBM or Oracle. "Solutions" from these three vendors rarely benefit the customer as much as they benefit Microsoft, IBM and Oracle. Java from IBM? Java from Oracle? Visualise great big dollar signs in your sales rep's eyes. Java is unlikely to be free (as in speech or beer) with either of these rapacious corporations controlling its destiny.

Still, I think the world has escaped with the lesser of two bleak futures. Sun in Oracle's clutches is probably better for all of us than Sun in IBM's.

In software, there are now three powerhouses (Microsoft, IBM and Oracle/Sun) instead of two (Microsoft and IBM/Sun).

In hardware, there are now three powerhouses (IBM, HP and Oracle/Sun) instead of two (IBM/Sun and HP).

As a devotee of a competitive market, I think things could have been worse than what has come to pass, and so I'm relatively grateful.

The takeover of Sun (by either party) is good news for Linux. Oracle was never an OS company, and they have been the corporate world's strongest Linux backers. IBM does support Linux, but only up to the point where it believes AIX should rule. Solaris would have lost out under IBM. I cannot imagine IBM tolerating the threat to AIX. Oracle will probably be opportunistic about both Linux and Solaris, although it will have no particular religion towards Solaris. The market will push Oracle to favour Linux over Solaris in a way that Sun would have been reluctant to do. So I think Solaris is the big loser in the bargain. But it's no real loss. The best parts of Solaris, like ZFS, have already been cannibalised and are used with Linux, so nothing of value has disappeared.

I think the Oracle takeover is paradoxically good for MySQL (the product, not the company). Either Oracle will push MySQL aggressively to block Microsoft SQLServer at the lower end of the market, or Oracle will view MySQL as a competitor to its own flagship product and discourage its use. Either way, MySQL will gain energy. In the latter case, its Open Source version will attract more developers and become more popular. (Thank goodness for the GPL, which prevents Oracle from killing it off!)

In any case, my favourite Open Source databases are Ingres, followed by PostgreSQL. MySQL, to my mind, is a distraction.

Oracle must be suffering J2EE app server indigestion. GlassFish would be their fifth app server, I think. (The previous four being AS, iAS, Orion and WebLogic). Good for all of us that the J2EE/JEE app server market is consolidating (read shrinking) with only a few big players left now (IBM with WebSphere, Oracle with whatever they choose as their flagship from their distended line-up and JBoss, whose market seems to have disappeared at about the same time as Marc Fleury disappeared with his millions). I'm a strong proponent of Spring/Tomcat, so the sooner these dinosaurs die out, the happier I'll be.

I'm not sure what this means for OpenOffice. Both IBM and Oracle would dearly love to poke Microsoft in the eye, though I can't tell who would have been a more effective seller of OpenOffice into the corporate market. With either, I don't believe corporate customers would find OpenOffice any cheaper than MS-Office. Microsoft can actually breathe easier now. Having Oracle push Open Source makes Microsoft sound truthful when it talks about Open Source being more expensive than its own products.

What about Web Services? Will Oracle want to pursue interoperability with Microsoft the way Sun did with Project Tango? One could argue that it really doesn't matter. The increasing popularity of REST is making SOAP-based Web Services less relevant with each passing day. It is only the blindest SOA practitioners, with their heads buried deepest inside corporate caves, who remain ignorant of REST.

Sun has recently developed a strong line of Identity Management products (OpenDS, OpenSSO, etc.) I wonder what Oracle will do with them.

I sometimes wonder about Oracle's strategy. They've bought so many disparate products and forced them into that clumsy box called Oracle Fusion that it's beginning to look like Oracle Confusion.

But the bottomline after all the analysis is, I believe, still Java. Thankfully, most of Java is now GPL-ed, so if Oracle doesn't behave, there should be a strong community push (with IBM's backing, no doubt) to "unencumber" the last few Java libraries and truly set it free.

It's going to be an interesting year.

AMQP Lurches Towards Completion

I really mean the word "lurches" that I used in the title of this post. I've been following the AMQP specification since the 0.8 version, and there have been more dramatic twists and turns in this than in an old Perry Mason novel. (In the latest twist, they seem to have dropped the very concept of an Exchange, a mainstay of previous versions).

I don't believe the latest version of the spec is available (I would dearly love to have a read), but the working group met recently in San Diego, and the materials presented are here.

My overall reactions are these:

1. We really do need an industry-standard messaging protocol to match the capabilities of proprietary products like IBM's MQ and TIBCO's EMS. These two vendors have built their hugely profitable businesses at the expense of customers who have nowhere else to go. The commoditisation that has lowered prices in other areas of distributed computing (TCP/IP for networking, HTTP for web, SMTP for email, etc.) has simply not occurred in enterprise messaging. AMQP aims to rectify that. All I can say is, "It's about time".

2. I note with amusement that Microsoft has joined the AMQP bandwagon. Nothing like a lack of market share to elicit good behaviour from Microsoft ;-)

3. As I said, I don't have access to the latest spec, but I know that an intermediate version had dropped support for streaming communications and file transfer. If these are still not part of the spec, I believe that's a huge mistake. An enterprise messaging protocol must natively support these significant aspects of distributed computing, otherwise the protocol as a whole will fail to pass muster in spite of its excellence in other areas.

4. I don't know what the AMQP Working Group is thinking, but I believe that the main competition for AMQP is not IBM and TIBCO but HTTP and REST. Whenever I talk to REST afficionados about AMQP, the response is a big ho-hum. "What can AMQP do that we can't already do with HTTP?" is the response. Asynchronous notification? Just use XMPP for that. Security? SSL is good enough. And so on. The AMQP Working Group should co-opt a particular breed of REST expert - one who understands the value of resource-orientation without being wedded to HTTP. I believe that if we can define an application protocol on top of the transport protocol that AMQP is (by usefully constraining it with special verbs, headers and status codes), we will have a more capable architectural style than REST, one which includes native support for event notification, end-to-end security, reliable message delivery, transactions, file transfer, streaming communications and process coordination. Rohit Khare's ARRESTED style can become a reality.

Is anyone listening?

The Computing Platform of the Future

What will the computing platform of the future be like? (The consumer-side platform, that is)

It'll be a super-slim, compact and lightweight laptop with some innovative folding smarts that will let people carry it around in their pockets, yet open it up to a regular laptop form factor for ease of use. It will feature several important innovations, some of which are already available, or tending towards a tipping point:

1. Really cheap hardware - $50 for the device that you can pick up at the supermarket and toss into your trolley without a second thought (impulse purchase territory)
2. Absolutely free software either on the device (like OpenOffice) or in the "cloud" (like Gmail)
3. Ubiquitous wireless connectivity (some cities do have 'em)
4. High network bandwidth (Australia is investing in this right now)
5. Breakthrough battery technology that lets you work a regular day without power supply

Needless to say, this device will double as a mobile phone, portable music player, portable video player, SatNav, game console and much else. This "computing platform" will be the enabling innovation that will let many others ride on its back. And it won't be a real innovation so much as a perfect storm of many tipping points being reached at the same time.

The key technology to watch is Android. By standardising and commoditising the core software layer, Android will enable innovation both below it (i.e., hardware) and above it (i.e.,applications) in the stack.

Death of the Directory

The LDAP directory is a strange beast. It's a data store, but it's hierarchical in nature, rather like the old hierarchical databases most of us only read about in database textbooks. Hierarchical models, as we know, have inferior expressive power when compared to network and relational models. So shouldn't LDAP directories suffer from similar limitations?

It turns out that they do have similar limitations, and the industry as a whole has been going along with the convenient illusion that the emperor has clothes.

Talking to some old industry hands, I became gradually convinced of something that I've suspected for a while - that the LDAP directory was an invention that solved a temporary problem. The problem has gone away, but the awkward solution remains with us.

Twenty years ago, relational databases were relatively slow and cumbersome. Even reads were slow. Relational databases were therefore not a good fit for read-mostly use cases.

Enter the directory server. Directory servers were designed to be very fast at lookups. They were abysmally slow at updates, but that didn't matter. They were designed mainly for lookups, not for updates. Systems requiring on-line transaction processing needed relational databases.

Somewhere along the way, relational databases got faster. Much faster. Faster at reads as well. And somewhere there, LDAP directories lost their raison d'etre. Only the world never noticed.

Today, LDAP directories do some things well - a very small number of things. They perform authentication very well. Checking a user's credentials (an encrypted password, for example) is a directory server's bread-and-butter. Plus it can enforce password policies, lock accounts on a certain number of failed attempts, etc. These are all built-in features, whereas relational databases are too general-purpose to be able to do these out of the box.

But relational databases are excellent at modelling complex relationships. Directories suck at this. If we try and shoehorn complex structures into a hierarchical model, we will end up with highly unsatisfactory results. Not only will it be inelegant and difficult to understand, the resulting directory bloat will also slow down the system, negating the performance advantage that prompted the choice of the directory in the first place.

The bottomline is, anyone looking to implement a "directory" today would be better off with a hybrid model. They should use an LDAP directory for the limited tasks which it is good at (authentication), and should use a relational database to model and store all other information. They then gain the best of both worlds. The directory server will hold just a handful of attributes needed for authentication.

Hybrids are all very fine, but how do we bridge between a relational database and an LDAP directory? In other words, what do we use to link a record in a database with the corresponding entry in the directory? One candidate would suggest itself to anyone who has been following this blog lately - the Universally Unique ID (UUID).

Of course, most relational databases don't support a native UUID datatype. And many DBAs are used to having identity columns for tables, which are autogenerated by the DBMS. In such cases, the UUID could be another attribute of the same table with a unique constraint defined on it. In other words, the UUID is another candidate key, and this is the one that is used to link an entity within the database to corresponding attributes in other systems. It is truly universal and unique.

How do we provision databases and directories so that their records are consistent? There may be no theoretically satisfactory answer (such as a two-phase commit transaction across both), but there are a number of real-world solutions that are adequate. Organisations may find that the hybrid data store is a pragmatic approach after all.

UUIDs and 2D Barcodes

I wrote about UUIDs (Universally Unique IDs) earlier and speculated about some of their possible uses. One of them, I thought, would be the ability to link them to real-world entities through corresponding 2D barcodes, which are a cool innovation in their own right.

So this weekend, I spent some time trying to combine both technologies.

Apparently, there is a standard called PDF 417 that defines 2D barcodes. There are Open Source encoders and decoders that implement this standard.

I downloaded the Pdf417lib encoder software from SourceForge. It's just a simple Java class that you compile and run, so it's not particularly complex to use. Of course, it's not the best-documented piece of software around, but then again, I can't look an Open Source horse in the mouth. I just wish I knew what the various parameters were so I could tune them. As it was, I tried fiddling around with some of them and settled for something that sort of worked. The main() method of the class is written to generate a PostScript file corresponding to any text string that is fed to it, so all I had to do was pipe in a fresh UUID as that text string, using the java.util.UUID class and its randomUUID() method. I didn't have the appetite to do much more than prove the concept.

I wrote the following bit of code based on the main() method of the Pdf417lib class itself. This is what my test class looks like:



The critical lines here are:

Pdf417lib pd = new Pdf417lib();
pd.setText( UUID.randomUUID().toString() );

I compiled and ran the class quite simply:

$ javac TestUUIDBarcode.java
$ java TestUUIDBarcode test1.ps
$ ps2pdf test1.ps test1.pdf

The "ps2pdf" command, as can be guessed, converts PostScript files to PDF.

I ran this once more just to check that I wasn't getting the same barcode again. The UUID generation is meant to be random, so I should get a new barcode every time.

Here are two of them. I hope they look different enough.






So that's what UUIDs look like when converted into 2D barcodes. All courtesy a standard JDK function and an Open Source library. During the dot-com era, someone could have started a company based on this idea ;-).

Java's Centre of Gravity Has Shifted

I was amused to read this slant on the rumours of IBM's takeover of Sun.

No, it wasn't industry analysts who were being asked their views on the merger. It was Rod Johnson, original developer of the Spring framework and CEO of SpringSource.

SpringSource isn't an IBM or an Oracle in size. Outside of the most tech-savvy segment of the Java development community, SpringSource and Rod Johnson are virtual unknowns. Yet the press was beating a path to this man's door. That in itself was somewhat surprising. Even more surprising was that Rod's reaction to what should have been major industry news was rather ho-hum. His reasoning was that in recent times, Java middleware has been influenced more by Open Source and independent developers than by large corporations, so it really doesn't matter either way if the merger goes ahead or not.

I tend to agree, with one caveat. The roadmap for an Open Source Java is still murky, and I'd like this to be clarified before I'm willing to yawn at the irrelevant plans of corporate titans. There are those like Stephen Colebourne who have almost a conspiracy theory about Sun's plans for Java. While I hope they're wrong, I too would like to see an open Java 7 specification that can be implemented by anyone and be certified as such.

If I can't have an open Java specification, I'm willing to settle for at least one safely Open Source (i.e., GPL-ed) implementation of Java, but I'm disappointed even there. Java is still not 100% Open Source. It still suffers from "encumbered code". There are a few libraries that have not been released under the GPL, and that is a lingering source of worry.

But in the main, Rod Johnson is right. A few years ago, JBoss began to shift the centre of gravity of Java away from the Weblogics of the J2EE world towards a more lightweight, Open Source implementation of J2EE. Today, that wrenching movement has continued with the marginalisation of J2EE itself.

I believe that Spring and Tomcat are ringing the death knell for heavyweight Java. If Java becomes a 100% open platform with a completely open specification and at least one completely GPL-ed implementation, then it won't matter which industry heavyweight buys which. It will be a mating of dinosaurs in the age of mammals.

The GUIDness of UUID

Microsoft calls it the GUID. The rest of the world calls it the UUID. But regardless of its name, the Universally Unique ID or Globally Unique ID is truly unique.

On the face of it, what's so great about a 128-bit number apart from its length?

Well, plenty. The length, humble as it may seem, crosses a tipping point of sorts, and imparts to the UUID a number of extremely useful properties.

340,282,366,920,938,463,463,374,607,431,768,211,456

That's what 2 to the power 128 is.

What's the probability of two randomly generated 128 bit numbers being the same?

2 to the power -128, or 0.0000000000000000000000000000000000000029.

That's 38 zeroes before the first significant digit. In contrast, the much-vaunted "five nines" reliability of the very best computer systems corresponds to 0.00001.

Now think about it. In the past, if we ever wanted to ensure that two entities were given IDs that had to be unique, the only way to ensure that uniqueness was through a sequence number generator, and one that was maintained at a single point.

Suddenly, the UUID provides a new way. We can randomly generate IDs of this length from any number of independent sources., and they're virtually guaranteed never to conflict! Of course, our pseudorandom number generator had better be cryptographically secure, or all bets are off...

Many languages provide libraries to generate random UUIDs. Java has the utility class java.util.UUID.

One generates a UUID in Java through the static method call UUID.randomUUID(). A toString() will generate a string representation of the UUID, which has 36 characters (32 hex characters plus 4 hyphens at defined places).

E.g., cd6b31ee-a877-41fe-a8f1-87d35d2045a6

This is more compact than the numeric representation and possibly more portable between systems.

There are any number of possible uses for UUIDs, especially when combined with other ingenious innovations like 2D barcodes. Complete identities and properties can be output in barcode format and affixed to real-world entities, providing a ready link to corresponding data in databases.

I've known of the existence of UUIDs for a while, but I'm only now waking up to their potential.

Update 10/09/2009: Here's the regular expression for a UUID:
[0-9A-Fa-f]{8}(-[0-9A-Fa-f]{4}){3}-[0-9A-Fa-f]{12}