Tuesday, January 29, 2008

My First Mashup

So WSO2's Mashup Server has finally been released. As I've blogged before, this is a very exciting product for a number of reasons.

My first mashup is now public at mooshup.com, the hosted online version of WSO2 Mashup Server, where anyone can get an account and start kicking the tyres of this revolutionary technology.

The mashup is really simple, but it illustrates some of the power of the platform. I have two independent "services", a Grocery List service and a Grocery Prices service (Click the "Try it" link on each). The idea is that I create my grocery list independently knowing only what products I need and the quantities of each that I need. What I don't know are the (unit) prices of these products. Another service provides the prices for a number of grocery products, which is a superset of the products in my list.

My grocery list:

<line-item prod="tomato" qty="4" />
<line-item prod="potato" qty="3" />
<line-item prod="onion" qty="6" />
<line-item prod="chilli" qty="40" />

Grocery prices (independently available):

<prod id="potato" unit-price="1.00" />
<prod id="tomato" unit-price="2.00" />
<prod id="onion" unit-price="3.00" />
<prod id="beans" unit-price="4.00" />
<prod id="chilli" unit-price="0.10" />

What I need to do is "mash up" my grocery list with the price list that is independently published, so that I can see my itemised grocery bill, with individual costs and total cost.

To anyone who shudders at the thought of the XML manipulation this entails, the brevity of the following code should be vastly reassuring:

var total = 0.0;
for each ( i in gList.descendants("line-item") )
i.@amount = i.@qty * gPrices..prod.(@id==i.@prod)["@unit-price"];
total += Number( i.@amount );
gList..order["total-amt"] = total;

That's really all it takes. The "for" loop iterates through all elements in the grocery list with the tag "line-item". The first line inside the loop is the most interesting one. It's basically multiplying the quantity by the unit price to arrive at the amount, but this is where the "mashup" happens. It's using the unit price of the product in the grocery price list whose id matches the id of the product in the grocery list. It's like this SQL statement:


Then of course, there's the running total which gets inserted as a new subelement ("total-amt") under the "order" element of the grocery list. Note that if an element or attribute doesn't exist, it gets quietly added (e.g., the new attribute "amount" and the new element "tot-amt"). What distinguishes attributes from elements is the "@" symbol. If you avoid hyphens altogether in element and attribute names, the syntax is much simpler, but I prefer to use hyphens for complex tags, so I wear the extra syntactic overhead, but it's not much really.

What I finally get is the "mashed-up" XML:

<line-item prod="tomato" qty="4" amount="8" />
<line-item prod="potato" qty="3" amount="3" />
<line-item prod="onion" qty="6" amount="18" />
<line-item prod="chilli" qty="40" amount="4" />

which I can redisplay as a table.

My JavaScript is a bit rusty these days, but E4X has excited me enough to start coding again. Those of you who have to work with XML on a daily basis and curse the experience should look at E4X. It might just cause you to desert Java or C# for JavaScript.

If you do a View Source on the HTML page, you'll see that I've even used E4X to manipulate HTML from within JavaScript! Who wants to deal with the DOM? Treat whatever HTML structures you're working with as XML (well, it's all supposed to be XHTML anyway, so that's not an invalid assumption, right?). Once you've created the HTML structure you want, just convert it to a string using "toXMLString()" and use (the admittedly nonstandard but universally supported) "innerHTML" to insert that string into the existing HTML document.

Think of the possibilities. This goes far beyond content aggregation, which is what mashups are about. Mashup Server can talk both SOAP and REST (as consumer and producer). You can use JavaScript to code arbitrary logic, even simple processes. I'm sure someone will write a JavaScript library very soon to match the capabilities of WS-BPEL, and a translator to convert BPEL scripts into JavaScript, which will let them run more complex processes on Mashup Server. A process server! Next, combine all this with AJAX. You now have a one-stop shop for all your enterprise and collaborative computing needs. I wonder if there are nice programmatic interfaces to databases, directories, message queues, e-mail, etc. Something like the Spring templates in JavaScript would be nice.

I never had much regard for JavaScript when I was a Java programmer, but I'm sure developing a lot of respect for it now. A friend recently pointed me to Sun's research project, the Lively Kernel, which is essentially recreating Java applets using the more lightweight and ubiquitous JavaScript. Sun's Phobos library makes server-side JavaScript more feasible. Could it be that JavaScript is enjoying a resurgence?

E4X is the clincher for me. I know I'm not going to manipulate XML any other way right now. And WSO2 Mashup Server is my new hammer which makes every integration problem look like a mashup-able nail.


sumedha said...

Hi Ganesh,

"I wonder if there are nice programmatic interfaces to databases..."

WSO2 Data Services allow you to expose relational databases,CSV files & Excel spread sheets as web services.
A call to service will return results in xml format(which is also customizable).

Listed bellow are few resources for your reference.

1. Data Sources as web services - http://www.xml.com/pub/a/2007/10/25/data-sources-as-web-services.html
2. Data Services in SOA: Issues and Possible Solutions - http://www.infoq.com/news/2007/11/data-services-in-soa
3. How to use MySQL 5 INOUT variables in Data Services? - http://wso2.org/blog/sumedha/3082
4. Bringing relational data into web - http://wso2.org/library/26175
5. How to call MySQL 5 Stored Function from Data Service? - http://wso2.org/library/2631
6. How to call MySQL 5 stored procedure from Data Services ? - http://wso2.org/blog/sumedha/2623
7. Getting started with Data Services using Oracle - http://wso2.org/blog/sumedha/3051
8. Getting started with Data Services using WSO2 WSAS-2.0 - http://wso2.org/blog/sumedha/2573 - using Derby as database
9. Getting started with Data Service Samples(Movie) - http://wso2.org/blog/sumedha/2590 - using Derby as database

10. Specification - http://wso2.org/wiki/display/wsf/Data+Services+and+Resources

We working on adding support for more data source types too.


Keith Chapman said...

Hi Ganesh,

As the WSO2 Mashup Server is based on the latest release of WSAS (2.2 to be exact) you inherently get all the data services stuff in the Mashup Server. you can access WSAS in the mashup server at http://localhost:7443/wsasadmin (you need to provide your mashup server admin username and password). You can try out sumedha's samples there.


Ganesh Prasad said...

Thanks, guys. That's useful info.


Tammo said...

"I'm sure someone will write a JavaScript library very soon to match the capabilities of WS-BPEL, and a translator to convert BPEL scripts into JavaScript, which will let them run more complex processes on Mashup Server.

I don't think so. BPEL is a) a parallel programming language, it might be difficult to express that in a nice way in javascript. b) BPEL has been created to support long-running transactions. That means it has first-level support for asynchronous messageing (via WS-Addressing or token-based correlation) and supports complex faulthandling and compensation techniques. It would be certainly possible to reimplement this stuff in a javascript library but I doubt that it will be efficient enough. BPEL engines should be able to have hundreds of process modells with thousands of instances running in parallel. This is only possible because processes can be mapped to process calculus which can be easily hibernated in a data storage until the next event arrives. I'm not sure whether this is possible with scripting languages (I'm not aware of persistent continuations yet). IMHO mashups are great to quickly put WSs together, to do some data integration but I don't think they can really compete against robust and reliable BPEL/workflow machines.

Ganesh Prasad said...


JavaScript execution can be multithreaded. I understand Rhino provides support for it, and further, that Rhino is how WSO2 Mashup Server implements JavaScript (the WSO2 guys can correct me on this). So I don't see any inherent impediment to parallel processing.

Second, persistence is also posible, as the first two comments here point out, so the process calculus, as you say, can probably be stored as well.

I wasn't thinking so much of a literal translation between BPEL and JavaScript. I was thinking of a set of pre-built JavaScript library classes that would have all the capabilities required to support BPEL. The actual translation of BPEL into code would probably just create configuration code, such as (very roughly):

a = new ProcessingNode();
a.someConfigParam = "xyz";
b = new DecisionNode();
a.nextNode = b;

and so on, with their actual operations being based on asynchronous behaviour, supporting parallelism.

How efficient will that be? Well, if the feature gets used a lot, it'll get optimised in time, I guess.


Tammo said...

Yes, it's definitely an interesting development. I think there is enough room for both but they won't replace each other. I believe that BPEL targets a more business oriented niche (where business process diagrams are mapped to BPEL graphs), while mashups are more for hackers (who are used to common programming languages and whose IT skills are not limited to drawing nice diagrams with powerpoint).

BPEL is one approach towards bridging/changing the gap between business people and IT staff. The language provides all concepts needed but currently it is the lack of tooling that makes it difficult to create BPEL processes. Therefore it currently appears easier to do the orchestration on the programmers level instead of doing that on the business level (where it actually belongs in a process-oriented SOA).

Regarding multi-threading: I don't really want to have such threads running one or more years and I also don't want to loose data/status information if the server crashes for some reason. So there is more sophisticated work required than just using threads. But of course it may be possible to implement a pi-calculus engine in javascript which provides all these features...

hamoun said...

Nice post. I just got familiar about WSO2 in your blog. However, its desirable for me to see a comparison of its mashups and what lets say google(GGE), or microsoft offers. about the comment "BPEL engines should be able to have hundreds of process modells with thousands of instances running in parallel", I guess that igoogle kindof already does that. plus they are having some libraries for persistence of some data, which of course is not comparable to BPEL Dehydration but is growing up fast.