Sunday, October 07, 2007

Life above the Service Tier

Or "How to Build Application Front-ends in a Service-Oriented World"

(This is a paper written by myself with two co-authors, Rajat Taneja and Vikrant Todankar, the same team that critiqued the EJB 3.0 spec in 2004.

You can download and read the paper and an accompanying presentation.

Update Sep 2008: How SOFEA fits into a SOA Ecosystem)

But let me talk a bit about it here.

When we look at the Business Logic Tier, although the industry hasn't reached nirvana yet, the story seems to be coming together rather well. The Service-Oriented Architecture (SOA) message is being heard, with its concepts of loose coupling between systems, removal of implicit dependencies and specification of all legitimate dependencies as explicit contracts. The SOA model yields flexibility and uniformity through the specification of a definite Service Interface, which hides domain models and implementation technologies.

Unfortunately, no such clear architectural blueprint exists for the Presentation Tier that lies just above the Service Interface. There are a dozen different technologies available here. There are also the thin-client and rich-client models to choose from. Which of these is the "right" one to use? What does industry best practice say about developing the front-end of an application? What is the best way to connect the front-end neatly into the Service Interface? Developers are confused and left without guidance.

And while on the subject of connecting the front-end to the Service tier, an additional source of confusion is the fact that the Service Tier may itself follow one of two different models - SOAP-based Web Services or REST. Which one should we target? We believe that both these models will continue to co-exist, and organisations will have legacy business services in both these models. Therefore a good Presentation Tier technology will be capable of interfacing with both SOAP and REST, not just one of them. It is important to note that both SOAP and REST use (or are capable of using) XML documents for data interchange.

When we put on our architects' hats and look at the Presentation Tier, we see that regardless of whether it is a thin client or a rich client, three essential processes always take place.

The first is what we call Application Download. In order to be able to run an application on a client device, it needs to be delivered to the device in some fashion.

When the application is in use, a sequence of screens is typically seen. We call this Presentation Flow.

Finally, any non-trivial application ultimately exchanges data with some back-end resource, a process we call Data Interchange. This is usually the crux of the application's functionality.

To reiterate, the three processes occurring in the Presentation Tier of any application are Application Download (AD), Presentation Flow (PF) and Data Interchange (DI).

In thin-client applications, the web server plays a part in all three processes. Application Download occurs piecemeal, in the form of individual web pages served up by the web server. The web server also drives the application's Presentation Flow, usually with the help of a web framework like Struts or Spring MVC. Finally, the web server acts as an intermediary for Data Interchange between browser and application server.

We believe that the thin-client model as described above suffers from at least three major architectural flaws. Unless these flaws are addressed, we cannot arrive at a clean architectural blueprint for the Presentation Tier.

The first flaw is that the thin-client model does not respect data. Look at the data that goes from the browser to the web server. There is no data structure. It's just a set of name-value pairs. There are no data types. Everything is a string. And there are no data constraints. We can put any values into those name-value pairs.

Look at the data that comes back from the web server to the browser. It's highly presentation-oriented, marked-up data. The thin-client model does not respect data as data. We believe we can do much better than this in this day and age.

The second flaw is that the thin-client model tightly couples the mutually orthogonal processes of Presentation Flow and Data Interchange. It is not possible to move through the steps of the Presentation Flow without initiating what amounts to Data Interchange operations. Web pages are displayed in response to GET and POST requests sent by the browser. Even worse, every Data Interchange operation initiated by the browser willy-nilly forces a Presentation Flow. An infamous result of this tight coupling is the "browser back-button problem". When a user tries to step backwards page-by-page through the Presentation Flow, the browser re-does the Data Interchange operations that resulted in each of those pages. If any of those operations is a POST, it spells trouble, because POST operations are not safe or "idempotent". They have side-effects if re-done.

True, there are patterns such as POST-Redirect-GET that are used to work around this problem, but the inescapable fact is that the fundamental architecture is broken because of this tight coupling.

The third flaw is that the web model is request/response. It does not support peer-to-peer interaction styles that are required for server event notification.

At this juncture, one would be tempted to conclude that AJAX is the answer to these problems. Unfortunately, AJAX itself is just a raw capability and not a prescriptive model. It is possible to use AJAX and still come up with a horrible hybrid model where the web server continues to drive Presentation Flow in response to Data Interchange operations, and the AJAX interaction just hangs off to one side, so to speak.

Clearly, what we need is a prescriptive architectural model that explicitly delineates the dos and don'ts of application design.

We call this model SOFEA (Service-Oriented Front-End Architecture).

In the SOFEA model, there is an entity called a Download Server. This role may be performed by a standard web server, but its role is very strictly restricted to enabling Application Download. It stays out of the loop thereafter and does not play a role in Presentation Flow or Data Interchange. This is the most significant difference between the SOFEA model and the traditional thin-client model.

The application is downloaded onto an Application Container, which could be a browser, a Java Virtual Machine, a Flash runtime or even the native operating system itself. The nature of the Application Container does not matter. What matters is that there is an environment within which the application can run.

The application is built using a proper MVC architecture, not the Front Controller pattern used by traditional thin-client applications. The Controller drives Presentation Flow.

The Controller also asynchronously initiates Data Interchange with the Service Interface as required.

We stipulate that a SOFEA-conforming application will support both SOAP and REST-based Data Interchange through XML documents. This is the seamless interface that we envisage between the Presentation Tier and the Service Tier. These XML documents, being specified through a schema definition language, will enforce data structures, data types and data constraints at the point of capture and ensure data integrity end-to-end. The XML documents of the service interface correspond to a representation of the domain, or (to a Java programmer) Data Transfer Objects (DTOs). They're ideal for transferring data marshalled from domain objects or unmarshalled into domain objects. Thus, while XML documents are a connection point between the Presentation and Service Tiers, they also decouple the client from the domain model, which is as it should be.

For maximum flexibility, Data Interchange should support Peer-to-Peer interaction. A strict client/server delineation as in the case of traditional thin-clients is likely to be limiting.

The SOFEA model unifies the "thin" and "rich" client models. Indeed, these labels are meaningless when systems conform to the SOFEA model. The differences between technologies form a spectrum based on the application's footprint and its startup time. There is no sharp demarcation between thin and rich clients anymore.

The SOFEA model also highlights why there is a plethora of web frameworks. SOFEA explicitly repudiates Front Controller as an anti-pattern. Driving Presentation Flow from the web server is tied to a fundamental architectural flaw (the coupling to Data Interchange). Therefore no web framework can ever be satisfactory. Continued innovation in search of a "better" web framework is, we believe, completely misguided.

The SOFEA model can be implemented through any of the following modern technologies:

I DHTML/AJAX frameworks for Current Browsers
1. Handcoded with third party JavaScript libraries
2. Google Web Toolkit (GWT, GWT-Ext)
3. TIBCO General Interface Builder
II XML Dialects for Advanced Browsers
4. XForms and XHTML 2.0
5. Mozilla XUL
6. Microsoft SilverLight/XAML
III Java frameworks
7. Java WebStart (with/without Spring Rich Client)
8. JavaFX
IV Adobe Flash-based frameworks
9. Adobe Flex
10. OpenLaszlo
Obviously, different technologies will support, out of the box, different subsets of the SOFEA model. It's up to designers to consciously apply SOFEA principles to their application design, making the appropriate decisions with regard to their technology of choice.

Our contribution through the SOFEA model is the following:
  1. A renewed emphasis on data integrity, which traditional thin-client technology does not and cannot enforce.
  2. A cleaner architectural model that decouples the orthogonal concerns of Presentation Flow and Data Interchange.
  3. Affirmation of MVC rather than Front Controller as the natural pattern to control Presentation Flow.
  4. Unification of the thin-client and rich-client models, now seen as an artificial distinction.
  5. Support for SOAP- and REST-based business services, and a natural integration point between the Presentation and Service Tiers.
  6. Positioning the web server as a Download Server alone. The evils of web server involvement in Presentation Flow and Data Interchange are avoided.
We believe we have provided an architectural blueprint for the Presentation Tier, with clear dos and don'ts, while still supporting considerable diversity in technologies. Life above the Service Tier now lives by similar rules as life within it.

Tell us what you think.

24 comments:

Leirith said...

Hi Ganesh,

I read through your paper on Friday. Very impressive! It seems like a robust architecture. I hope it catches on!

Luke

Unknown said...

Hello Ganesh,

I'm not yet seeing the full implications of this architectural style (though I haven't yet read the full paper).

Do you propose a software framework that implements as a reference implementation of this architecture? That would certainly make the points much more concrete for me.

I'll write more when I've had time to read the longer paper, but alas I need to spend time writing myself!

Jim

prasadgc said...

Jim,

There are at least 10 different technologies that can be used to build a reference architecture for SOFEA. Like you, we have also been strapped for time! But yes, this exercise is definitely on the agenda "real soon now".

Ganesh

lOlive said...

i have built such a MVC client-side presentation framework.
my goal was to have a clean and manageable front tier for all my SOA apps.
i can guarantee you that it is *much much* cleaner and versatile than JSP or XSLT.

John said...

Ganesh,

I am interested in a slightly different problem. I want to orchestrate autonomous services with Human powered services... and I recognize that there are no clear guidelines for developing HPS... please take a look at my blog Human Powered Services.

Your architecture for Service Oriented Front Ends seems like a good starting point... What do you think?

Krishna said...

A Very Creative Thought.

Andrei said...

Ganesh,

Very interesting paper with many solid ideas. It is definitely a good starting point. There are few things though that need further clarification:

- What are the SOFEA design goals? The paper states the key principles without defining what it is trying to accomplish.

- What is the scope of SOFEA? Is security (e.g. SSO) part of it? What about application integration (mashups), human workflows or content management?

- Is it intented to be platform-independent framework or Java is the the main target platform?

ss said...

Looking for some implementations.
May u post links to some implementation resources available.
SS

Qwert said...

There are 2 fallacies that I could find directly.
1. Browsers are popular because of their flexibility. The moment you want something downloaded from the server, you talking about an applet/flash object/something that will have system dependencies.
2. I still wont say front controller is an anti-pattern. I believe that the current implementations are incorrect/incomplete.

Antti said...

Thank you for your clear presentation, it was a pleasure to read.

After browsing your paper stating the flaws, it was clear to me that there already is a solution for each and every flaw you were able to find. The answer is Seam:
Flaw 1 fix: Seam integrates Hibernate validator, so you define data integrity in one place. Seam's model treats data as data, have a look at Seam-gen for an example.
Flaw 2 fix: Seam has conversations and you use a separate technology just to define page flow, thus separating the concern.
Flaw 3 fix: Ajax is easily used with Seam, while Seam handles plumbing.

And Seam is a pleasure to use, after it soon will be used with help of Maven2.

prasadgc said...

Antti,

I have had a look at JBoss Seam. Its starting premise makes me laugh but also leaves me with a very bad feeling.

"JSF and EJB 3.0 are two of the best new features of Java EE 5. EJB3 is a brand new component model for server side business and persistence logic. Meanwhile, JSF is a great component model for the presentation tier."

I think EJB and JSF are the worst examples of design-by-committee in the Java world. They're monstrosities. Just compare Spring with EJB and Spring Web Flow/Spring MVC with JSF to understand how horribly bloated EJB and JSF are.

Sorry for not being enthusiastic about a technology that promises to graft these two monsters together.

Ganesh

David said...

Your paper has changed the way I think about web-apps. The historical perspective is fascinating. Thanks for this great work!

Looking forward to your thoughts on how well GWT can support SOFEA.

Unknown said...

These resaults seem very interesting. I work with Ruby on Rails, but I have worked with Java/J2EE for a couple of years as well. I always felt that the old way of doing web applications was not natural and I was hoping for a change. This could be the right movement!

Cesidio

G said...

Hi Ganesh and gang,

I just have to drop in and say that this is a very well-thought paper. I like its refreshing concept.

I believe the days of SOFEA frameworks will be here soon. Then, it'll be another headache to choose that "perfect" SOFEA framework. ;-)

wiisniper said...

Hi Ganesh,

Very good article.
You may add SAP NetWeaver Business Client (project Muse) as SOFEA technology option. You can find more about it at https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/4766

Unknown said...

Hello Ganesh,

I wonder if you think that XForms is a technology that adopts some of the principles behind your presentation, because it wasn't mentioned.

Thank you very much for the wonderful article.

Kiril Kirov

prasadgc said...

Kiril,

XForms is definitely a technology that conforms to SOFEA principles. See page 30 of our article (under "XML Dialects for Advanced Browsers").

Ganesh

Unknown said...

Yes man, for sure i didn't see XForms at the first place.

Well we are now implementing a large system using IBM tools such as WebSphere Integration Developer and WebSphere Portal Server with XForms as part of the front end technology as a way to separate business data (xml) and the presentation logic. But still the main concept is that the flow of the views should be controlled by the business processes within the backend server - a WebSphere Process Server witch runs a bunch of BPEL defined processes and interacts with Service Components (usually pure java). So my question is: In your opinion should the business logic of the process flow be delivered to the front end?

Thank you,
Kiril

Gabriel Kastenbaum said...

Hi everybody!

Nice piece of work indeed. Lots of news ideas and new perspectives.

Something i do not understand: why is client side controller always better than server side controller?
- For instance : submitting a form. First you validate it, then you send the datas. In our good old way ;-) the datas are persisted/used on the server, then the controller decides what to do, and send s a new page/thing to display/anything back. If the validation gives an error (you want to keep some Validation on the server side, right?) it send an error page. What if the flow engine is in on client side? There will be one remote call to the server to persis datas, then the controller decides what to do on client side then if needed another call to the server for display another page with another datas? Isn't is to much remote calls? 2 instead of one. Isn't it exchanging a flaw against another flaw? (there is a "But": in a way this works the same way as post+redirect).
- what about all the exception management? validation, remote exceptions...
- what about security? If the client side decides what to see, maybe this is could be an issue?


Thank you again for sharing your interistings thoughts,
Gabriel K.

prasadgc said...

Gabriel,

Fortunately, I think I have the answers to all the questions you raised :-).

> why is client side controller always better than server side controller?
> - For instance : submitting a form. [...] (you want to keep some validation on the server side, right?)

Yes, since the validation is part of business logic, it belongs on the server side and will be invoked by the client as a service. The only logic that belongs on the client side is navigation logic. Note that though the client _triggers_ validation, it needs to be supported by the server (through services) especially when more complex rules are to be applied.

> Isn't it too much remote calls? 2 instead of one.

The goal is not to reduce the number of remote calls (AJAX has shown us that remote calls can be very fast and efficient). The goal is to do things correctly. Business logic on the server, presentation logic on the client - that's the model we believe is right.

> - what about all the exception management? validation, remote exceptions...

All that is defined by the service interface, which is non-visual. The client tier invokes non-visual services and receives either meaningful data in response or an error response. It's up to the client to handle the display and further navigation. It's a reasonable separation of duties.

> - what about security? If the client side decides what to see, maybe this is could be an issue?

The client side can only see what the server decides to display. If required, the client can choose to see _less_ than what is allowed, but never more. The server will always protect its functions and data from unauthorised access. That's part of the service interface.

Regards,
Ganesh

SunC said...

Just Came across your paper, very well presented. But, I am not sure this architecture applies to all web applications. For example, a banking web application has different functions geared towards users with different roles, in the architecture you are proposing , when the user downloads the application , he will have to download the whole enchilada.

RC

SunC said...

Please ignore my previous comment. The paper explains this issue.

Thanks

Jasper van den Bosch said...

...
Generic verb (CRUD) HTTP verb
Create POST
Retrieve (also Search/Find) GET
Update PUT
Delete DELETE
...

Not sure everyone will agree with this table.. A POST could be a partial "Update" (containing the property to be updated for the resource at target location), and a PUT can most definitely be a "Create".
In general, much respect for your initiative.

prasadgc said...

Jasper,

You're right. We didn't update the paper, but this is probably a better mapping:

GET - Retrieve, List, Search
POST - Append
PUT - Create, Update or CreateOrUpdate
DELETE - Delete

Ganesh