Here's my fundamental beef with an ESB: It follows a hub-and-spokes model that is more suited to an EAI initiative of the 1990s. In the year 2010, we should be doing SOA, not EAI.
SOA is federated where EAI is brokered. It's the architectural antithesis of EAI. The smarts are pushed out from the middle to the endpoints until there's no "middle" anymore. I wrote about the difference between "federated" and bad old "point-to-point" before. Federated is a standardised smartening of the endpoints.
It doesn't matter whether we do SOAP or REST. Either way, it's SOA, and it's inherently federated.
Both SOAP's Web Service Endpoints and REST's Resources are represented by URIs. URIs offer a "flat" address space. They're completely opaque. DNS can hide where the services physically sit, so any node can talk to any other without the need for an intermediary.
Look at a SOAP message. Look at the headers - WS-Addressing, WS-Security, WS-ReliableExchange. They're all end-to-end protocols. The SOAP engines at the endpoints negotiate and handshake every Quality of Service they need without depending on anything in between.
WS-ReliableExchange uses endpoint-based logic such as acknowledgements, timeouts and retries to provide a measure of reliability even with unreliable transports, much like TCP provides connection-oriented communications over a plain packet routing protocol like IP. There is no need for a "guaranteed delivery" transport like a message queue.
WS-Trust does key exchange, WS-SecureConversation sets up a session key and WS-Security encrypts messages, all in an end-to-end fashion. There is no need for a secure transport.
Any node that talks SOAP and WS-* and conforms to the WS-I Basic Profile (and Basic Security Profile and Basic Reliability Profile) can participate. That's federation.
This isn't an academic discussion. The main drawbacks of a hub-and-spokes model are Availability and Scalability. The ESB is a single point of failure and a performance bottleneck. The normal "solution" is to beef up the ESB by providing for redundancy and "High Availability", but this costs a fair bit and only postpones the inevitable. That's why I said this isn't an academic discussion. There are serious dollars involved. Organisations would do well to think about the implications of an ESB approach and take early steps to avoid the inevitable costs that lie along that path. The correct solution would be to do away with the centralised ESB altogether and embrace the inherently federated model of SOAP (or REST).
Otherwise, we're forcing inherently federated protocols into a hub-and-spokes model, - a needless throwback to an earlier era , - and inviting problems that need to be "solved" at great expense. (Of course, it's a different matter if we use multiple instances of the ESB as brokers close to endpoints, but economics militates against such a topology, so I suspect federated ESBs will not be feasible.)
In the year 2010, when clouds are all the rage, we should recognise that what SOAP and REST give us are "service clouds".
So that's my buzzword for today - ESC (Enterprise Service Cloud)!
The ESB is dead, long live the ESC!
14 comments:
Ganesh:
I am not sure when you lost your wisdom, but your post makes no sense at all. An ESB enables oneself to achieve one of the most fundamental principle of SOA in a generic way: decoupling the interface from the implementation. It enables me to organize interfaces in a consumer oriented way be it for security, simplicity, semantics or even versioning (http://www.ebpml.org/blog/40.htm).
REST on the other does not promote this level of decoupling. Quite the opposite, it couples identity and access in a terrible way and provides no consumer oriented boundaries.
You can always use (or even see) an ESB as a HUB and Spoke architecture, just like people use a hammer to nail a screw or Visio to create UML diagrams. But it simply shows your ignorance of the tool, not that there is anything wrong about the tool.
WCF had completely forgotten this decoupling and some people in Microsoft Consulting Services had to build their ESB (Managed Service Engine - MSE) because the product would never listen to them.
JJ-
JJ,
I am not even talking about REST here - LOL. Forget REST for the sake of this argument. Take SOAP Messaging in the spirit in which it has been designed. It's a protocol to build capability through end-to-end handshakes, whether it be WS-Security, WS-ReliableExchange, WS-AtomicTransaction or anything else. There is no need for any component "in the middle". We can have message security even over a non-secure transport. We can have a degree of reliability even over an unreliable transport. Where is the need for a broker? (An ESB is a broker with additional smarts, standardised to a certain extent.)
Regards,
Ganesh
Ganesh:
forget REST and SOAP. Do you see any benefit to separate Interface from Implementation? Do you see any benefit to inject specific behavior between the interface of a component and the component implementation itself?
This is why all Component technologies failed before, and sure enough even with SOAP some people (via code generation) still hardwire the endpoint to a particular object method.
There lies the problem, it is not at the protocol level, it is below the protocol between the endpoint and the service implementation.
JJ-
JJ,
There's absolutely no argument there on decoupling interface from implementation, and that's why I'm surprised at your disagreement with my "federated versus brokered" argument of the current post. This is an orthogonal argument.
I the past I have argued against generating interfaces from implementation classes, the way many development tools encourage one to do. You may remember past posts where I recommend "meet in the middle" mapping tools instead of code generation. See the last two paras of this post. Also, the concept of the
"Viewpoint Flip" between implementation (Domain Object methods) and interface (Service verbs) is meant to introduce this decoupling.
I think we're arguing about nothing here. Let me emphatically state that I am *against* coupling interface to implementation, so this is not a point of contention. I talk about this so very often in my blog.
Do you need a separate broker component to enforce this decoupling, or can an endpoint do it itself? I believe an endpoint can and should do it itself. A broker decouples in an EAI style. We need to follow a SOA style, not an EAI style. This implies federation, not brokered mediation.
And I think you've still not understood that REST does this decoupling extremely well. Your domain object methods are never going to be GET, POST, PUT or DELETE. They will be domain-specific methods, which *you* as the developer will map in a meaningful way to the HTTP verbs. Think of it as polymorphic behaviour. You as the developer will introduce the specific behaviour you need, either within the implementation or through a cross-cutting concern, as this will be completely opaque to the service interface. (Without going off on a tangent, let's not forget the potential of AOP to help us decouple functionality even further.)
So no, I don't get what the argument is about, sorry. I think you've just misunderstood the entire point of this post. Please go back and read it again.
Regards,
Ganesh
glad we agree, but an ESB is not a broker it is a service container. Only some people see it as a broker. More generally, you see now the concept of Service Delivery Platform emerging from ESBs with even more decoupling between Interface and Implementation.
So I am not quite sure you are saying that because a hammer is not good for screws it is a bad tool altogether. Of course, many EAI broker vendors duck taped a screw drivers on their hammer and claim it is just as good as a screw driver.
Overall, I'd be careful making the kinds of claim you are making, I know perfectly good ESBs and saying every ESB is bad is not necessary helpful because often it is the only "service container" you can buy.
I do not believe at all in endpoints should be responsible for their own decoupling. REST has an infinite number of endpoints... I guess you see the problem. The decoupling happens between the endpoint and the implementation, well below the protocol. Protocols by definition do not bring any level of decoupling. It is what you do with the request that bring decoupling.
JJ-
PS: ESC = Enterprise Service Container.
JJ,
How many ESBs does an organisation need? If you say "one", then it's a centralised broker, no matter how you define ESB. If it's more than one, in what topology do you distribute them (logical topology, I mean)? I suspect you would have one ESB each to service a particular application domain, or a particular set of use cases. In that case, the ESBs move closer to the endpoints and become more like adapters. Take that to its logical conclusion and use something like Spring Integration to inject specific pieces of ESB functionality as required into your endpoints themselves, and you have a federated ESB. That's the model I propose, and that's the "cloud" model, because what's in-between is not emptiness but routing components that are transparent to the endpoints. Some of them could even have the routing smarts that ESBs have, but don't have to be full-fledged ESBs.
In other words, I have two problems with ESBs:
1. All the functionality doesn't have to be bundled in a single "container", as you call it. It can be decomposed and injected into any application node that requires it. This is the "smart endpoints" model that I prefer.
2. ESB doesn't have to be a centralised box. There can be many smaller and specialised boxes strewn around the network to form a routing network like IP. This is the "cloud" model that I prefer.
Take the two together, and most commercial ESBs are unnecessary.
You may say that ESB doesn't have to be deployed in a hub-and-spokes fashion, but this is only true in theory. In practice, given the licensing model of an ESB vendor, customer organisations prefer to buy just one and place it at the "centre". Just look around you to verify the truth of this statement.
We need a competing architecture to help organisations break out of the hub-and-spokes model. The "cloud" is the right one for the times, and the technology is also ready. As I said, it doesn't matter whether it's SOAP or REST, we do our logical decoupling between the interface and the implementation, and we also federate the way the protocol works. No single component is required to provide connectivity or qualities of service!
Regards,
Ganesh
Before I forget, JJ:
> REST has an infinite number of endpoints... I guess you see the problem.
No, I _don't_ see the problem. Your characterisation of this as a problem is the problem!
The web has an infinite number of endpoints. By your reasoning, it should have come down by now. We don't have a central server in Geneva, right?
I don't know if there's some subtle point around "federation" that we're not able to agree on.
Regards,
Ganesh
Ganesh,
will all due respect, I think you are making an error of interpreting the topology of an ESB. The ESB is a broker between an end-point and the service implementation (good) it is not a broker between the consumer of the end-point and the service implementation (bad). BizTalk server is such a broker. Axis is not.
If you talk about "one" ESB means everything is Hub and Spoke, it is like saying because you use one JEE app server, all applications are coupled. No a JEE app server is an application container. An ESB is logically a service container, no matter how you deploy it. An ESB is logically centrally administered, like a farm of JEE app servers. As a service container you can take down one service without taking down every service (if you do, this is not a Service Container, it is a broker).
So again, I find it quite strange that we agree between the decoupling between endpoint / implementation and we agree that hub and spoke is bad, but we strongly disagree on the definition of an ESB.
I think all these concept should be taught in a technology/product independent way. Again, I can use an ESB as a broker. It does not mean that this what you should do or that an ESB is bad. What people should focus on is what they try to achieve and how (not to mention why...). They should not believe that REST, SOAP, HTML5 or anything else is a magic bullet that will make all their problem go away. It would be like expecting that a hammer can build a house by itself after hitting it N number of times. You can only build a house with a hammer by thinking how you use the hammer, for what result.
JJ:
> The ESB is a broker between an end-point and the service implementation (good) it is not a broker between the consumer of the end-point and the service implementation (bad).
I'm afraid I see both of these as bad.
Service implementations do not need an ESB to expose themselves as service endpoints (which is the "good" use case you refer to). A few lines of Spring Integration config will do that ;-).
Once you have a federated cloud of service endpoints (which have the smarts to negotiate security, reliability, etc., with other smart endpoints), the need for the ESB goes away, except for smart routing, for which we can deploy specialised routing nodes (not full-fledged ESBs) in a transparent manner within the cloud.
Saying that all services must be exposed through an ESB is needlessly limiting, especially in the cloud age.
Regards,
Ganesh
Ganesh:
what is spring in that case? isn't it the broker between the endpoint and the service implementation? We are talking about the same thing, you just reject the name "ESB". Call it foo, just look at what it does architecturally, not what vendor tell you about it.
The problem with REST is precisely two-fold in the decoupling of the endpoint from the implementation:
a) multiplication of endpoints without any boundaries
b) coupling access with identity
/customers/123 is different from /customers/v1/123
REST simply does not work, no matter how you think it works. REST is building an integration spaghetti at the scale of the Web precisely because none of the RESTafarians understand the decoupling between interface and implementation.
All right, we'll call it "foo". I like my "foo" federated, thank you. That's what I want it to do architecturally. I don't want it to do hub-and-spokes mediation. I don't care what you say about ESB not being hub-and-spokes, that's the way it is deployed in 99%, perhaps 100%, of user organisations. This is a bad model.
About REST, I'm waiting for you to wake up one morning and understand it. It's not "integration" at all, which is the way corporate architects tend to approach it, it's inherent interoperability through standardised interfaces. You have had this argument with many people over many years and you still don't see what so many on the SOAP side of the fence have seen and been converted to (Steve Vinoski, Jim Webber, etc.)
Surely so many people can't be dreaming. I think you should make another attempt to understand REST, without pre-judgement this time.
Regards,
Ganesh
>> it's inherent interoperability through standardised interfaces.
Even Stu Charlton, a Restafarian at large, but whom I respect because precisely he tries to understand what he is doing just published : (http://www.stucharlton.com/blog/archives/2010/05/wsrest.html). "One of the takeways: contracts matter".
Yeap, the "uniform interface does not exist".
Why don't you look at the RESTful APIs around you?
https://gsma.securespsite.com/access/Access%20API%20Wiki/Payment%20RESTful%20API.aspx
Are they RESTful? Just point to me one single application that was built 100% with RESTful principles. Just one.
I have no respect for the people that you reference because they are marketers. They care about selling their story or their book. Even Jim Webber is now promoting the concept of a contract via RESTfulie.
I think if there is someone who needs to wake up and stop dreaming, it is you. REST is simply a fraud and I feel sorry that someone of your caliber is saying what you are saying.
Once more, I respect Roy's REST, it gave us the Web. The other REST, the one you are talking about does not exist. It is an illusion for people who don't understand a thing about SOA.
JJ,
If you think that a uniform interface is the opposite of a contract, then you have misunderstood a basic aspect of REST. We can have contracts _and_ a uniform interface. Uniform does not mean exactly the same, it means following a common pattern in a predictable manner. The details would obviously differ (as specified by their contracts), otherwise all applications would be the same!
> Just point to me one single application that was built 100% with RESTful principles. Just one.
We don't have to _build_ entire systems RESTfully. We only have to expose their functionality RESTfully. You have previously said you want to see an ERP built using RESTful principles. It doesn't have to be, as long as the modules don't need to expose their functionality to the outside world. When they do, REST is an excellent way to do so. Internally, a module could use proprietary APIs. Between modules, you could expose services using SOAP or REST.
High cohesion, low coupling as always.
Let's just agree to disagree and close this discussion. It's not going anywhere, I'm afraid.
Regards,
Ganesh
I think we'll have to disagree unfortunately. Even Jim is no longer talking about cohesion. You seem to have missed a few shows.
JJ
Post a Comment