A colleague asked me this question today, and I'm really glad it came out in the open, because it's so much easier to deal with ideas when they're plainly stated.
I believe there are at least two strong reasons to actively avoid vendor lock-in.
The first reason, paradoxically, lies in the very justification for agreeing to be locked in - "ease of integration". If an organisation already has a product from a certain vendor and is looking for another product that needs to integrate with this one, then conventional thinking is to go back to the same vendor and buy their offering rather than a competitor's. After all, they're far more likely to be "well integrated". There are typically so many problems integrating products from different vendors that it doesn't seem worth the effort. The best-of-breed approach leads to integration problems, so customer organisations often throw in the towel and go for an "integrated stack" of products from one company.
This approach is antithetical to SOA thinking, however. What they're really saying here is that they don't mind implicit contracts between systems as long as they work. But implicit contracts are a form of tight coupling, and as we know, tight coupling is brittle. Upgrade one product and you'll very likely need to upgrade the other as well. In production systems, we have seen product upgrades delayed for years because the implicit dependencies between "well-integrated" components could cause some of them to break on an upgrade, which is unacceptable in mission-critical systems. As a result, many useful features of later versions are forfeited. That's one of the unseen, downstream costs of the tight coupling that comes from vendor lock-in.
SOA shows us the solution. For robust integration between systems, we need loose coupling between them, which seems a bit paradoxical. Shouldn't tight coupling be more robust? Hard experience has taught us otherwise. But what is loose coupling? It's a hard concept to visualise, so let's define it in terms of tight coupling, which is easier to understand. In practical terms, loose coupling between two systems can be thought of as tight coupling to a mutually agreed, rigid contract rather than to each other. Such contracts are very often nothing more than open standards.
Even though people generally nod their heads when asked about their preference for open standards, the contradiction between that stated preference and their practical choices in favour of "integrated stacks" is hard to understand. If pressed, they might say that creating contracts external to both systems and insisting on adherence to them seems like a waste of time. Why not something that just works "out of the box"? The answer is that this is not an either-or choice. A browser and a web server work together "out of the box", but they do so not because they come from the same company but because they both adhere to the same rigid contract, which is the set of IETF and W3C standards (HTTP, HTML, CSS and JavaScript).
The key is to hold vendors' feet to the fire on adherence to open standards. This isn't that hard. With Open Source products available in the market to keep vendors honest, customers have only themselves to blame if they let themselves get locked in.
The second reason why vendor lock-in is a bad idea is because it often implies vendor lock-out. Very often, customers are held hostage to the politics of vendor competition. Once you start buying into a particular vendor's stack, it will become increasingly hard to look elsewhere. Any third-party component you procure will be less likely to play well with the ones you already have, causing you more integration headaches. My favourite example from just a few years ago is Oracle Portal Server, which claimed to support authentication against LDAP. It turned out later that it wasn't just any LDAP server but just Oracle's own OID. This meant that the corporate directory server which happened to be IBM Tivoli LDAP couldn't be used. The data in it had to be painfully replicated to OID to allow Oracle Portal Server to work.
My solution to incipient vendor lock-in would be to aggressively seek standard interfaces even between products from the same company. I remember asking an IBM rep why they weren't enabling SMTP, IMAP and vCalendar as the mail protocols between Notes client and server. The rep sneered at me as if I was mad. Why would you use these protocols, he wanted to know, when Notes client and server are so "tightly integrated"? Others on my side of the fence agreed with him. Well, the answer came back to bite the company many years later, when they wanted to migrate to Outlook on the client side. Their "tight integration" had resulted in vendor lock-out, preventing them from connecting to Notes server using Outlook (which standard protocols would have allowed) and they were stuck with Notes client indefinitely. By that time, there were too many other dependencies that had been allowed to work their way in, so enabling open protocols at that stage was no longer an option. That was a great outcome for IBM, of course, but to this day, there are people in the customer organisation who don't see that what happened to them was a direct consequence of their neglect of open interfaces in favour of closed, "tight" integration.
Ultimately, vendor lock-in has implications of cost, time and effort, which basically boils down to cost, cost and cost.
As users of technology, we simply cannot afford it.
I believe there are at least two strong reasons to actively avoid vendor lock-in.
The first reason, paradoxically, lies in the very justification for agreeing to be locked in - "ease of integration". If an organisation already has a product from a certain vendor and is looking for another product that needs to integrate with this one, then conventional thinking is to go back to the same vendor and buy their offering rather than a competitor's. After all, they're far more likely to be "well integrated". There are typically so many problems integrating products from different vendors that it doesn't seem worth the effort. The best-of-breed approach leads to integration problems, so customer organisations often throw in the towel and go for an "integrated stack" of products from one company.
This approach is antithetical to SOA thinking, however. What they're really saying here is that they don't mind implicit contracts between systems as long as they work. But implicit contracts are a form of tight coupling, and as we know, tight coupling is brittle. Upgrade one product and you'll very likely need to upgrade the other as well. In production systems, we have seen product upgrades delayed for years because the implicit dependencies between "well-integrated" components could cause some of them to break on an upgrade, which is unacceptable in mission-critical systems. As a result, many useful features of later versions are forfeited. That's one of the unseen, downstream costs of the tight coupling that comes from vendor lock-in.
SOA shows us the solution. For robust integration between systems, we need loose coupling between them, which seems a bit paradoxical. Shouldn't tight coupling be more robust? Hard experience has taught us otherwise. But what is loose coupling? It's a hard concept to visualise, so let's define it in terms of tight coupling, which is easier to understand. In practical terms, loose coupling between two systems can be thought of as tight coupling to a mutually agreed, rigid contract rather than to each other. Such contracts are very often nothing more than open standards.
Even though people generally nod their heads when asked about their preference for open standards, the contradiction between that stated preference and their practical choices in favour of "integrated stacks" is hard to understand. If pressed, they might say that creating contracts external to both systems and insisting on adherence to them seems like a waste of time. Why not something that just works "out of the box"? The answer is that this is not an either-or choice. A browser and a web server work together "out of the box", but they do so not because they come from the same company but because they both adhere to the same rigid contract, which is the set of IETF and W3C standards (HTTP, HTML, CSS and JavaScript).
The key is to hold vendors' feet to the fire on adherence to open standards. This isn't that hard. With Open Source products available in the market to keep vendors honest, customers have only themselves to blame if they let themselves get locked in.
The second reason why vendor lock-in is a bad idea is because it often implies vendor lock-out. Very often, customers are held hostage to the politics of vendor competition. Once you start buying into a particular vendor's stack, it will become increasingly hard to look elsewhere. Any third-party component you procure will be less likely to play well with the ones you already have, causing you more integration headaches. My favourite example from just a few years ago is Oracle Portal Server, which claimed to support authentication against LDAP. It turned out later that it wasn't just any LDAP server but just Oracle's own OID. This meant that the corporate directory server which happened to be IBM Tivoli LDAP couldn't be used. The data in it had to be painfully replicated to OID to allow Oracle Portal Server to work.
My solution to incipient vendor lock-in would be to aggressively seek standard interfaces even between products from the same company. I remember asking an IBM rep why they weren't enabling SMTP, IMAP and vCalendar as the mail protocols between Notes client and server. The rep sneered at me as if I was mad. Why would you use these protocols, he wanted to know, when Notes client and server are so "tightly integrated"? Others on my side of the fence agreed with him. Well, the answer came back to bite the company many years later, when they wanted to migrate to Outlook on the client side. Their "tight integration" had resulted in vendor lock-out, preventing them from connecting to Notes server using Outlook (which standard protocols would have allowed) and they were stuck with Notes client indefinitely. By that time, there were too many other dependencies that had been allowed to work their way in, so enabling open protocols at that stage was no longer an option. That was a great outcome for IBM, of course, but to this day, there are people in the customer organisation who don't see that what happened to them was a direct consequence of their neglect of open interfaces in favour of closed, "tight" integration.
Ultimately, vendor lock-in has implications of cost, time and effort, which basically boils down to cost, cost and cost.
As users of technology, we simply cannot afford it.
5 comments:
You make a couple of very good points! But there is a risk to taking your stance: to be perceived as the guy with an academically sound philosophy that doesn't cut it in the real world.
I fear that A LOT of people think "Going with Microsoft" or "Going with Oracle" etc. is a sound strategy, and they can point to a lot of examples of why that is so.
My point is, we need more examples. Not just generic vendor lock-in stories but specifically how going with a suite of products actually turned out worse than going best of breed.
Also, examples of people that have gone the best-of-breed route, that admit that route had its challenges but that they conquered them.
Without the power of good examples, the risk is people agree with you in principle but won't dare to deviate from their former practice of choosing what's safe.
You know, the "no one got fired for choosing IBM/Microsoft/.." strategy has something that people need: a feeling of comfort.
My other question is, how do you address the downside of going best of breed? For instance what happens when you've chosen best of breed products to play together, and you encounter problems and the different vendors start blaming each other? The comfort of being able to pressure a single vendor isn't there.
There is more to this story and I hope to see you continue this.
Good point you got here!
Here is the point I get:
Going "best of breed" is very difficult because consumers allow themselves to be manipulated into choosing stacked products. In other words, there are very few examples of "best of breed" working better, because most people allow themselves to be forced into bad decisions.
The point I am getting from the article is that this vendor lock-in problem is our own fault and it is causing problems. The only way to fix it is to either quit using proprietary software entirely, or to start demanding open standards from proprietary software (as in, refuse to use proprietary software that does not adhere to the open standards you are choosing to use). The point is that we have let the software "industry" become corrupt, and fixing it is going to cost us. Of course, we do not have to fix it, but if we do not then the total cost will be much more than if we do.
So, while you are right that a lot of people think "Going with MS", or such, sounds good, they are wrong. We do not need more examples, we need action, otherwise we will end up with a bunch of different factions using specific proprietary standards, who will not even be able to communicate effectively, and eventually we will incur costs that are too large to handle. And, ironically, it will be our own darn fault, because we decided to go with what seemed easiest at the time, without measuring the long term costs. (Imagine two groups that speak different languages, and have no means of translation or of learning each others' languages. The consequences would probably be war.)
My point is that we have seen the problems caused by vendor lock-in. If we think we need examples to show that "best of breed" is better, then we are stupid. When one thing is a problem, we should try other things until we either find something that works, or run out of alternatives. Since the "best of breed" approach is an alternative, and it has not been tried enough to establish whether it is better or not, we would be stupid not to try it, since the current method obviously does not work well. If we all just sit on our butts waiting for someone else to try things, we will all end up failing together. We need people who will get off their butts and make examples, so we can see if the one really is better than the other. So quit whining that there are no examples, and either try it yourself, or convince someone else with the resources to try it. This is what the author is trying to do: convince those with the resources to try it.
Lord Rybec
So as a CIO, I should be more concerned with whether a product is going to be "loosely coupled" than whether it will fulfill the needs of my business?
Or I should do as you say an "demand" that everything I look at provide some kind of "open standard" interface instead of providing a complete integration with my other systems, provide a competitive advantage to my business, and allow me to continue running a business? Or I should just say ph*q it and not bother with buying software at all?
Or I should go open-source and hire a bunch of programmers to make it work properly?
Yeah.. sounds like a really good idea. Don't call me, I'll call you.. and all that.
Technology Guy,
Whoa! You make it sound like my recommendation was for a theoretical ideal that has no practical benefit:
"Or I should do as you say an "demand" that everything I look at provide some kind of "open standard" interface instead of providing a complete integration with my other systems, provide a competitive advantage to my business, and allow me to continue running a business? Or I should just say ph*q it and not bother with buying software at all?"
A lot of strawman arguments there.
Keeping interfaces open and systems loosely coupled is not an academic recommendation. That is in fact the proven and most cost-effective way to "provide a complete integration with my other systems, provide a competitive advantage to my business, and allow me to continue running a business".
If you disagree with that, I can deduce that your current approach is vendor-specific and high-cost, like most of your peers. You've standardised mediocrity (Industry Best Practice!) and are now trying to elevate it to the virtue of "pragmatism". I've heard these arguments a hundred times before, which is why I wrote this post in the first place.
Check out the principles of SOA once again (and no, I don't mean SOA vendor products but SOA principles).
Regards,
Ganesh Prasad
Post a Comment