Opinion: When Web Services are not the right tool

Discussions

News: Opinion: When Web Services are not the right tool

  1. In a recent blog entry, called "Stop the hype about webservices!", I show some situations in which webservices are being used but are definetely not the best tool for the job. The problems related to their use in those scenarios is discussed, as well as better alternatives.

    The entry states that "Webservices are generally not the right technology for integrating two systems written in Java... [neither] for integrating two systems for which there are better forms of integration". Webservices are considered probably a good choice when "you are integrating with .NET" or other languages that don't support CORBA.

    Read Michael Santos in (Stop the hype about webservices!

    Do you think webservices are being misused and overhyped? Do you think there are other situations they shouldn't be used? Please share your thoughts.

    Threaded Messages (21)

  2. Just a joke[ Go to top ]

    A joke I read about two years ago from www.javaworld.com -

    Web Service is like high school sex. Everybody is talking about it. Very few is doing it. And for those few who are doing it, nobody is doing it right!
  3. read the tea leaves[ Go to top ]

    The Java community, by listening to retrograde promotions for RMI/IIOP, is about to be blindsided again by Microsoft's push into a service oriented architecture. We would all do well to study Indigo very carefully to not repeat the last mistake the Java community made by not supporting SOAP in the JDK from the outset. Take a look at Microsoft's FAQ for Indigo for starters. Read the FAQ very carefully when it talks about 'uncoupling' systems, since that is the key point.

    Also, the Java community should be very careful to support any contentions about performance with carefully designed and executed benchmarks. We recently responded to a benchmark for XSLT performance that documented poor performance in a Java environment. In fact, our tests, using the same methodology but a different Java deployment, demonstrated excellent performance.

    I would love to see some documented, reproducible benchmarks for web services vs. RMI/IIOP performance for a service oriented architecture (coarse-grained service interfaces). The benchmarks should indicate the percentage of overall processing that the interface takes. I would be amazed if the interface for a mature web service platform took more than a small fraction of the overall processing. If it did, the next step would be to profile the execution and find the bottlenecks. Marshalling and unmarshalling arguments can't possibly be the most significant overhead of any reasonably designed distributed application.

    SZ
    Objects by Design
  4. Webservices are considered probably a good choice when "you are integrating with .NET" or other languages that don't support CORBA. <


    I'm probably being anal here, but implicitly dismissing WebServices for everything BUT Java and .NET integration turned that statement quite a bit weird. So, .NET won't talk CORBA, COM+, DCOM, COM or even plain old DLL integration (through JNI or what-have-you)? Are you *sure* there's no other options here, Michael?

    That being said, I would agree if you stated that "webservices are not for integration" or even "webservices suck when used as RPC". Webservices are, surprisingly enough, made to provide *services*. Through the *web*. Thus, their name. Anything else is just asking for trouble.

    > Do you think webservices are being misused and overhyped? Do you think there are other situations they shouldn't be used? Please share your thoughts. <

    Misused, overhyped, mis-lots-of-things, and over-lots-of-other-things. So was J2EE, when it began - and still is, IMHO. But J2EE being overhyped and misused is not news anymore, I guess...

    [/flames]
  5. I'm probably being anal here, but implicitly dismissing WebServices for

    > everything BUT Java and .NET integration turned that statement quite a bit
    > weird.

    I never meant it. That's what you got from my statement, though. :-D Webservices certainly have their place.

    > So, .NET won't talk CORBA, COM+, DCOM, COM or even plain old DLL integration
    > (through JNI or what-have-you)? Are you *sure* there's no other options here,
    > Michael?

    It's terribly hard to use any other integration strategy with .NET in most cases. But notice I said "probably" in my original sentence. In some cases, JNI and/or a (D)COM(+) bridge are nice solutions. CORBA never worked perfectly with Microsoft stuff, in my experience. As I usually say, there isn't a "yes/no" answer for nearly anything :-D

    > That being said, I would agree if you stated that "webservices are not for
    > integration"

    I wouldn't. They might be used for integration sometimes (you know it :-P)

    > "webservices suck when used as RPC".

    That's something I agree with.

    > Webservices are, surprisingly enough, made to provide *services*. Through the
    > *web*. Thus, their name.

    Totally agree.

    > Anything else is just asking for trouble.

    It depends. I'd say _nearly_ anything else.

    > Misused, overhyped, mis-lots-of-things, and over-lots-of-other-things. So was
    > J2EE, when it began - and still is, IMHO. But J2EE being overhyped and
    > misused is not news anymore, I guess...

    I have to agree, again. Just hope people get the idea faster.
  6. you mean banal not anal. do you?
  7. you mean banal not anal. do you?

    No, he means anal as in anal retentive. A freudian term - essentially means compulsively neat.
  8. One of the *GREAT* seminal leap forwards of RMI, CORBA (and previous RPC) was that it freed the developer of worrying about low-level marshalling, i.e. When I have to implement my own custom serialization/deserialization routines (e.g. Apache Axis) for a "complex" object, I cannot help but think that Web Services is not that great a leap forward.

    Let's keep the distinction clear: ease of use for the application developer (WS client) who would definitely prefer language-specific object materialization vs. ease of use for the WS server developer who prefers to plop an XML blob on the wire and let the downstream client worry about materialization.
  9. Misses the point[ Go to top ]

    I like the idea of this blog (that people are mis-using web services), but I think it misses the point. By making these assertions that whether or not to use Web Services should have to do with what language you are writing in seems a bit wrong.

    Whether or not to use Web Services is more of an architectural decision: how flexible does your system need to be, what types of clients does it need to support, how much control do you have over what those client are, and will you want to change them in the future.

    Let's say you build a system using RMI that has 3 different clients going against one server. Then you have a buisiness need to add a fourth client, but this next one needs to be a .Net client, or some other client. You're now in a bit of a bind. You can either add a new pathway on the server that uses Web Services at the risk of creating duplicate code that might not be easily refactored, or you need to change all your clients to use a new protocol that the .Net client can also use.
  10. Re: Misses the point[ Go to top ]

    By making these assertions that whether or not to use Web Services should

    > have to do with what language you are writing in seems a bit wrong.

    It's not only about language, but language definetely plays an important role when it's time to choose the technology for the job.

    > Whether or not to use Web Services is more of an architectural decision: how
    > flexible does your system need to be, what types of clients does it need to
    > support, how much control do you have over what those client are, and will
    > you want to change them in the future.

    I agree. But does your system _has_ to support non-Java clients _now_? That's about the XP idea of "doing the simplest thing that could possibly work". If you design thinking about SOA - note, that's an important part of the trick so that things work -, you'll end up with interfaces that can be converted to WSDL and it's very likely that they can even be directly mapped to it. Generally, the most important thing is SOA, really.

    > Let's say you build a system using RMI that has 3 different clients going
    > against one server. Then you have a buisiness need to add a fourth client,
    > but this next one needs to be a .Net client, or some other client. You're now
    > in a bit of a bind. You can either add a new pathway on the server that uses
    > Web Services at the risk of creating duplicate code that might not be easily
    > refactored, or you need to change all your clients to use a new protocol that
    > the .Net client can also use.

    If your RMI interfaces represent services, just use them for generating the webservice, all the time. If you don't rely on the fact remote objects can mantain state - like when you get the return of a call and use it, knowing it will change a object on the server side -, it should be transparent. As I said in the comment area for my blog, if you designed with it in mind, a next -> next -> finish wizard may be enough.
  11. Re: Misses the point[ Go to top ]

    [[[
    But does your system _has_ to support non-Java clients _now_? That's about the XP idea of "doing the simplest thing that could possibly work".
    ]]]

    Sounds more like warping DTSTCPW to fit your argument!


    [[[
    If you design thinking about SOA - note, that's an important part of the trick so that things work -, you'll end up with interfaces that can be converted to WSDL and it's very likely that they can even be directly mapped to it. Generally, the most important thing is SOA, really.
    ]]]

    How is this TSTTCPW?

    Anyway. Webservices != SOA.

    Your blog entry imo, was an interesting language/technology driven argument and had nothing much to do with SOA style. If you design thinking about SOA, you don't concern yourself with point to point integrations, or what the published object interfaces happen to be, or by what can be blown out to WSDL, or... Those are all artefacts of your implementation.

    SOA is about forcing /out/ assumptions based on implementation technology not spotwelding systems together with them. You drive SOA design from a business contract expressed as a series of documents and very minimal assumptions about what the recipient is running (modulo internet protocols). Object interfaces, WSDL, IDL et al come much later and after the service contract is agreed. These days the baseline technology assumptions are HTTP/SSL and XML. That may change, but not soon.

    I hear what you're saying about webservices abuse, but disagree with the reasons you cite.
  12. Re: Misses the point[ Go to top ]

    Sounds more like warping DTSTCPW to fit your argument!


    Not really. I just meant it, really. People are often too concerned about:" what if in the future we need to support other kinds of clients?" when the project scope explicitly says it must support Java clients. So, each thing at its own time.

    > How is this TSTTCPW?

    You don't want to get systems to be tightly coupled, but you also don't want them to integrate using lots of things. Java interfaces are the simplest answer to your problem. If they represent services - and not a bunch of getters and setters -, you know you won't have too much job in the future. So, you get all this with a very simple solution. SOA is simple.

    > Anyway. Webservices != SOA.

    Totally agree. But, in general, it's a mistake to use webservices without SOA; the opposite is not true.

    > If you design thinking about SOA, you don't concern yourself with point to
    > point integrations, or what the published object interfaces happen to be, or
    > by what can be blown out to WSDL, or... Those are all artefacts of your
    > implementation.

    I agree, again. But Í wouldn't blindly choose webservices unless there were obvious reasons for that.

    > SOA is about forcing /out/ assumptions based on implementation technology not
    > spotwelding systems together with them. You drive SOA design from a business
    > contract expressed as a series of documents and very minimal assumptions
    > about what the recipient is running (modulo internet protocols). Object
    > interfaces, WSDL, IDL et al come much later and after the service contract is
    > agreed.

    AFAIK, SOA is about exposing services. It has nothing to do with technology - it doesn't require one and also does not require you to be language agnostic. You can implement SOA using pure Java interfaces, RMI, CORBA, webservices and other technologies. They are by no means directly attached to each other.

    > I hear what you're saying about webservices abuse, but disagree with the
    > reasons you cite.

    Choosing webservices when they are not needed - that's what I meant. My blog is about SOA because SOA does not require you to use webservices - and also doesn't stop you from using them when it's a wise decision.
  13. Web Service performance[ Go to top ]

    Actually, since this topic is brought up, I'd like to find out about the problems people are running into with Web Service performance. It's not surprising me that people are using Web Services incorrectly to get bad performance. Probably the most common example of this would be trying to use Web Services as a straight-up replacement for RMI, or by using fine grained calls via Web Services.

    I have a harder time believing that well designed Web Services run into large problems with XML. Is the problem actually parsing a SOAP Packet? Is the dealing with the XML payload in your app? (Have you tried XMLBeans?) I find the numbers Michael brought up with regards to a general app that uses XML to be too general to be useful. I mean if an app that runs XSL over and XML to create and HTML document spends 95% of the time doing processing XML, I wouldn't be terribly surprised (since it's entire purpose is to process XML).

    Take the Amazon Web Services. I have a hard time believing doing a product search through their Web services takes a lot more time than doing a product search through their Web pages.
  14. Re: Web Service performance[ Go to top ]

    Probably the most common example of this would be trying to use Web Services

    > as a straight-up replacement for RMI, or by using fine grained calls via Web
    > Services.

    That is, indeed, as serious problem. Have you seen a webservice that exposes getters and setters? I have :-D

    > I find the numbers Michael brought up with regards to a general app that uses
    > XML to be too general to be useful. I mean if an app that runs XSL over and
    > XML to create and HTML document spends 95% of the time doing processing XML,
    > I wouldn't be terribly surprised (since it's entire purpose is to process
    > XML).

    Oh, that was not the situation. In many cases, you have a full application doing some fairly complex business logic to perform an operation spending several seconds to generate a view of the output, when the business process takes ms! That's pretty bad...

    > Take the Amazon Web Services.

    That's a good application of webservices.

    > I have a hard time believing doing a product search through their Web
    > services takes a lot more time than doing a product search through their Web
    > pages.

    Can't say for sure, but as far as I remember, they have some machines _just_ for processing webservice calls. Most projects don't run in such a fantastic infrastructure and their business scenario is not even close to Amazon's. That's way these projects shouldn't have webservices: there is no service to be exposed! Amazon and Google are nice examples of when you should use webservices, although there is some controversy about Google using SOAP instead of REST.
  15. Web Service performance[ Go to top ]

    Actually, since this topic is brought up, I'd like to find out about the problems people are running into with Web Service performance.

    Most developers learnt on a LAN. Fine grained communication is a typical mistake on a WAN.
  16. I've just completed a service that provides data to the enterprise. I'm on release 2.0 (before going to production, I might add) because of data binding issues. I'm using Axis 1.1 (which by the way, is an awesome tool!). What I found was that when my data structure became more complicated (nested beans, arrays of beans), my binding tool (a free runtime tool with limited functionality) broke. It was difficult to determine the problem so I made the decision to use a more robust free tool (Castor) and, while I was at it, improve performance by switching to Doc/literal from RPC encoded. This was a painful task (I went through a commercial binding tool (borlandxml) and found it didn't meet my needs. When I switched to Castor, and with the help of IBM devloperworks (http://www-106.ibm.com/developerworks/library/ws-castor) I was able to get it working. My performance has been improved and my data structures are handled.

    Web Services aren't the answer for every problem. However, you should consider the critical mass behind the technology. With large communities using web services and building applications and tools around them, you should be able to build a solid application that meets your customer's needs.

    Hat's off to the Axis and Castor development teams!

    Dan
  17. Web Services done the Woing Way[ Go to top ]

    I've just seen a third party company implement web services the wrong way.


    Classic mistake number one:

    They implemented a web service call for each database operation they though the clients would need. The result: provisioning a new client in the system requires several web service calls. Of course there is no support for rollback, so if, say I had to do 5 web service calls in order to provision a new client in their system, if, say the first two web service requests succeeded but the third one failed, I was forced to try to send more web service requests in an attempt to "undo" the first two calls which had been successful. Yuk.

    Morale of the story: A web service should be just that, a service. I should be able to send my client provisioning request in a single web service request, which would trigger a series of operations on the other side, and the response would either be success or failure, and at least the operation would be atomic.


    Classic mistake number two:

    The third party created web service requests that would be about 16kb in size on average. Double that if you include the response.

    And the third party server was located on the other side of the continent, linked only with a 1 mbps link or something like that. Talk about a bottleneck.

    Allow me to use an analogy. Those of us who have connected to an Oracle server located somewhere else on the internet through JDBC know how much badwidth this consumes. Would you consider doing this in production? I think not.

    The problem is that web services's value reside in easier interoperability between two systems that were typically developed by two distinct teams. And if the web service is used to connect two systems that are not physically in the same 100mbps ethernet network, then the size of XML messages becomes a real problem. Machines may get faster as years go by, but internet bandwidth does not increase at the same pace.

    Call me an old fart if you want, but in the past I have developed communication protocols where the message sizes were twenty times smaller while conveying the same amount of information. The messages were simple ascii streams sent though sockets. Parsing the messages was simpler and faster, plus I could squeeze twenty times more messages with the same available bandwidth. The time spent developing a communications protocol was time well spent and the overall results were blazing fast performance. Ah, the good old days :)

    Web services is a good fit if you have a small, well defined interface between two systems and lots of available bandwidth.
  18. Web Services done the Woing Way[ Go to top ]

    Classic mistake number two: The third party created web service requests that would be about 16kb in size on average.

    The assumption being that bulk adds latency. Maybe not on a WAN. Two 8kb calls in sequence likely much slower than one 16 kb call. Ie, fat calls not a liability.

    Web services is a good fit if you have a small, well defined interface between two systems and lots of available bandwidth.

    I won't gamble against monotonicly cheapening bandwidth. Nor encourage others to.
  19. Michael,

    I've noticed that you were thinking about an "enterprise lightweight container" in the blog comments, providing transactions, remoting, etc in a more flexible way than EJB does. This is *exactly* what Spring is about: Besides the core container, we provide declarative transactions via AOP on POJOs (with pluggable transaction strategies), and remoting via a variety of integrated tools - all out-of-the-box. You might browse through the current Martin Fowler thread here on TSS, where some of those services are discussed.

    Essentially, when you want to do stateless remoting with an existing Spring-managed middle tier, you'll either export an existing POJO service object via the remoting protocols of your choice (even multiple ones at the same time), or write simple facade beans for your service that then actually get exported. You can also host a subset of your Spring-managed middle tier within a remote EJB implementation, to export your POJO services via an EJB facade.

    Our new JPetStore version (included in the 1.0 M4 distribution) provides a remote order service that is accessible via Hessian, Burlap, JAX-RPC (with Axis as default implementation), and an RMI invoker. As far as possible, the wire protocol is considered a configuration detail rather than something that the remote service code or remote client code needs to care about. Switching between those protocols is primarily a matter of configuration.

    In case you're wondering, Hessian and Burlap are two lightweight HTTP-based remoting protocols from Caucho, the makers of the Resin web app server. Hessian is binary, while Burlap is a slim XML-based protocol. Both are not Java-only but Java-oriented enough to make a compelling solution for Java-to-Java remoting. They are available as open source under the Apache license, working in any servlet container. See http://www.caucho.com/hessian respectively http://www.caucho.com/burlap for details.

    Juergen
  20. I've noticed that you were thinking about an "enterprise lightweight

    > container" in the blog comments, providing transactions, remoting, etc in a
    > more flexible way than EJB does.

    Not exactly, but I found your comment interesting, though :-D

    > In case you're wondering, Hessian and Burlap are two lightweight HTTP-based
    > remoting protocols from Caucho, the makers of the Resin web app server.
    > Hessian is binary, while Burlap is a slim XML-based protocol. Both are not
    > Java-only but Java-oriented enough to make a compelling solution for Java-to-
    > Java remoting. They are available as open source under the Apache license,
    > working in any servlet container.

    I'll try them out the next time I have to do a distributed architecture.
  21. Curious[ Go to top ]

    I would have thought that the decision to use web services is a business case, not one driven by technological concerns. Once the decision is made then one should expect platform and language choices to follow. Surely this would avoid misuse of web services: given the problem, what is the best solution?

    I personally would love to see an article, or book for that matter, guiding developers and architects from a domain -> problem -> requirments -> solution perspective, that used web services: why the choices were made, what factors were considered when choosing the technology, and how it performed in relation to the initial requirements. And what failed. I dont particularly care what language/s are involved, but such case studies would clarify many concerns, especially questions regarding whether a web service solution is overkill or not.
  22. Re: Curious[ Go to top ]

    I would have thought that the decision to use web services is a business

    > case, not one driven by technological concerns.

    Not really. I can't think of a good reason to choose webservices to integrate two apps written in Java that'll reside in the same network. That's not business, but I can say that it's extremely unlikely I'd use webservices for this scenario no matter what the business requirements would be (integrating with other platforms is technical for me).

    > Once the decision is made then one should expect platform and language
    > choices to follow. Surely this would avoid misuse of web services: given the
    > problem, what is the best solution?

    That's nice in theory. In practice, it doesn't work quite well. It's a good principle to follow sometimes; in other occasions, it's impossible or not good.

    > I personally would love to see an article, or book for that matter, guiding
    > developers and architects from a domain -> problem -> requirments -> solution
    > perspective, that used web services

    I have a blog entry I'm writing about when you should use webservices. I'll update this thread when I finish (in a couple of weeks, I guess).