Discussions

News: Don't Be Afraid to Drop the SOAP

  1. Don't Be Afraid to Drop the SOAP (72 messages)

    Sam Tregar has written up his experience with using SOAP in a CMS system. He discusses how he was initially excited about the API, but on the next project ended up with a simpler XML/HTTP system. Is this a common experience? Is XML over HTTP enough?

    Conclusion
    SOAP isn't a bad technology, but it does have limits. My experience developing a SOAP interface for Bricolage taught me some important lessons that I've tried to apply to Krang. So far the experiment is a success, but Krang is young and problems may take time to appear.

    Does this mean you shouldn't use SOAP for your next project? Not necessarily. It does mean that you should take a close look at your requirements and consider whether an alternative implementation would help you avoid some of the pitfalls I've described.

    The best candidates for SOAP applications are lightweight network applications without significant performance requirements. If your application doesn't absolutely require network interaction, or if it will deal with large amounts of data then you should avoid SOAP. Maybe you can use TAR instead!
    Don't Be Afraid to Drop the SOAP

    Threaded Messages (72)

  2. XML over HTTP NOT enough[ Go to top ]

    Is XML over HTTP enough?

    No. And neither is SOAP... unless your system's needs don't include reliablility.

    Neither HTTP and SOAP are reliable protocols that provide exactly once delivery of a message. However, these schemes are fine for simple non-critical functions (such as grabbing a stock price for the sake of displaying the value for pure 'interest'). And in fact I'd agree that the simpler HTTP is not a bad idea in the cases where reliablility doesn't matter, and the "calls" and "responses" are simple to make and interpret.

    But if it's a financial transaction (such as the transmission of an order, invoice, etc.), then you need even more than SOAP to keep your system running clean. You need something like ebXML in order to give your application the ability to send a message and expect that it got there reliably (no lost messages, and no duplication).
  3. Large Attachments[ Go to top ]

    If sending and receiving large attachments is a requirement then SOAP provides a binary transport layer. Using XML/HTTP one must base64 encode attachments which adds a 30% increase in size. Moreover, the ability to use SOAP frameworks like Apache Axis with WSDL2Java streamlines system integration for trading partners. SOAP does have its limitations that ebXML attempts to solve such as reliable messaging. However, ebXML is not widely adopted and has very limited open source support.
  4. XML over HTTP NOT enough[ Go to top ]

    Hmmm, dont forget that SOAP doesnt mandate a transport...
    There is no reason why you cannot do SOAP over JMS.

    And reliability that you describe is only really essential in asynchronous interactions.

    -Nick
  5. Wow! Thanks! That is exactly what I've been doing for the last several years, but was afraid to propose as a general solution.

    Sure, what do we want from SOAP? A delivery of an XML request, and an XML response. Fine; both can be done over HTTP; and if you do not use SOAP-specific headers, you are free to test it with a browser. More, if you do not use SOAP, you are also free to use all the power of XML, not the castrated SOAP version, no attributes, no nothing.

    Fine! Now I proud at last. I absolutely second this opinion. Any intelligent fool can make things complicate and large, said Einstein. Have to get rid of that SOAP scum that has been clogging our pipes.
  6. Be proud[ Go to top ]

    Wow! Thanks! That is exactly what I've been doing for the last several years.
    Me too, and anyone who wants to make a decent solution and can see through the emperors new cloths. A solution should not be massively more complex than it needs to be; no matter whose standards it might meet

    cheers
  7. Don't Be Afraid to Drop the SOAP[ Go to top ]

    People should absolutely be using SOAP as it 1) Alerts people to the fact that others have spent time thinking about the common problems that everyone is trying to solve and they've built standards to attempt to solve those problems, and 2) Along the standards lines, SOAP uses XSD. I don't know how many times I've had to write an integration without SOAP and so each one had a different date/time format. People don't need to waste their time implementing date/time parsing when a perfectly good spec and implementations exist to do this for you.
  8. People should absolutely be using SOAP as it 1) Alerts people to the fact that others have spent time thinking about the common problems that everyone is trying to solve and they've built standards to attempt to solve those problems, and
    SOAP is a largely work of people who did not respect previous work and research in the area (CORBA), or were not capable of comprehending it.


    SOAP and CORBA is like DOS and Unix, dependings on point of view one is success or another.
    2) Along the standards lines, SOAP uses XSD. I don't know how many times I've had to write an integration without SOAP and so each one had a different date/time format. People don't need to waste their time implementing date/time parsing when a perfectly good spec and implementations exist to do this for you.
    Yeap, do not reinvent wheels. Or at least do not make them square.
  9. SOAP is a largely work of people who did not respect previous work and research in the area (CORBA), or were not capable of comprehending it.SOAP and CORBA is like DOS and Unix, dependings on point of view one is success or another.
    I would go so far to add that SOAP is largely the work of people who do not understand SOAP :-). Initially SOAP was not conceived as some Corba substitute but has been bended one way and that to become one.

    Now SOAP is getting to a stage known from Corba where you are stuck with two languages (IDL / XML and your programming languages) and you have insufficient language binding. Also functionality wise SOAP is miles from where Corba was years ago (when you were able to stick with one vendor, of course :-)).

    Oh, someone was talking about "common problems solved by SOAP". Hey great, how about one single concise security spec, one single and vendor compatible notion of a user session and so on....and I am not starting about reliability and this kind of stuff. I sometimes feel that SOAP Specs deliberately ignores simple stuff and concentrates on complicated and cpu intensive stuff with limited usage, for example stuff like SOAP message routing.
  10. Don't Be Afraid to Drop the SOAP[ Go to top ]

    People should absolutely be using SOAP as it 1) Alerts people to the fact that others have spent time thinking about the common problems that everyone is trying to solve and they've built standards to attempt to solve those problems, and 2) Along the standards lines, SOAP uses XSD. I don't know how many times I've had to write an integration without SOAP and so each one had a different date/time format. People don't need to waste their time implementing date/time parsing when a perfectly good spec and implementations exist to do this for you.
    Well put. Think about if you have to build new systems on top of existing ones as time goes. I often find that though you may feel that 'insert your flavour of the week technology' will be around forever, technology changes and the systems need to be able to build on each other.

    The fact remains, SOAP works great to integrate .NET and J2EE.
  11. The fact remains, SOAP works great to integrate .NET and J2EE.
    I guess not for long. I have read some guidances which tell that developers should use 'binary' remouting when they want to create something 'real' and 'efficient' and use SOAP if thei want to ensure interoperability. So, they use WSDL as IDL and go binary (kind of makes sense, but IDL is more human friendly IMO), that supports claim that XML-RPC is 100% certified as inefficient protocol. (But a great way to generate CPU sales ).
  12. steer clear of xsd:dateTime[ Go to top ]

    I would absolutely (a) agree with your 'dont define a new dateTime format' and back it up with (b) dont use xsd:dateTime if your SOAP/XSD stack maps it to native objects

    Axis uses the Java calendar type, and on self tests in the UK (and only the UK region) xsd:time objects lose an hour on localhost/localhost messages.

    .NET uses their Date type which doesnt contain any zone info; all timestamps are assumed to be in local time. If they are in GMT you have to presubtract/add the offset before sending.

    All my recent XSDs for WSDL use time_t defined as an integer in UTC, measuring seconds since 1/1/1970. Of course, then I have to deal with leap seconds, or ignore them...
  13. Don't Be Afraid to Drop the SOAP[ Go to top ]

    1) Alerts people to the fact that others have spent time thinking about the common problems that everyone is trying to solve and they've built standards to attempt to solve those problems


    aka - alerts people to the fact that you have nice brochure with great marketing literature surrounding an over-engineered standard. In addition, as soon as the standard has a bad rep (which SOAP does in terms of performance, reliability, bloat, complexity, etc), this alerts people to stay clear of your product ;-)
    2) Along the standards lines, SOAP uses XSD. I don't know how many times I've had to write an integration without SOAP and so each one had a different date/time format. People don't need to waste their time implementing date/time parsing when a perfectly good spec and implementations exist to do this for you.
    I don't know how many times you've used XSD :-) if you have such a glowing review of it. It is a pain in the ass. I highly recommend looking into RelaxNG http://www.relaxng.org/. Now the fact that WSDL enforces the use of XSD is a limitation not a qualification.
  14. I'm afraid that I've been coming to the conclusion that SOAP is the biggest problem with web services. Not a problem with SOAP, but with the way it is used.

    A lot of post pro SOAP has been going on about XSD, standards, and code generation; personally I see these as reasons that you shouldn't use SOAP.

    Consider standards; WSDL (better in WSDL 2.0) has TWO standard bindings, SOAP and HTTP (==XML over HTTP) though you wouldn't know it from looking at "Industry standard" usage. You can concisely define non-SOAP XML services using WSDL: you get standards compliance, XSD, type reuse, even (in WSDL 2) interface extension - all without so much as a bubble of SOAP (and you have parameterised GET operations, which is cool).

    As for autogeneration, by all means autogenerate your platfrom specific code but auto generation of WSDL is eeevil!! I've just come out of a project where we implemented a web service using Apache Axis and auto WSDL generation, and the fact of the matter is that WSDL auto generation doesn't work (software engineering; it works literally). Types are converted ambiguiously (eg nillable or not?) so all that XSD is nearly useless for message validation. Documentation must be manually supervised - sure variable names get through (if you're lucky) but as for descriptions, you'll be hand coding it.
    Sure we get some cross platform benefits, but we end up using just another opaque RPC serialisation protocol, XML only in fact, not in spirit.
    (plus, with auto generation WS ends up being used as nothing more than an inefficient RPC/remoting system, with no attempt to distinguish method and operation - so much for message oriented programming.)


    I've come to realise that the only way to implement concise platform independent XML definitions is to actually write them. I'm going to be campaigning for WSDL as the primary standard in any future webservices projects I'm involved in; if I'm going to "use" XML then I want to actually use XML (that way if it isn't important enough to code XML for its not going to be added). I'll use SOAP if I need to, if I need SOAP features, or unfortunately, more often (since SOAP has so much momentum) anywhere I can't influence both ends).

    Ok , that turned into a rant, but you get the idea... SOAP does not have a monopoly on standards, and is used not only to support interoperability but, unfortunately, to perpetuate legacy paradigms.

    (as a side issue, security with SOAP leaves a lot to be desired - consider signing the message: SOAP adds a few tags and a few extra namespaces, no worries.... canonicalise and the 'few' namespace declarations are replicated to every child element, massively increasing size ... then you hash the whole lot; even if you don't send in canonical form this makes processing very slow for complex messages; encryption is faster, but encryption won't always do)
  15. It's Not a Terrible Idea[ Go to top ]

    I developed a software product that used XML over HTTP, with great success. At the time, the choice was made because of incompatibilities between Microsoft and Sun SOAP implementations. It probably succeeded largely because we had control over the software at both ends of the pipeline. While we had our XML API documented, we never expected anyone to use it, and they never did. So in that controlled environment, the lack of validation against a schema wasn't a problem, and "dropping the SOAP" was a bonus.

    Now, with greater standards support, more libraries for XML development, and so on, I'm not sure we'd make the same decision, but I can say with confidence that using XML over HTTP without SOAP wasn't a BAD idea.
  16. Re:It's Not a Terrible Idea[ Go to top ]

    I developed a software product that used XML over HTTP, with great success. At
    >the time, the choice was made because of incompatibilities between Microsoft
    >and Sun SOAP implementations.
    And what does it mean incompatibilities here?


    Thank you,
     Dmitry Namiot
     http://www.servletsuite.com
  17. Re:It's Not a Terrible Idea[ Go to top ]

    I developed a software product that used XML over HTTP, with great success. At
    >> the time, the choice was made because of incompatibilities between Microsoft
    >> and Sun SOAP implementations.

    > And what does it mean incompatibilities here?

    Note that I said "at the time" - essentially, the SOAP libraries used with IIS handled headers differently than the standard. I didn't experience this myself, but on a tight project schedule, there was no way I was going to put the project at risk by hoping the anecdotal evidence of incompatibilities I was hearing was wrong.

    I wouldn't be as concerned about it now. The key point wasn't that Microsoft and Sun couldn't interoperate using SOAP. The point was that we used XML of HTTP, without SOAP, in an enterprise product and didn't regret it - in fact, it was quite nice.
  18. Don't Be Afraid to Drop the SOAP[ Go to top ]

    I always thought http://www.xmlrpc.com/spec was enough. Back in 2000 we used that spec to communicate between python clients and JBoss. We had to extend it a little to be able to plug in context based information like security, but it was really simple. The code was simple and compact and easy to morph to our needs.

    We used it for both synchronous RPCs and an asynchronous custom queue.

    Bill
  19. The Question is: Why SOAP?[ Go to top ]

    As a consultant I’ve been to many projects where the decision to use SOAP was taken “just because it’s a standard!”

    Although this may seem a pretty rational choice at first glance, it shouldn’t be the only reason. SOAP carries a lot of overhead when implemented and most projects don’t really dig enough into this subject before the decision is made, hence issues are discovered too late and you hear developers asking “Why are we doing this?!”

    The overheads range from performance problems, development hardships, and difficulty to test and debug, to the fact that different SOAP stacks don’t necessarily work that well together.

    The decision to SOAP or not to SOAP has to be made in light of all the issues surrounding this topic and there has to be a very good reason to SOAP and not just settle for proprietary XML formats over HTTP (or anything else for that matter). And as obvious as it may sound; knowing what you’re getting yourself into is key, but for some reason this is not as obvious to many people that just follow the hype as the sole justification for their technology choices.

    </AT>
  20. ambiguous title ;-)[ Go to top ]

    reminds me of these movie scenes under the prison showers... you know what I mean ;-)
  21. ambiguous title ;-)[ Go to top ]

    Ya Think? I don't think it was ambiguous at all.
  22. Don't Be Afraid to Drop the SOAP[ Go to top ]

    We originally chose to use SOAP in a large scale project as the means of communication between many agents and a server. I fought a lot of the developers over my interest in using SOAP, based on the argument of the API being easier to deal with.

    In the end, we found that the system would just not be scalable due to the overly verbose nature of all the messaging. SOAP is definitely not the first thing to look at when all you need is a simple serialization mechanism where the two parties involved in communicating know what types of objects they're expecting to exchange. All too often people fall into the trap of choosing SOAP when simpler solutions could do the trick.
  23. Burlap/Hessian[ Go to top ]

    Any your experience comments about this:
    http://www.caucho.com/burlap/index.xtp
    http://www.caucho.com/hessian/index.xtp

    Thanks,

    José.
  24. Burlap/Hessian[ Go to top ]

    Hessian is very, very good, excellent piece of work !

    It's binary, fast and makes a lot of sense at the protocol level. Also server setup is next to zero.

    Programming effort is also next to zero, provided that you're using either Java or Python, for which Caucho has libraries.
    I wrote a C++ implementation. Also added GZIP compression on the stream based on message size and added .NET style self-documentation of a service.

    I've found it to be a really good choice. Also the license is Apache, so there you go.
  25. Burlap/Hessian[ Go to top ]

    Spring has excellent support for Hessian and Burlap (as well as other remoting protocols). I've heard good things about those protocols, although not actually used them myself.

    Have to say, my experience with SOAP hasn't inspired great confidence. Sure there's a big theoretical promise in a standard that will provide interoperability, but unfortunately this one is far from "simple", despite its name. In fact if you look at what goes over the wire for simple invocations it's downright scary. OTOH there is a potential for wasted effort in implementing proprietary protocols, so SOAP may still be the lesser evil.

    Rgds
    Rod
    J2EE without EJB | www.springframework.org
  26. Burlap/Hessian[ Go to top ]

    If I remember correctly, Burlap is XML-based and Hessian is binary, correct?
    What are the pros and cons of Hessian vs. Burlap? Is Hessian faster? Just wondering.
  27. Burlap/Hessian[ Go to top ]

    Hessian and Burlap are essentially identical except at the very lowest level. The structure of the serialization is the same. Both can be thought of as small extensions to XML-RPC.

    Hessian is a little more compact and a little easier to parse. Parsing Hessian is basically a switch statement while parsing Burlap requires XML tag parsing.

    So Burlap marks data with XML tags: <int>34</int>

    And Hessian marks data with a byte code: I\x00\x00\x00\x22, where 'I' stands for Integer, 'D' stands for Double, 'S' stands for String, etc.

    For binary and string data, Hessian sends chunks of bytes. b\x80\x00 marks a middle 32k bytes of data and B\x00\x05 marks the final 5 byte chunk of data. Burlap needs to use base64 encoding. Again, making Hessian a bit more compact and easier to parse.

    (Burlap was created to demonstrate that SOAP's complexity was not really required for its task. Others can judge if it was successful.)
  28. Scott: "Burlap was created to demonstrate that SOAP's complexity was not really required for its task".

    Most of the current tools today for design and implementation of Web Services works with objects, IMO it is better to use the document/literal style based messaging so you can deal directly with the XML data rather than a hierarchy of objects. That way, interact by "exchanging documents" rather than by "invoking methods" is not so much different from XML-RPC but have the additional advantage that it play nicely with the different emerging standards (for example transactions).

    Many of the early Java implementations used the RPC/Encoding style by default -MS has document/literal style as default. AS usual the fight is between the OO-people and the KISS-people.

    Regards
    Rolf Tollerud
  29. AS usual the fight is between the OO-people and the KISS-people


    KISS and OO are not mutually exclusive. Sometimes "keeping it simple" actually leads to more complexity and difficulty. Keeping it as simple as possible is as best we can do. I've done plenty of non OO apps and looking back, OO would have made much of it simpler.
  30. AS usual the fight is between the OO-people and the KISS-people
    KISS and OO are not mutually exclusive. Sometimes "keeping it simple" actually leads to more complexity and difficulty. Keeping it as simple as possible is as best we can do. I've done plenty of non OO apps and looking back, OO would have made much of it simpler.
    There is nothing wrong in OO, but it is very easy to use it in the wrong way like XML -> OO model for XML -> OO model for ORM -> RDBMS -> OO model for ORM -> OO model for XML -> XML.
    Result is the same as XML-> SQL ->XML (no programming), but it is evil because there is no objects, is not it ?
  31. There is nothing wrong in OO, but it is very easy to use it in the wrong way like XML -> OO model for XML -> OO model for ORM -> RDBMS -> OO model for ORM -> OO model for XML -> XML.
    It is not the wrong way if the application is an OO application. If part the app is OO and part of it is RDBMS with all the business rules embedded in the db, then there is no point it.
    Result is the same as XML-> SQL ->XML (no programming), but it is evil because there is no objects, is not it ?
    If the application is an OO application then the problem is not that there are no objects but that a layer was skipped. But if you don't believe in OO applications, then your point is valid.
  32. Yes, I do not believe object oriented programming is a good way to transform data, because it is possible to do it without any oriented programming, I think OOP is the best way to implement transformation engine, but not transformation itself.
  33. Yes, I do not believe object oriented programming is a good way to transform data, because it is possible to do it without any oriented programming, I think OOP is the best way to implement transformation engine, but not transformation itself.
    Never would have guessed that you believe that. :) Again, it comes back to presupposition. Guess what is good for the goose (transformation engine) is not good for the gander(the transformation).
  34. xalan is this kind of this engine, you can transform xml to DML and execute it, it is more trivial to transform resultset to XML, there is an example in JAVA tutorials too. The same is about "business logic", it must be better to write a tool to declare business rules if triggers and constraints can not help than to hardcode stuff I do not undersdand (I am not a business analys) in objects or procedures. I have no problems with JAVA, OOP or ASM. I am talking about "That" vs. "How" stuff, not about object vs. procedure. Probably I am too sarcastic in this forum, but I see no mojor difference between OOP, procedures/structs and ASM, it is about the same stuff ("How").
  35. Burlap/Hessian[ Go to top ]

    I just createt simply test to compare CORBA/Hessian/Burlap.
    Started from IDL:

    module KGIRiaTest
    {

      struct ContactVO{
        string type;
        string value;
      };


      struct UserVO{
        string login;
        string fullName;
        sequence <ContactVO> contacts;
      };

      typedef sequence <UserVO> ListUserVO;

      interface RiaServer
      {
        const string SERVER_NAME = "ria_server";
        UserVO getUser( in string login);
        ListUserVO listUsers();
      };
    };

    Than run CORBA service:

    public class RiaServerImpl extends RiaServerPOA{

      public UserVO getUser(String login) {
        return createUser( login );
      }

      /**
       *
       * @return 13 users
       */
      public UserVO[] listUsers() {
        return users;
      }


      public static UserVO createUser( String v ){
        UserVO userVO = new UserVO();
        userVO.login = v;
        userVO.fullName = v + " " + v;
        userVO.contacts = new ContactVO[]{
         new ContactVO( "email", v +"@" +v )
        };
        return userVO;
      }

      public static UserVO[] users = new UserVO[]{
        createUser( "A"),
      ...

    And implemented Hessian and Burlap servlet to expose the same implementation class:

    public class RiaServer(Burlap|Hessian) extends (Burlap|Hessian)Servlet implements RiaServerOperations{

        RiaServerImpl impl = new RiaServerImpl();

      public UserVO getUser(String login) {
        return impl.getUser( login );
      }

      public UserVO[] listUsers() {
        return impl.listUsers();
      }
      
    }

    And then create tests like this:
    String url = "http://localhost:8080/protocol_compare/RiaServerHessian";
          HessianProxyFactory factory = new HessianProxyFactory();
          RiaServerOperations riaServerOperations = (RiaServerOperations) factory.create(RiaServerOperations.class, url);
          TestingTimer.timeListUsers( "Hessian", riaServerOperations );

    and run then side by side:
    public class AllTestsRunner {

      public static void main(String[] args) {
        CorbaImplTest.main( args );
        HessianImplTest.main( args );
        BurlapImplTest.main( args );
      }
    }

    Summary: output from the method above:
    CORBA::time for 10 iterations = 33 ms
    Hessian::time for 10 iterations = 389 ms
    Burlap::time for 10 iterations = 265 ms

    I did not add Axis to the mix because it requires much more steps to setup on server and to compile client. I am too lazy :(
    When I compared Axis with CORBA couple of years ago Axis was much slower (about 50 times).

    Note: CORBA, Hessian and Burlap client uses EXACTLY the same interface defined in the same class for all servers.
  36. OK, I have added Axis to the mix and now results look like this:
    CORBA::time for 10 iterations = 82 ms
    Hessian::time for 10 iterations = 372 ms
    Burlap::time for 10 iterations = 159 ms
    Axis::time for 10 iterations = 1781 ms

    Disclaimer:I do not pretend that my test is comprehensive one and that it simulates real life environment, but it shows relative performance of 4 protocols.
  37. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    Comparing Hessian with CORBA (or in java RMI/IIOP) is like comparing apples to oranges. If you are in an environement where you can use CORBA or RMI/IIOP I'd say that it's the best choice from the speed point of view. You don't need hessian at all. If you need RPC outside your LAN and want to go HTTP to pass any proxy/firewall etc. Than you should consider using Hessian or a similar HTTP based remoting portocol.

    Do I need to mention that Spring has excellent support for these things ? :-) I really enjoied exposing my business logic to both thin and thick clients and being able to configure the thick client to use RMI for remoting when clients are deployed in LAN and hessian for remoting anywhere over http :-)

    Anyways, shouldn't hessian be faster than burlap ? theoretically ?
  38. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    Comparing Hessian with CORBA (or in java RMI/IIOP) is like comparing apples to oranges.
    Not quite if we consider just one scenario. That is what I did: compared those 4 for simple RPC call.
    If you are in an environement where you can use CORBA or RMI/IIOP I'd say that it's the best choice from the speed point of view. You don't need hessian at all.
    It depends, Hessian/Burlap is more convenient to use than RMI - no need for rmic/idlj etc.
    If you need RPC outside your LAN and want to go HTTP to pass any proxy/firewall etc. Than you should consider using Hessian or a similar HTTP based remoting portocol.
    SOAP,Hessian and Burlap meet proxy/fw transparency requirements, but B/H is vastly more convenient to use than SOAP if we can give jar file with interface classes to clients.

    I did my test mostly to compare convenience of use because I am sick of claims that SOAP is easier to use than CORBA.
    IMO CORBA is more convenient and that craziness about open ports on FW is just that: crazyness. It was said many times: allowing dangerous traffic on well known port does not make it less dangerous.
    Anyways, shouldn't hessian be faster than burlap ? theoretically ?
    Yeap, that is what I expected too. Caucho people, could you explain?
    More details and sources at http://www.kgionline.com/articles/protocol_compare/doc/index.jsp
  39. <em>Anyways, shouldn't hessian be faster than burlap ? theoretically ?</em>

    <em>Yeap, that is what I expected too. Caucho people, could you explain?</em>

    It looks like a Hessian implementation issue, not a protocol issue. The Hessian implementation wasn't reusing some StringBuffers like Burlap does (it's just been fixed). We do probably need some more performance work on the Burlap/Hessian implementation now that people are starting to use it more seriously.

    It's not surprising that CORBA is faster than Burlap/Hessian, although we can make the numbers closer. Even if both had dedicated underlying protocols (instead of Hessian using HTTP), CORBA knows the order of the serialized fields, while Hessian needs to do a map lookup for each field value. That makes Hessian easier to implement (especially for scripting languages), but does hurt the performance a bit.

    (For the main point of the article, straight XML for a POST/GET is a great solution for many applications, is probably underused, and has the great advantage of not needing any additional frameworks.)
  40. I call it "SOAP Light"[ Go to top ]

    Scott: "(For the main point of the article, straight XML for a POST/GET is a great solution for many applications, is probably underused, and has the great advantage of not needing any additional frameworks.)"

    That wraps it up nicely. But of course no computer "scientist" will admit that! :)

    And it must be asynchronous.

    Regards
    Rolf Tollerud
  41. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    (For the main point of the article, straight XML for a POST/GET is a great solution for many applications, is probably underused, and has the great advantage of not needing any additional frameworks.)
    Actually, the main point of the article was that avoiding HTTP or any other network protocol entirely is vastly more efficient. It talked about using a simple XML file format and loading the data directly with a command-line tool.
  42. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    Actually, the main point of the article was that avoiding HTTP or any other network protocol entirely is vastly more efficient. It talked about using a simple XML file format and loading the data directly with a command-line tool.
    Yes, the best way is to load flat file using awk if possible and if nobody makes you to use XML. I am sure POSIX will be reinvented again after this hype.
  43. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    Actually, the main point of the article was that avoiding HTTP or any other network protocol entirely is vastly more efficient. It talked about using a simple XML file format and loading the data directly with a command-line tool.
    Yes, the best way is to load flat file using awk if possible and if nobody makes you to use XML. I am sure POSIX will be reinvented again after this hype.
    Thanks for bringing back such good memories of how much, uh , fun I had last summer dealing with that nightmare. Luckily I had great people to work with.

    Yeah, its just data.
  44. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    Actually, the main point of the article was that avoiding HTTP or any other network protocol entirely is vastly more efficient. It talked about using a simple XML file format and loading the data directly with a command-line tool.
    Yes, the best way is to load flat file using awk if possible and if nobody makes you to use XML. I am sure POSIX will be reinvented again after this hype.
    Thanks for bringing back such good memories of how much, uh , fun I had last summer dealing with that nightmare. Luckily I had great people to work with.Yeah, its just data.
    Application server with WS and XML stuff, objects with business logic and classloaders or single evil command with cryptic awk (transformation engine) RE was this nightmare ?
  45. worst nightmare ever[ Go to top ]

    he Big Elephant EJB Java Application Servers are the worst nightmares ever to have roamed the earth. Nobody can guess the total cost of all the expensive project disasters. They could very well have been the real culprit behind the recent recession! 100 billion? 200 billion? 300 billion? Nobody will ever know.
  46. worst nightmare ever[ Go to top ]

    he Big Elephant EJB Java Application Servers are the worst nightmares ever to have roamed the earth. Nobody can guess the total cost of all the expensive project disasters. They could very well have been the real culprit behind the recent recession! 100 billion? 200 billion? 300 billion? Nobody will ever know.
    This elephant is a very powerfull thing to host components, but problem is with home made components and homepages not with containers, component must be reusable and customizible so it must contain no business logic (business logic is not reusable and it is not a subject of codding). ActiveX is a good example to understand this concept, see MS Office, this stuff is based on COP, but Word and Excel has no business logic and it makes them powerfull.

    Hint: Excel can take data from flat file using HTTP connection and to draw cool charts, give it for business analyst and he will implement business logic using Excel himself in five minutes, business logic is not a OOP business.
  47. Burlap/Hessian + CORBA + SOAP[ Go to top ]

    Is there a client for PHP or .Net applications for Hessian ?
  48. FYI, Spring's JPetStore sample app contains a similar demo: Its OrderService is exposed through 4 different protocols, namely Hessian, Burlap, Axis, RMI invoker, and Spring's own HTTP invoker. All of them are exported through the various remote exporters included in Spring, all delegating to the same target service instance.

    The demo client allows to specify an order id to look up, and a number of iterations. The response times are measured and printed, which gives a rough indication of protocol performance. Usually, Hessian, Burlap, RMI invoker and HTTP invoker are roughly at the same level, with Axis being far behind.

    An important point regarding remoting protocols is that it doesn't have to be an either/or choice: You can easily expose the same service through multiple protocols, at different endpoints. Clients can then choose the protocol for talking to the server: A Java client can choose Hessian, while a .NET client will choose SOAP.

    There's no need to use SOAP for Java-to-Java communication, just because there are .NET clients too... It's recommendable to choose a more efficient protocol for communication between the same platform, not SOAP which has been designed for cross-platform communication (incurring the tradeoffs and complexity involved there).

    Juergen
  49. Burlap/Hessian[ Go to top ]

    My friend Konstantin believe me most of Software people don't care about your pragmatic approach of comparing technology. As you know most of us are so shallow and just try to catch the buzz words. SOAP is the biggest buzz ever pumped to Software industry and it happened becuase of Microsoft and IBM monoply on the market as well as consortiums. But hey they got it what they wanna achieve, 4 years later all the CORBA vendors have preempted by SOAP and the webservices platform are totally dominated by these two vendors...
  50. After running into issues using a pretty widely accepted XML over HTTP format, I find it odd that people think SOAP is so bad. Right now we can't even communicate with some partners because different software vendors have implemented the standard differently. Now we are having to play lawyer with the standards and decide who is correct.

    If you have a limited number of partners, a simple lightweight (I think it's funny that XML over HTTP is called lightweight) solution may be fine. But the amount of time we waste implementing these solutions and trying to make them be all things to all people is not worth it. It's so easy to implement web service and get it working. I can't see any significant difference between the performance either. Where we use them we see no bottlenecks. Ironically, the processes that are bottlenecks are the custom XML over HTTP solutions. I think it's really a question of where you want to invest your development time. Not using tools such as web services means bringing in a bunch of hot-shot-kid consultants that make a big mess of our code and leave. There's no question in my mind as to what's a better approach.
  51. For small or middle size projects, it is unnecessary to consider so many standards.

    Make the design and implementation as simple as possible by using the technologies you are already familiar. If old technologies really cannot satisfy your requirement, donot hesitate to looking for new things.

    Even a few years ago, i believed that all the advertisement of SOAP will only benifit IBM and Microsoft etc big companies. Through washing your brain, they get what they want.

    Insigma Technolog Singapore.
    http://www.insigma.com.sg
  52. Don't Be Afraid to Drop the SOAP[ Go to top ]

    Is XML over HTTP enough?
    No. It's too much. Too much data. Too much processing. Too much hazzle.
  53. xml-over-http rules[ Go to top ]

    The SOAP vs. XML-over-HTTP debate is the analog of "EJB vs. lightweight" debate. Before deciding that you need SOAP, look at your requirements. If you have a fairly rich set of semantics for a request/response protocol, then the most strightforward way to implement is with XML-over-HTTP. SOAP will actually limit flexibility of your document or datatypes due to interoperability issues, and "profiles" (WS-I).

    Begin by defining an XML schema for your requests and responses. Then bind the schema to Java objects with JAXB, XMLBeans, or another binding framework. Use simple servlets to unmarshal the objects from the servlet input stream. Perform your business logic. Compose the response objects and marshal them to the servlet output stream. You can have your system done in a matter of a few days. It is extremely reliable, tunable, portable, and fast. Interoperability is based exclusively on XML schema. Therefore you don't run into the (ironic) interoperability problems that SOAP introduces.

    -geoff
  54. Most of the 'interop' problems between SOAP stacks is in different representations of stuff: dateTime, hashtables, integers (java doesnt do unsigned, for example). This is just the same as the classic O/R mapping problem, with the addition that the source/dest is some random implementation of the mapping in the other direction.

    If the SOAP stacks stopped trying to seamlessly map XML to native datatypes and presented you with a good XML interface to the data, and only did doc/lit at the same time I think things would be better. The SOAP stack in NetKernel from 1060.org does this, and I am thinking of doing the same in Java. No code there, just a working title: "Alpine".
  55. If the SOAP stacks stopped trying to seamlessly map XML to native datatypes and presented you with a good XML interface to the data, and only did doc/lit at the same time I think things would be better.
    1) You can configure this behavior in Axis if you wish. The service style is called "message". This opens the following methods:
        * public Element [] method(Element [] bodies);
        * public SOAPBodyElement [] method (SOAPBodyElement [] bodies);
        * public Document method(Document body);
        * public void method(SOAPEnvelope req, SOAPEnvelope resp);

    See their user documentation for more details: http://ws.apache.org/axis/java/user-guide.html

    2) ActiveSOAP (http://activesoap.codehaus.org/) is the new kid on the block, and this looks very very promising. This is a lightweight, embedable StAX based framework for document styled SOAP services. This at it's core, does what you are talking about. I suggest skipping "Alpine" and contribute your ideas to the ActiveSOAP project. ActiveSOAP may heal a lot of the Axis blues.

    - Mike
  56. I don't hate Soap. I dislike some of the implementations, and RelaxNG is a much more intuitive schema language than XSD, but I don't dislike SOAP.


    I HATE RPC Style Soap services. SOAP may not be perfect, but RPC style messages are the ugliest part. If you are going to incorporate SOAP in your applications, first please know why you are (and "it's the standard" is not a sufficient reason). Second, if you *must* use soap, implement document style services.


    http://www-106.ibm.com/developerworks/webservices/library/ws-docstyle.html


    - Mike
  57. With the help of JAXM[ Go to top ]

    as described here
    http://www.onjava.com/lpt/a/3653

    José.
  58. Not XML over HTTP, XML inside TAR![ Go to top ]

    Thanks for linking to my article, but I think you may have missed the point. I didn't use XML over HTTP for Krang, I cut the network out entirely and used a disk-based format combining TAR and XML.

    -sam
  59. Not XML over HTTP, XML inside TAR![ Go to top ]

    Thanks for linking to my article, but I think you may have missed the point. I didn't use XML over HTTP for Krang, I cut the network out entirely and used a disk-based format combining TAR and XML.-sam
    Haha, they didn't even read the article, I guess. How funny!
  60. I would second some of the comments[ Go to top ]

    For the last 3 years I've been working SOAP in one form another and my experience has been scaling soap webservices blows chunks. the amount of data is just too much to be efficient. The only cases where SOAP might be more efficient is over the internet where there's long lags and you compress the message with gzip. In this case, it's really only faster if the zipped content is smaller. Cases where the zipped message is still larger than the java serializable format, SOAP still looses.

    I've done simply performance tests with .NET ASMX page that simply responds with a XML document. I've also done simple echo tests to measure the memory and CPU usage. SOAP still has a long way to go before it really scales easily. Not that it can't scale, just that you have to scale horizontally. For small shops, that's a major problem, because it's not practical to tell a customer to buy 10 servers to handle the same load as non-soap version. SOAP has it's benefits, but performance and scalability is not one of them. that's my biased opinion after running a couple hundred benchmarks using real applications and synthetic tests.
  61. Soap based services[ Go to top ]

    We did use soap based services for integrating with 3rd party products. We also used Apache Axis with java clients and SOAP:Lite with perl clients for our EAI efforts and I would say, the going has been pretty smooth. It wasn't designed to handle heavy load(requests), but for the scope and the functionality, it was appropriate.

    I, however, agree that it chokes when the xml payload is pretty large and doesn't scale well. I also observerd that if we use the document style it scales well than when using the rpc-encoding style.

    Murali Kosaraju
  62. The problem is that people are trying to do very complicated things over a very simple base.. all of the WS-* standards (and their competing standards as well..) are layered on top of a fairly simple base.. the original SOAP spec. You had XML/RPC running via an HTTP transport, and SOAP was an evolution of that.. now you have a whole set of rather confusing standards clamoring for attention, and all adding a great deal of complexity to the XML message/payload.

    You also have sideways-competing systems, such as REST, which with Amazon and eBay are far more popular with developers than their SOAP counterpart.

    You have the overhead of the XML processing.. unless you use one of the binary transports.. which are also being pushed back into SOAP.. as in the case of the Fast Web Services being worked on by Sun. (ASN.1 encoding of the XML content to speed transport and processing) Also in the case of some of the Coucho work..

    Finally.. all of the "native" binary transports.. RMI.. CORBA.. etc..

    So.. I don't blame people for being confused. Its very difficult to understand how to use the expanding SOAP bubble.. its pretty easy to understand XML/RPC and REST, but they aren't as powerful. In a recent interview, a Microsoft product manager said that they expect the toolset to handle all of this for the user.. which is certainly one way to mitigate the complexity for the user..

    I guess one problem is that SOAP is making is difficult to do the 80% of what people do. 80% of people do not run banks.. they are not building trading systems.. they are building sync/async systems that need to talk.. frequently in a read-only/lookup capacity. The other 20% certainly need the emerging complexity.. but does everybody?

    I want SOAP to make it easy to establish a session.. authenticate myself.. secure my messages in some way.. and to exchange data with the server. I would guess thats probably what the the 80% of users need as well. As for the rest? Thats great.. but.. why drag everybody along?
  63. Perl[ Go to top ]

    We all realize this is a Perl application the article is about, right? My cat writes some decent Perl but he too is unable to debug his WebServices effectively.
  64. SWIFT[ Go to top ]

    I'm developing an application interfacing SWIFT services. SWIFT is an organization which produces standards for financial message exchange. The latest standard is based on xml and it's very sofisticated, yet it does not use soap.
  65. I've been using XML over HTTP for 6 years now, and it worked great for me. For the second version of my tools using it (3 and a half years ago), I considered using SOAP and web services, but like someone already said, there was too many incompatibility problems. Moreover, I was using a very deep XML tree, justified, but impossible to translate into automatic XML/Object mapping. If I did my web service in COM, COM was able to translate it into an IXMLDOMDocument object, but Java toolkits wouldn't understand anything, and the other way around. So I stuck with my custom solution.

    Now, I'd like to add advanced authentication, encryption, publisher/subscriber scenarios, and I like the WS-Attachment specification and philosophy, it looks like what I did; I'm not going to come up with my own solution to all this, now that web services are interoperable (somewhat:)). Moreover, with toolkits in use nowadays, I can access the XML very easily directly, it's even the preferred solution (just read Microsoft white papers for using WSE 2.0, they put the XML examples forward, they must have realized that they would never get to max speed with XML/Object mapping based on reflection...). After all, XML IS data in a standard form, with a very efficient requesting language (XPath), and you can also read it sequentially if you like. Why bother transforming automatically this XML in a big fat unreadable object with reflection APIs which are the slowest crawling the face of the earth? If you think XML is too verbose, zip it on the fly, use very small tag and attributes names.

    My 2 cents :)
  66. Don't be afraid to keep something (common sense, reason and logic)

    Fran: "Now, I'd like to add advanced authentication, encryption, publisher/subscriber scenarios, and I like the WS-Attachment specification and philosophy"

    Isn't it strange how it goes from one extreme to another? First you are only allowed to use standards - period, second you throw out everything with the bathwater, even the child.

    Why don't you stay within the standard and benefit from authentication, encryption, etc but still keep to the principles of KISS?

    Of the regular Web Services infrastructure plumbing provided of your tool just use the server-side stub - the incoming request stream is most often simple SQL anyway. Then you still have the default handler (WebServiceHandlerFactory), that enables the client to enter the URL of the web service and have the server side dynamically create an HTML page describing the service's capabilities and methods.

    But on the client-side don’t use the Soap proxy just plain old XmlHTTP. The data can be the simplest possible - just an XML document wrapped by the SOAP envelope.

    Avoid the deserialization of the XML data into an object hierarchy. Read the XML sequentially. Still KISS in spite of SOAP.

    Regards
    Rolf Tollerud
  67. Read the XML sequentially.
    And I thought you wanted to keep it simple. Automatic mapping vs hand coded xml parsing. Hmmm.

    What do you do with the, uh, "data" when you parse it? I hope you aren't using Swing, SWT, WinForms, VB, an OO Web Framework or the like.
  68. Now it is better that you sit down Mark, because you will get a nasty shock..

    Sometimes I don’t even need to read the data at all, just two "splits".

    hi hi..

    Hope you survived!
    Mark? Mark? Maaaaark?

    Hmm..
  69. Now it is better that you sit down Mark, because you will get a nasty shock..Sometimes I don’t even need to read the data at all, just two "splits".hi hi..Hope you survived!Mark? Mark? Maaaaark?Hmm..
    Luckily I was sitting down.

    No data? How will you survive? Fooey. Now I have to go wash my hands again cause I typed "data".
  70. Read again my comment...[ Go to top ]

    ... I actually said that I would go on reading the stream sequentially.

    What do I do with the XML I get? It depends. Sometimes, I just keep a DomDocument in memory to play with (I have made a very useful treeview whose model is based on a DomDocument and XPath queries to walk down the tree, a real pleasure to use:) I loooove XPath), sometimes I'll just extract some kind of return code, sometimes I get rid of it without reading it, and sometimes I will actually deserialize it automatically (in some very rare cases), but I choose to do it. I don't let the framework deserialize my XML stream in an object each time just because I use an OO language, I let him do it if it's useful. And in most cases, it's just a step i can live without. Think about it: if you receive a huge chunk of XML as a response but you just want to read a small part of it, do you think it's a good idea to deserialize the mammoth to read your small part? Of course not.

    It's not KISS vs OO in my opinion. It's just using the right tool at the right place. Maybe OO boys will say I'm a KISS jerk for saying this, I don't know :)

    Peace

    PS: I happen to be french, and actually my first name is Francois, but the "c" is normally written with a little curve below named "cedille" which means the "c" is pronounced like "ss". But it seems like the database holding our names has a character set missing my "cedille" :)
  71. Read again my comment...[ Go to top ]

    Yes, if you like XML then you must like XPath too (but I like flat files and awk more for many reasons) and there is nothing wrong to transform it directly to DML, RDBMS doe's not accept unauthorized DML and invalid data anyway (It can be a problem if constaints and access control is harcoded in objects)
  72. Style matters[ Go to top ]

    Still, SOAP does add value over plain XML/HTTP by explicitly defining operations and bindings. For broadly used services it is important: technically everything you need to know to use a service is in the WSDL and referenced schemas. With plain XML over HTTP you have to provide (and maintain) a separate HOW-TO document.

    It looks like many unhappy experiences with SOAP are coming from using RPC style generated directly from Java bindings, while document/literal style with a well-thought-through XSD and customized XML parsing is a better choice in most cases. Unfortunately, web service engine may force you to do extra serialization/parsing on event body to present it as a standalone XML and validate it against XSD (at least with Axis), which has nothing to do with SOAP protocol itself. I think addressing this problem is one of primary goals of ActiveSOAP project.
  73. Leon: "It looks like many unhappy experiences with SOAP are coming from using RPC style generated directly from Java bindings, while document/literal style with a well-thought-through XSD and customized XML parsing is a better choice in most cases"

    The people in TSS can not distinguish RPC/encoded from document/literal better than they can distinguish Mozart from Beethoven! ;)

    Regards
    Rolf Tollerud