Discussions

XML & Web services: SOA increases number of messages?

  1. SOA increases number of messages? (9 messages)

    I was looking into the several interpretations of ESB in the SOA world and one thing that struck me was a claim from a related infrastructure vendor that SOA means increased number of messages and message volumes within the enterprise and I am having trouble understanding why this should be the case.

    Adopting SOA, from an enterprise's persective, can translate into a mindless rewrite of all inter-application interfaces in the enterprise to use Web Services instead of what it is using today. To take the madness one step further, one may even consider making related applications, for example Accounts Receivable and Invoicing from the same ERP package vendor, to communicate with one another using Web Services. If an enterprise chooses to go down such an unreasonable path, I can understand why the # of messages and the volume can go up.

    However, if we are to step back and look at SOA to mean re-implementing only those interfaces in the enterprise that add real business benefit as Web Services, SOA should not and does not increase the number of messages in my understanding. The messaging volumes might go up by some fraction owing to the overhead of XML, but beyond that there isnt much of an impact.

    Or am I completely off the mark here?

    Surajit Bhattacharjee
    Virtusa Corporation
    www.virtusa.com
  2. You sound to be on the right track to me :)

    Use the right tool for the job. e.g. don't use RMI when things are deployed in the same JVM. Similarly if 2 Java processes are communicating with each other and use the classpath, consider moving around Objects over JMS, RMI (or Spring remoting). You only need to use Web Services where they make sense to do so - such as communication across different versions or languages of software (such as Java & C & .Net).

    There are techniques for managing the increase in number of messages, such as by using caching in the SOA layer or using aggregation to introduce more coarse grained service requests.

    James
    LogicBlaze
  3. Thanks for introducing me to ServiceMix [ and JBI - I had ignored that JSR so far :) ]. I do not know if a stanadard for a Java ESB is going to take off - especially since it can undermine the market share of big players like IBM, WebMethods et al - but it is interesting to note that ServiceMix decided to use ActiveMQ for the messaging fabric.

    The reason I say it is "interesting" is due to the popularity of messaging infrastructures that have a peer-to-peer architecture at their heart, thereby allowing such infrastructure to scale much better than more centralized "server-like" implementations that used to dominate the hub-spoke days. I am not saying that hub-spoke is no good - you do need some of that, especially for administration, security, business process auditing et al (something that products like Fiorano have been able to incorporate in their ESB architecture).

    Surajit

    Virtusa Corporation
    www.virtusa.com
  4. I just ran the benchmark case on ActiveMQ and it rocks! I am yet to run loads on this other messaging fabric that is "serverless" at its heart - but I can't wait to see how they compare.

    It says on some site that ActiveMQ uses Straight Thru Processing, Reactive Scalable Flow Control, High Performance Journaling etc to give the throughtput it gives. Is there any place I can read more on these?

    Surajit
    Virtusa Corporation
    www.virtusa.com
  5. I just ran the benchmark case on ActiveMQ and it rocks! I am yet to run loads on this other messaging fabric that is "serverless" at its heart - but I can't wait to see how they compare.

    BTW ActiveMQ also supports a peer based serverless mode as well - just connect with peer://clusterName and you're all set.

    By all means email us at dev at logicblaze.com and we can send you a performance report.
    It says on some site that ActiveMQ uses Straight Thru Processing, Reactive Scalable Flow Control, High Performance Journaling etc to give the throughtput it gives. Is there any place I can read more on these?SurajitVirtusa Corporationwww.virtusa.com

    We don't yet have great documentation describing the implementations of the broker itself - but here's a start

    http://activemq.org/Code+Overview

    James
    LogicBlaze
  6. Thanks for introducing me to ServiceMix [ and JBI - I had ignored that JSR so far :) ]. I do not know if a stanadard for a Java ESB is going to take off - especially since it can undermine the market share of big players like IBM, WebMethods et al - but it is interesting to note that ServiceMix decided to use ActiveMQ for the messaging fabric.The reason I say it is "interesting" is due to the popularity of messaging infrastructures that have a peer-to-peer architecture at their heart, thereby allowing such infrastructure to scale much better than more centralized "server-like" implementations that used to dominate the hub-spoke days. I am not saying that hub-spoke is no good - you do need some of that, especially for administration, security, business process auditing et al (something that products like Fiorano have been able to incorporate in their ESB architecture).SurajitVirtusa Corporationwww.virtusa.com

    Agreed. I think the fact that there is a good open source JBI container available may help encourage market adoption of JBI.

    BTW its also worth noting that ServiceMix can be embedded in the edges of the network in a peer type deployment as well as being deployed in a federated nework or hub & spoke - so users can choose the most suitable deployment topology for their needs.

    James
    LogicBlaze
  7. Thanks! Just shot out the mail for ActiveMQ benchmarks :).

    Also went through the support for peer-to-peer deployment in ActiveMQ. Why do you say that TCP is better than multicast for the heavy lifting associated with the real messages? In a pub-sub (1-to-many) environment, wouldn't multicast have a lesser strain on bandwidth or does the overhead of UDP reliability negate all of that gain?

    Surajit

    Virtusa Corporation
    www.virtusa.com
  8. Thanks! Just shot out the mail for ActiveMQ benchmarks :).Also went through the support for peer-to-peer deployment in ActiveMQ. Why do you say that TCP is better than multicast for the heavy lifting associated with the real messages? In a pub-sub (1-to-many) environment, wouldn't multicast have a lesser strain on bandwidth or does the overhead of UDP reliability negate all of that gain?SurajitVirtusa Corporationwww.virtusa.com

    If you don't care about reliability and you're worying about network bandwidth and not too worried about performance, UDP/multicast is fine.

    Though most of the time folks are worried about performance and reliability. On all our testing, TCP beats it hands down as the hardware is doing all the hard work - rather than reiimplementing TCP/IP in Java code at the packet (datagram) level.

    With RDMA and Infiniband, increasingly networking is happening on the hardware network card, not the CPU - so we find increasingly for reliable, fast networking in a heterogenious environment with different messgage flows, that TCP wins hands down.

    Also these days there's normally tons of network bandwidth - its CPU & latency that tend to be the main issues. Note though that multicast is great for discovery (such as in our peer:// transport).

    James
    LogicBlaze
  9. Thanks! Just shot out the mail for ActiveMQ benchmarks :).Also went through the support for peer-to-peer deployment in ActiveMQ. Why do you say that TCP is better than multicast for the heavy lifting associated with the real messages? In a pub-sub (1-to-many) environment, wouldn't multicast have a lesser strain on bandwidth or does the overhead of UDP reliability negate all of that gain?SurajitVirtusa Corporationwww.virtusa.com

    Hi Surajit,

    In a coincidence I saw the benchmark document that LogicBlaze are publishing comparing ActiveMQ to some other MQ vendors and are displaying MantaRayin a very distorted way, and I can understand why the do so :-)

    I just want to raise few distortions found in their document regarding MantaRay:

    1.
    The MantaRay architecture supports more than one persistency mechanism. And while it is true that users can use database in order to persist messages it is strongly recommended that they use the file based persistency mechanism. Using database in order to persist messages would result in a ten fold degradation in performance. In other words if you use the same 1 sender / 1 receiver / Queue scenario they claimed to have run, with file persistency you would get somewhere between 6000 to 7000 messages per second rates instead of 425 they claimed to have seen.
    So as rule of thumb developers should always opt to use the file system persistency mechanism as opposed to the database one. You can now imagine why they use the DB persistency at their benchmark; otherwise they couldn't show how ActiveMQ performs better.

    2.
    Topics - Again continuing the theme of presenting only what suits them, I was not surprised to see that they neglected to mention MantaRay in the Topics section. MantaRay easily outperforms any other messaging system they checked (or presented in the document). In a 1x1 Topic scenario MantaRay will do around 7000 messages per second. Now the real strength of MantaRay stems from its distributed nature. Running MantaRay in a 50 producers / 50 consumers / 50 Topics scenario would yield ~250,000 messages per second, imagine what that would do to the graphs they put in that document, it will be very hard to read or see the ActiveMQ numbers at the graph.

    Please email me if you want me to send you the MantaRay benchmark document.

    Best regards,

    Alon Eizenman
    Coridan
  10. James,

    It is very interesting to see Google Ads from competitors (Cordian, Arjuna, SwiftMQ) on your web site.

    As a coincidence, I am looking into MantaRay from Coridan :).

    Surajit