Open source ESB projects Synapse and Celtix contributed

Discussions

News: Open source ESB projects Synapse and Celtix contributed

  1. The open source trend that's already existent for other "enterprise" technologies, including IDEs, message queues and identity management tools, is introducing itself in ESBs, commoditizing the normally costly technology.

    Iona recently announced that they would be contributing Celtix, an open-source version of their Artix product, to ObjectWeb. The Celtix preliminary features list includes Eclipse-based administration tools, binding support for SOAP and XML payloads and transports including HTTP, Java Message Service and the Web Services Reliable Messaging standard (WM-RS).

    Meanwhile, Sonic has similarly announced that they would be contributing Synapse, an open source Web service mediation framework, to Apache. Synapse is intended to intermediate between Web services to provide transformation and routing, promote loose coupling between services, and support greater reliability and resiliency.

    Both products are implementations of an enterprise service bus (ESB), which is an emerging standard for integrating in an implementation-independent fashion at a coarse-grained service level (leveraging the principles of SOA) via an event-driven and XML-based messaging engine (the bus).

    If you use technology like this, what effect will these releases have on your business? Where do you see ESBs being used?

    Threaded Messages (17)

  2. Please allow me to clarify that Infravio is donating all of the initial code of Synapse from its X-Broker project to seed Synapse, not Sonic.

    I speak with Glen Daniels and Dave Chappell on this team effort quite regularly. I have great confidence that the contribution of their efforts will be immensely valuable to Synapse.

    Other companies involved in Synapse include IONA, Blue Titan and WSO2. IONA's involvement in both Synapse and Celtix will help ensure that these two projects will have a mutually beneficial relationship.

    Just wanted to clarify this point.

    Based on the title of their recent press release
    "Sonic Software Advances Interoperability of SOA Infrastructure with Contribution to Apache Synapse Incubator Project" I could easily see how you could have gotten that impression.

    http://home.businesswire.com/portal/site/google/index.jsp?ndmViewId=news_view&newsId=20050824005113&newsLang=en

    Miko Matsumura
    VP of Technology and Standards,
    Infravio
  3. This is a Very Good Thing[ Go to top ]

    The ESB concept is a marvellous thing, especially when combined with a standardised API such as JBI (JSR-208).

    We are building a new back office system based on the ESB principles and are planning to use a JBI implementation to achieve this. Building a back office system is very much about integration with other legacy systems, especially when you are heading for a true straight through processing (STP) system. I'm not sure canonicalisation of the message payload data is a part of the ESB concept, but we do that also in order to wash/transform external system data to and from a local defined (but ISO-20022 close) XML format. This reduces the n-square integration problem into a linear complexity level when introducing new external protocols.

    We are taking the integration concept even further by modularising our own application into different service engines (JBI SE's) and use the ESB to integrate the different processing modules into a whole application. This breaks the usual monolithic approach and makes it easier to integrate all or only parts of our product into a customers (often very complex) system map. Combine the JBI-SE concept with Spring framework and JDO 2 and we have really mean and lean development process for complex business logic.

    The modularisation of our own application also makes it possible to scale up development of business logic, where different groups should be able to develop different modules and protocol adapters using the specification of the canonical/normalised messages as a contract.

    The JBI standard is already making it much more easy (and cheaper) for us to build a very complex system. By the use of JBI we can use ready made open source JBI-components such as bindings for JMS, SOAP, file, etc., as well as expose our service modules through ready made JBI-BPEL orchestration modules. Also, we can use ready made JBI components for content based routing. Still, there are a huge amount of challenges to build a back office system left, but the ESB architecture and the JBI standard will really remove a significant part of the work for us.

    Cheers,
    Johan Strandler
    Chief Architect, OMX Technology - Banks & Brokers
    http://www.omxgroup.com
  4. This is a Very Good Thing[ Go to top ]

    Interesting story...

    I'm writing the proposal of a new front-end application for a bank. They wish to replace their current not-scalable application with a new one.

    I think that an architecture based on ESB + JBI + JDO would result maybe the best choice for a system that had to talk with legacy systems.

    I'd like to know more about your success story...

    bye,
    Luca Garulli
    www.Pro-Netics.com (member of Orixo.com - The XML business alliance)
    OrienTechnologies.com - Light ODBMS, All in one JDO solution
  5. This is a Very Good Thing[ Go to top ]

    Johan,

    I very much agree with your assessment of JBI in general. It is particularly true that architecting applications following JBI concepts provides for a very clear componentization of business logic. In combination with dynamic (e.g. content-based) routing this enables a software system to be configured/assembled in a very flexible yet easily comprehensible way. Personally, I think the resulting architectures have a beautifully spartan, fluid elegance to them.

    Which JBI implementation are you using? I don't doubt that JBI will be widely adopted, but in the meantime I feel risk-exposed unless there are at least a few high-quality open-source implementations around. Currently, I only know of ServiceMix in that league.

      cheers,
      gerald

    http://www.gerald-loeffler.net
  6. This is a Very Good Thing[ Go to top ]

    Which JBI implementation are you using? I don't doubt that JBI will be widely adopted, but in the meantime I feel risk-exposed unless there are at least a few high-quality open-source implementations around. Currently, I only know of ServiceMix in that league.

    We don't use JBI yet but we will very soon. Currently we are testing the concept with Mule. But I think the best thing is to use JBI since it is a standard that makes it possible to mix different JBI containers developed by different vendors. ServiceMix seems to be a very good product well worth a try, especially since it is integrated with an application server (Geronimo).

    By using the JBI standard we also hope to more easily integrate our product with many of our customers systems, if they use products that supports JBI. For example, Tibco has announced JBI container support (http://www.pcworld.idg.com.au/index.php/id;1123996020), which I guess means that we could easily integrate/bridge our own product with customer systems that are using Tibco.

    Cheers,
    Johan
  7. This is a Very Good Thing[ Go to top ]

    Currently we are testing the concept with Mule. But I think the best thing is to use JBI since it is a standard that makes it possible to mix different JBI containers developed by different vendors...

    By using the JBI standard we also hope to more easily integrate our product with many of our customers systems, if they use products that supports JBI.

    The standard says nothing about mixing containers per say, rather, it deals with management, implementation and deployment of components, not containers. There is nothing in the standard that prevents you from invoking a service in one container from another, say through BCs (unlikely you could do so through different NMR impls) however nothing that enables it either. This would be akin to saying that you want to use multiple J2EE containers. Ok, fine, but the standard doesn't really deal with it directly. I'm not sure I would really recommend using multiple containers though, that would really increase the complexity of your deployment needlessly.
    For example, Tibco has announced JBI container support which I guess means that we could easily integrate/bridge our own product with customer systems that are using Tibco.Cheers,Johan

    "easily" would be a stretch considering the vague postiure of the spec. The spec is intentionally open-ended on things like remote invocations of SEs and security. Expect to see vendor extensions that have a lock-in effect for deployments of JBI components. As an example, ServiceMix allows clustering of various SEs, however only though a tight coupling with ActiveMQ. ServiceMix's POJO support also relies on a MessageExchangeListener interface, also not in the spec, also not portable. I'm not trying to roast SM really, they are adding convenience and functionality where the spec is lacking, just don't be mislead into thinking that if you write JBI components that they will be portable. Remember the lessions from EJB portability, until we have more vendors and can see how the landscape has shaken out, portability for JBI is a red herring.
  8. This is a Very Good Thing[ Go to top ]

    Hi Erik,

    You've pointed out somethings which probably means our documentation still needs improving :)
    ServiceMix allows clustering of various SEs, however only though a tight coupling with ActiveMQ

    Well yes, ServiceMix is tightly coupled to ActiveMQ/ActiveCluster - but only for the internal distrbution of message exchanges within the NMR for clustering and networks of JBI Containers - the SEs and BCs are completely independent from the use of ActiveMQ/ActiveCluster. Any JBI compliant Component can be dropped into ServiceMix and benefit from our container's remoting/clustering and networking.

    ServiceMix's POJO support also relies on a MessageExchangeListener interface, also not in the spec, also not portable.
    This is a convience - our POJO support doesn't rely on it - if a Component uses the interface - we will use it - else message exchanges happen in the normal way.

    The whole point of JBI *IS* to promote reuse of portable components and we have been working on creating standard deployments from our Pojo's that can be run in any JBI compliant container - but that's going to be the 1.1 release - which ain't to far away :).


    cheers,

    Rob
  9. This is a Very Good Thing[ Go to top ]

    Well yes, ServiceMix is tightly coupled to ActiveMQ/ActiveCluster - but only for the internal distrbution of message exchanges within the NMR for clustering and networks of JBI Containers - the SEs and BCs are completely independent from the use of ActiveMQ/ActiveCluster. Any JBI compliant Component can be dropped into ServiceMix and benefit from our container's remoting/clustering and networking.

    I see your point and it's certainly true for vanilla examples, however it's quite possible for NMR implementations and features to influence how components are built. For example, SM provides an XPath Router, but others may not. So, as part of a component implementation that needs to have runtime evaluation of the next endpoint, I may have to perform similar XPath expression analysis inside my component to ensure that things get routed properly unable to rely on the NMR to do it for me; these types of things are easily overlooked when implementing what you think is a portable component.

    Clustering is another good example, it's not uncommon for people to write cluster-aware implementations only to find out that clustering is done differently between vendors and certain assumptions in how components were implemented are not valid between containers. In component implementations it's likely that state management would be different in a clustered JBI container than a non-custered deployment for example.

    I see vendor provided services like clustering, security, remote invocation of components eventually leading to multiple extensions to the deployment and implementations of components. Clustering was one example, handling of lifecycle extension and installer configuration MBeans are others - the spec doesn't define how those components are handled, it's only a matter of time until vendors produce different interpretations that will affect component implementations.
    This is a convience - our POJO support doesn't rely on it - if a Component uses the interface - we will use it - else message exchanges happen in the normal way.

    And it's a great convenience, sort of an MDB without the DB part, I wouldn't want to do POJO SE development any other way. But, the original argument is still valid (at least until that shiny 1.1 comes out). Some of your best features like POJO support and the component support helper classes are vendor specific. It's not bad, it's not wrong, it's just an observation, something sure other vendors will also do, and something people building components in SM should know that isn't clear in the examples. It's strange how all the package imports on all the examples, could really help clarify where all the vendor-specific extensions are ;)
    The whole point of JBI *IS* to promote reuse of portable components

    Yes, as was the point of COM+ and EJB, both of which never produced a thriving middle market of component vendors demanding portability. That may be your interpretation of the point of JBI, I however tend to think the real value is more as a pattern for service implementation, it makes one think in terms of architectural services and adapters to those services and that is where I see real value, not in portability.

    Don't get me wrong, I think vendor independence is overrated and often requires slower, more bloated implementations. From first hand experience inside dozens of IT shops, it's very rare that you'll see multiple J2EE app servers in production and I don't think vendor independence is a real concern for most application teams, same with DBs (I like Jeremy Zawodny's take on DBs: http://jeremy.zawodny.com/blog/archives/002194.html). I believe the same will be true with JBI containers. Shops will pick a vendor and stick to it usually coupling themselves to spec extensions and utility classes like POJO support. I don't think that's bad, it just makes me cringe when people get up on the portability soapbox, especially with something as wide open as JBI.
  10. Don't get me wrong, I think vendor independence is overrated and often requires slower, more bloated implementations. From first hand experience inside dozens of IT shops, it's very rare that you'll see multiple J2EE app servers in production and I don't think vendor independence is a real concern for most application teams, it just makes me cringe when people get up on the portability soapbox, especially with something as wide open as JBI.
    From first hand experience inside dozens of IT shops (almost, dozens is a lot, >24 :-), it's very common that company migrate from WebXXXX to WebYYYY, or from WebXXXX to some J2EE open source alternative (or vice versa).
    Considering that commiting a company to an integration framework (such as an ESBalaJBI) is more important in the long term than choosing an app server for one or several application, and with the EAI so-so recent history, I fully understand why people are concerned with the real portability brought by JBI.
    Jacques
  11. Currently we are testing the concept with Mule. But I think the best thing is to use JBI since it is a standard that makes it possible to mix different JBI containers developed by different vendors... By using the JBI standard we also hope to more easily integrate our product with many of our customers systems, if they use products that supports JBI.
    The standard says nothing about mixing containers per say, rather, it deals with management, implementation and deployment of components, not containers.
    This dialog shows how confusing the JBI story is for potential users
    IMHO, JBI is not for end user integration programmers.
    It is for SW editors in the EAI space.
    It tries to enable a third-party market for BC and SE components
    and a base market for full size (NMR+ BC and SE basic components) JBI-complinat ESBs, just like JSR168 for portals. (As of to-day, there is BTW no JSR168 portlet market)
    The NMR itself remains proprietary, just as a JMS server, AFAIU.
    Jacques
  12. This is a Very Good Thing[ Go to top ]

    You can also integrate different JBI containers using Mule -
    http://mule.codehaus.org/JBI

    Not to mention expose JBI components using Mule transports, or embed your Mule components/transformers inside JBI.
    This gives you a lot of flexibility to mix different technologies based on the messaging needs of the application. Not all projects will want/need or be able to use JBI and may prefer to use something SOAP-based like Synapse or the Mule service container itself (Which doesn't assume Xml for the message format).

    Cheers,

    Ross
  13. This is a Very Good Thing[ Go to top ]

    I'd like to know more about your success story...

    Well, we don't know yet if it is a success story or not. :-) But from what we have done this far it looks good. The modularisation of our product really seems to benefit from the JBI/ESB infrastructure/architecture.

    FYI: In our case we use .NET for the client application and uses web services between the client and the server engines. The JBI abstract WSDL centric model with SOAP bindings fits very good into this architecture, at least in theory, but now we have to prove it in real life.

    Cheers,
    Johan
  14. Incidentally the ServiceMix and Celtix open source projects are working together to create a unified open source JBI container and powerful JBI component suite with complete integration capabilities along with enterprise messaging features. Each project will reuse and redistribute the other's code to provide a complete integration stack and avoid duplicated efforts.

    The ServiceMix and Celtix teams are also contributing to Apache Synapse, which is a broker/ESB built around Apache Axis - in particularly the ServiceMix team will be helping to add JBI support to Synapse so that Synapse can be deployed easily in any JBI container (such as ServiceMix's).

    The ServiceMix JBI container is already integrated into Apache Geronimo to enable Geronimo to auto-deploy any JBI deployment unit today; so these efforts will enable any Synapse or Celtix JBI components to be easily auto-deployed into Geronimo and managed through ServiceMix's JBI management stack.

    Its great to see consolidation and co-operation in the open source integration space.

    James
    LogicBlaze
  15. I do not see the point of ESBs[ Go to top ]

    My opinion about ESBs is that they are more a marketing term that a sound concept. However, given the frenzy that can be read about them in many places, maybe I am wrong.

    Beyond the useful facilities of asynchronous messaging and publish/subscribe (which provide the temporal and functional decoupling needed in many interactions - but by no means in all interactions), an ESB is just yet another container of web services, like a servlet container, a J2EE server or a .Net server. The only meaningful difference I see with them is that the services hosted in a ESB are usually created by scripting other web services (often, designed graphically) - i.e. something like a Unix shell, allowing to create new services by easy recombination of other existing services.

    Now this is a handy thing to have in many cases, but just as the Unix shell is not the center of the architecture of the OS or the applications running on top of it, I do not understand how an ESB is supposed to be the cornerstone of a SOA. For me, such a "service scripting engine" would be much useful to implement some new services, but just at the same level as other services, being just another way among many others of creating them: you can use it for some things, but you have not to.

    I think that a typical ESB will end up being a bag of domain (business) logic, on top of other business logic. And, being a new thing, the ESB must provide new means to develop, organize, manage and govern all that new logic - different from the way other more mature containers do, and thus that can be better - or worse.

    Also, giving it such a paramount importance is just the same as saying that the other containers (e.g. a J2EE server) are not a means good enough for creating web services.

    Besides, the ESB collides with the concept of a service registy/directory used at runtime (i.e. UDDI), which I think is one of the foundations of the flexibility of a SOA, providing a means for services to discover each other and thus allowing for dynamic binding.

    For me, one of the best things of a SOA is that it is a republic - all services are equal and can talk to each other freely, not being a need (or vendor recommendation) for them to always go through a new, higher level component other than a directory. I do not see there a place for a "bus" through which all traffic must go through.

    Cheers
  16. Celtix MIA ?[ Go to top ]

    Have IONA released Celtix yet ?

    Mik
  17. Celtix MIA ?[ Go to top ]

    Not yet. check out http://celtix.objectweb.org for more information. Project has been making lot of progress since its announcement. Milestone 1 is currently targetted for Mid September and Milestone 2 (major one) for sometime end of year. See their recent update on the project, http://www.objectweb.org/wws/arc/celtix/2005-08/msg00001.html

    - Adi
  18. Both products are implementations of an enterprise service bus (ESB), which is an emerging standard for integrating in an implementation-independent fashion at a coarse-grained service level (leveraging the principles of SOA) via an event-driven and XML-based messaging engine.

    Some corrections here IMHO.

    1) From Dave Chappell, the guy who literally wrote the book on ESBs:

    "We decided we were not going to call Synapse an ESB."

    And, it's not one. It is a nice layer of WS mediation for use across an enterprise, but not by any means an ESB.

    2) ESBs have been around for quite some time and I wouldn't call them a "standard", there is no well agreed-on term for what an ESB is and there is no group of people working on such terminology. JBI is a standard, ESBs are products that help implement enterprise integration through things like messaging and intelligent routing.

    3) While XML is important to an ESB, my personal vision of what an ESB is includes many message formats that are not XML. For example, JBI allows BCs that speak many wire formats such as pulling messages from an IMAP folder. Mainframes rarely speak in XML documents. There are numerous examples but it's important to not lose sight of the fact that integration is never an XML-or-nothing world.