Discussions

News: JavaOne Preview: Unlocking the ESB Secrets

  1. JavaOne Preview: Unlocking the ESB Secrets (15 messages)

    During JavaOne, developers will get bombarded with the buzzwords du jour such as SOA, and ESB (Enterprise Service Bus). Dave Chappell, in his latest book spells out why ESBs will transform how Java devs think about APIs, data sharing and workflow. See what he says, and give your opinion.

    Excerpt
    ESBs Update Pure Java-Driven Integration

    Chappell says that the standards and technologies that will comprise an ESB have matured enough that it’s safe for Java devs to begin thinking about leaving 100% Java-driven integration behind.

    Traditional Java messaging and middleware approaches, he says, focus on “one point-to-one-point” or “one-to-many” data transfers. These approaches do not easily upgrade to a services-oriented architecture “many-to-many” approach for sharing data, as well as higher-level application and business logic as services.

    Chappell cites five (5) reasons why CIOs will consider using ESBs, and shift away from traditional integration approaches:

    • Increased ability to integrate inter-departmentally, as well as with business partners using standard interfaces and standard protocols.
    • Leverage existing IT staff as architects: The amount of information available on Java standards such as JAXP, JCA, and JMS, and XML-related standards such as SOAP, WSDL, XSLT, XPATH and XQuery is expanding at an ever-increasing rate. This means that the average IT professional can readily attain the expertise he or she needs to become the in-house integration architect.
    • Reduced vendor lock-in by using a standards-based approach to integrating at the protocol, data and business intelligence layers. ESBs will reduce the appeal of proprietary middleware or integration stacks.
    • Reduced need for wide appserver deployments: Because integration functionality and business process routing is inherent in the ESB fabric,, IT shops can avoid the need to purchase and install J2EE app servers at all target and source integration points throughout their organization, simply for the purpose of providing integration functionality.
    • More predictable project milestones/deliverables: The ESB standards-based integration approach means that devs can reuse common design patterns and successful templates/models from project to project, without expensive consultants and vendor lock-in.
    Read: JavaOne: Unlocking the ESB Secrets
  2. As a Brit I find it very hard to get used to ESB not meaning "Extra Special Bitter", the beer of choice for getting paralytic for school kids and Scots. :-)
    I'm not a great fan of marketing although I have to admit Microsoft seem to do very well with it. ESB, apart from a rather rough beer means a whole number of things, it's not unlike WebServices, what is the definition of WebServices? I agree with a lot of what Chappell says, certainly the old integration problems of the past seem to be still here and still as big as they were 5, 10 or 20 years ago, there's just more things to integrate now. My company (C24) make a business out of integration, we came from Braid in the 80s then Mercator in the late 90s and finally set off on our own in 2000 to develop business process monitoring, modelling and management systems. At the time we thought EAI was solved, a few years later having given up trying to sell BPM engines, we're now selling the next generation integration components and bizarrely the ESB vendors seem to like OEMing us. To us it's come around full circle, EAI is still at the core of most enterprise systems, it's just that they now call is SOA or ESB. I'm sure the marketing people would love to tell you that ESB is really a level above EAI, perhaps, but they are really the same thing at the core, we've just got a slightly more mature approach to the design and a lot more tools.

    From a technical stand-point I see a rather nice niche in Grid enabled ESB, we're using our Integration Objects (Java-ized versions of data but with executable code for rules etc.) for integration, the old EAI with GigaSpaces' Jini and JavaSpaces for the Grid. We can deploy generic workers in the JavaSpace to execute rules and ECA actions on the Java-ized data. Result, truly parallel, scalable, very high performance ESB (for want of a better acronym).

    I like Sonic, they have good vision and good products, they have also just signed some partnership deals with Enigmatic, a very wise move, so I think we're going to see a lot in this space.

    In short, ESB to me will always remind me of the last few years of school until someone can define it better and convince me that it's totally different from EAI.

    -John-
    CTO, C24
  3. To Take An Opposite View[ Go to top ]

    First of all, I like ESB. Then again, I can only get the watered down American version. John, can you sneak me in a few pints?

    Perhaps I am missing the boat, but does writing software to standards or exposing functional via Web Services really require it's three letter acronym? From what I can tell, the ESB Services Container with ESB plug-ins is pretty much the same thing as an app server and EJBs (CMP or BMP, your choice.) The ESB Container simply manages whole business processes in the same way that an app server manages EJBs.

    I am not against the idea, however, I don't think we've come upon something revoluntionary or unique. For instance, how are legacy applications going to integrate into the ESB? They will have to be rewritten so that they conform to the latest standards. Okay, so that's not really a step up since every time something new comes along, it means I have to rewrite my suite of applications.

    Okay, so with new applications being architected from the ground, we should build them with an eye towards integration. Okay, I am good with that. That makes sense, but I was doing that with Web Services and XML long before the ESB concept comes along.

    I guess I am very skeptical about what this is going to add for me. I read over the CIO justifications and I would like to address them:
    1. Increased ability to integrate. Writing software to integrate across the company makes it easier to integrate across the company. True with or without ESB. Also, from my experience, intra-company integration is a myth unless a company advances a lot of time and labor to writing everything to a standard.

    2. Leverage existing IT staff as architects. The amount of knowledge increase, but because it's expanding, it's easy to become knowledgeable. I had to read that twice to make sure I understood since it would seem larger bodies of knowledge make it tougher to know it all, but I think this has to do with availability of knowledge. I don't really see this as much of an advantage, though. No matter how the company chooses to integrate an in-house expert can come from inside the company.

    3. Reduced vendor lock-in. Only if the vendors conform to standards. And if they already are, why fuss about ESB?

    4. Reduced need for wide appserver deployments. To be replaced with ESB Service Container deployments?

    5. More predictable project milestones/deliverables because of pattern-driven development cross projects. Perhaps ESB can be aided by being pattern-driven, but a good developer is going to do this anyway. Right?

    So, I guess I have to say I am skeptical and I am not really understanding the need for this particular buzz acronym.
  4. My understandnig about SOA and ESB is that wrap all model components with service layer. Meaning make all model components (EJBs) as web services. Ofcourse not 10 EJBs into 10 web service components. Package lets say 5 related EJBs into one web service. And these EJBs can overlap into multiple web service.
    This you call it as Service Oriented Architecture. And the container that hosts these services you call it as Enterprise Service Bus. APP Server will be hosting ESB.
    Probably APP Server may be phased out eventually and only ESB may survive.

    Does this make sense at all? Is my understanding correct?
  5. This you call it as Service Oriented Architecture. And the container that hosts these services you call it as Enterprise Service Bus. ... Is my understanding correct?
    ESB = SOA + Pub/Sub
  6. My understandnig about SOA and ESB is that wrap all model components with service layer. Meaning make all model components (EJBs) as web services. Ofcourse not 10 EJBs into 10 web service components. Package lets say 5 related EJBs into one web service. And these EJBs can overlap into multiple web service.This you call it as Service Oriented Architecture. And the container that hosts these services you call it as Enterprise Service Bus. APP Server will be hosting ESB. Probably APP Server may be phased out eventually and only ESB may survive. Does this make sense at all? Is my understanding correct?
    More likely you will have a lot of applications that want to talk to each other. Rather that change these applications you would write an 'adapter' that mediates between the app an the bus. This will convert application specific data into a common event/message format on the bus and vice versa. The esb acts as the wire between your applications (a bit like a distributed IoC... kinda).

    Have a look at Mule (http://www.muleumo.org), it's an open source ESB that I work on. The documentation will give a good overview of ESBs.
  7. My understanding on SOA and ESB[ Go to top ]

    More likely you will have a lot of applications that want to talk to each other. Rather that change these applications you would write an 'adapter' that mediates between the app an the bus.
    That's classic Enterprise Application Integration. It's what webMethods, TIBCo, Mercator, et al, have been doing for years. In this sense ESB is just a new acronym for old stuff. The best thing about ESB is it's reliance on WSDL, which alone provides universal interoperability of implementations. But WSDL has been offered by EAI brokers for years. So what exactly is new about ESB?
    This will convert application specific data into a common event/message format on the bus and vice versa.
    Your mention of a common message format is vague. Do you mean that messages can be handled as generic documents? Or do you mean that the bus only accepts data after the data has been semanticly transformed into a common information model? I once co-invented a tool that did this. Does ESB have an explicit notion of transforming endpoint data to and from a cannonical semantic form? I doubt it. Presumably when say common message format you are alluding to nothing more than ubiquitous SOAP payload. If so, then ESB really is nothing new.
  8. To Take An Opposite View[ Go to top ]

    I tend to think of ESBs an enterprise intergarion pattern for managing communication with disparate systems using a common communication channel.

    An ESB is different from web services in that it's event driven; a call is not directly made to a component on the bus, instead an events are passed around synchronously or asynchronously containing a payload plus other meta info that a component can processed, after which the event is routed somewhere else.

    WS implementations often end up being a bit 'hub and spoke' making them brittle. One of the key technologies to the ESB is JMS, it provides the ideal transport in a distributed environment; persistent queue and durable subscriptions, easy to cluster and load balance, etc.

    I see web services as just one part of the puzzle. It is likely that at least one component in the system will communicate to the bus via web services, but I don't beleive WS are the best/most flexible technology on which to base your enterprise intergation solution.

    I don't think ESBs are anytihng new, people have been solving integration issues using messaging for years, it's just that Gartner gave us an acronym for it :)
  9. To Take An Opposite View[ Go to top ]

    I agree with your definition of ESB. But, web services and ESB are two different things, it is like comparing JMS and ESB. As you pointed out JMS can be used to implement an ESB, the same is true for web services.

    If you have heard of recent specification like WS-Reliable Messaging, WS-Addressing and WS-Eventing, you will note that it is possible to have Web Services only ESB (i.e. cover the same functionality).

    I am not saying we are anywhere close to having WS only ESB, but this is where it is heading. For large corporation where number of systems > 100 and almost every technology (.NET, J2EE, LAMP etc.) is used, this has very good implications.
  10. ESB Flexible[ Go to top ]

    During JavaOne, developers will get bombarded with the buzzwords du jour such as SOA, and ESB (Enterprise Service Bus). Dave Chappell, in his latest book spells out why ESBs will transform how Java devs think about APIs, data sharing and workflow. See what he says, and give your opinion.Excerpt
    ESBs Update Pure Java-Driven IntegrationChappell says that the standards and technologies that will comprise an ESB have matured enough that it’s safe for Java devs to begin thinking about leaving 100% Java-driven integration behind. Traditional Java messaging and middleware approaches, he says, focus on “one point-to-one-point” or “one-to-many” data transfers. These approaches do not easily upgrade to a services-oriented architecture “many-to-many” approach for sharing data, as well as higher-level application and business logic as services.Chappell cites five (5) reasons why CIOs will consider using ESBs, and shift away from traditional integration approaches:
    • Increased ability to integrate inter-departmentally, as well as with business partners using standard interfaces and standard protocols.
    • Leverage existing IT staff as architects: The amount of information available on Java standards such as JAXP, JCA, and JMS, and XML-related standards such as SOAP, WSDL, XSLT, XPATH and XQuery is expanding at an ever-increasing rate. This means that the average IT professional can readily attain the expertise he or she needs to become the in-house integration architect.
    • Reduced vendor lock-in by using a standards-based approach to integrating at the protocol, data and business intelligence layers. ESBs will reduce the appeal of proprietary middleware or integration stacks.
    • Reduced need for wide appserver deployments: Because integration functionality and business process routing is inherent in the ESB fabric,, IT shops can avoid the need to purchase and install J2EE app servers at all target and source integration points throughout their organization, simply for the purpose of providing integration functionality.
    • More predictable project milestones/deliverables: The ESB standards-based integration approach means that devs can reuse common design patterns and successful templates/models from project to project, without expensive consultants and vendor lock-in.
    Read: JavaOne: Unlocking the ESB Secrets
  11. Some thoughts

    Free road way approach to integration than building Railway networks :)

    Advantages

    1. Flexible and Incremental approach for integration with existing appications.
    2. Above approach provides definite timeframes for implementing projects

    Disadvantages

    1. Does not provide message correlation.Need to built with custom ESB services
    2. Does not support stateful workflows(needs a orchestration server to compliment it.)

    Approach to solution

    1.Stress need to be given more on Architecture and Design
    2.Need to avoid point to point integration and approach the solution to provide true event based SOA solution for enterprise.
  12. A standards-based orchestration server (e.g. a BPEL process execution engine) can address a few elements that are essential to make SOA applications work. For example: state management, message correlation and compensating transactions, all encountered in the context of long-running business processes or workflows.

    IMHO, ESB is just one middleware environment that can host a BPEL engine. Other hosting environments can be a portal, app server, even an ERP system, etc. SOA apps come in different flavors and hence a SOA cannot be addressed using a one-size fits all middleware environment, even using one as (presumably) flexible as an ESB.

    $.02,

    Doron.
  13. ...ESB is just one middleware environment that can host a BPEL engine.
    Can BPEL describe publish/subscripe topics?
  14. ESB - Extremely Stupid Buzzword[ Go to top ]

    For me ESB is yet another buzzword marching down the high street. The problem is that a lot of vendors will try to sell "their" ESB or "their" SOA and that by doing so they completely miss the point. What we then get is yet another message bus/ejb container/corba orb/.net infrastructure/<next big thing> with a couple of value added services and probably a web service interface to slow down things a wee bit.

    As of now, I am still waiting to see an "integration" product that really embraces change and diversity rather than actually fighting it. I think the original "ESB" idea was not wrong in recognizing that this at least needs to support asynchronous communication but that is about it.

    While there are products that ease the process more than others,
    Managers need to understand that there will very likely not be a silver bullet ESB platform or container coming their way, but that EJB and SOA must be embraced in the way the company develops and deploys software. If this is just based on platform and product - as it is mostly seen nowadays - the next merger or the next CIO will wipe away all the effort.
  15. ESB - Extremely Stupid Buzzword[ Go to top ]

    +1 Totally agree with your sentiments.

    ESB sounds familiar from the late 90's just around the dot com crash. Architects at that time were falling over themselves to create infrastructure around this 'magic' bus (typically TIBCO). All you needed to do to talk to other applications was put a message on this bus and wait for a reply to come back on this bus. No application needed to ever talk to another application directly again. Even applications that needed synchronous communication would be forced to use this bus. All the same promises were made. Any massaging of the data would be through this wonderful adapters that you needed to write. All your integration problems were gone as soon as you did this.
    Of course all those block diagrams looked great and kept a lot of architects employed for a while. And yes any practical problems in writing these adapters were simply 'implementation issues'.
  16. IMHO, the ESB could be an initial goal towards SOA standardization. An ESB is a concrete concept with a direct implementation on the radar. SOA is far more abstract and generalistic. A consortium of the major software vendors could come to agreement on a standard for an ESB fairly rapidly whereas I fear SOA could take some time.