It appears that as larger companies start to e-enable their enterprise, they will be trying to build large scale projects integrating legacy systems and new systems. The article describes a general technical architecture pattern for such systems and talks about some products that fit in to this type of system. Click here
to find the paper at ejbinfo.
I read it. I have to say it's a pretty bad article. Or, just a misleading title and a somewhat bad article. A more appropriate title may be 'J2EE and Workflow, state of the union'. Application Servers and traditional EAI vendors and EAI functionality are converging. All sorts of partnerships are being made between App server vendors and EAI vendors. They are a perfect match. His article starts talking about EAI and then he jumps into Workflow seemingly as a *replacement* to EAI. If he believes this he's nuts.
One thing I can agree on is that Application server need better Workflow offerings.
To respond, you're right, I could have picked a better title. The main thrust of the article was to point out that for building large applications, process automation/workflow/EAI products have a crucial role and are possibly more appropriate than J2EE servers in this key enabling role. I'm not saying you don't need J2EE, just that maybe these sorts of components are more useful as a top-level organizing mechanism in larger systems. You need something to tie the various legacy and new systems together, this is where the workflow/broker/messaging (EAI) products come in. Example, your web site uses an application server for the front end but you're using a mainframe based application for your back end business. Brokerages are good examples of this. A suite consisting of a multi-platform message transport, a message broker and a workflow engine can go along way to helping achieve this integration and keep it flexible as new products are added or policies updated. This set of components also normally describes an EAI product.
The distinction you see between what I call workflow and what you call EAI is rapidly converging IMHO. Workflow normally brings insurance claim processing to peoples minds but these products are evolving rapidly to be more flexible. Example, IBM recently added UPES functionality to MQWF to allow it to be more useful in process automation scenarios, going the other way, Vitria added worklists to their product to make it more useful in workflow scenarios.
Most vendors are now selling suites consisting of a message broker, adapters for various message transports and a integrated workflow/process automation engine and calling it EAI (witness Tibco, IBM, Sybase, Vitria).
I'd disagree that workflow and J2EE are converging, I'd rather say that workflow products are <I>leveraging</I> J2EE services (clustering, load balancing, fault tolerance, transactions, security, messaging, development tools, management tools, less platforms to support) to make them more portable and easier/cheaper to develop for the vendors and more open (choose your own JMS transport or database, for example). I think that Java technologies such as the connector framework (when it arrives and gets support) may simplify integration for these vendors with back end systems and allow J2EE based agents to become the 'norm'. I do think that as J2EE servers become more standardized, vendors will use products like jFlow etc to differentiate themselves from the crowd. They will do this by developing them or buying them. Vendors will find they need to offer this functionality to be competitive. If what you mean by convergence is this then we agree.
I feel it was quite an interesting topic, a good one.
But have you left out one more possible solution to
the EAI using the forte's Conductor engine as workflow
engine and using other j2ee api's in other layers .
Like we have a process controller listening to the clients
and inturn communicates to weblogic server ,which co-ordinates with the workflow engine (Forte's conductor).forte'sConductor engine can be integrated with legacy systems using adapters.Please give me your serious opinion
regarding this arhitecture,
Note :Forte is the product name ,Its owned by Sun microsystems.
I never intended it to be a definitive set of solutions possible. It was really just a thinker piece to get people thinking about an alternative way for architecting these systems. I only mentioned IBM, Sybase and BEA because they are the most widely used app servers and they also offered products in this space. It's not quite clear from your post how the your pieces work together so it's difficult to give an opinion, but I would say that if it works for you and you're happy with the solution then it's fine, it meets your expectations.
Well, I didn't notice I was replying to the author. I hope I didn't offend. I know it's difficult writing articles with so many people out there. I apologize if my tone was too aggressive.
I took exception to your article ( primarily the title ) because I'm living it right now. In your title only that is! We are using a J2EE app server both as a building tool but also as a legacy EAI solution. I don't see Workflow as ever being a direct means to tie in Legacy systems as, at least in my case, Workflow doesn't necessarily touch all areas of the business and Workflow is not designed to be real time. IMHO, EAI defines the act of tieing the organization together to form one cohesive integrated enterprise with both real time and asynch. capabilities. Real time is key for B2B transactions. IMHO, the J2EE app server ( or any app server in theory ) is the infrastructure. Workflow and/or rules engines ( like JFlow or Versta ) should be an integrated product *in* the infrastructure. I don't see these tools being accessed outside the infrastructure.
Products like TSIsoft/Mercator and Level8/Geneva. These are classic EAI products that have jumped on board of the J2EE bandwagon. They will integrate with Application servers, provide component level interfaces ( e.g. EJB ) but they are not the infrastructure (even though historically many of them were). I view Workflow the same way. Workflow products , CRM products, rules engines ( business process automation), all of these will integrate into an Application server architecture/infrastructure. They will not supplant them in my view. From your comments and your article, it seems here is where we disagree.
EAI products ( not Workflow but products from vendors like Level8 and TSISoft) are probably the closest category of products right now , IMHO, which will probably become embeded into the application server infrastructure. The J2EE connector architecture speaks to this as do some recent mergers and partnerships. This is where I see convergence. EAI products ( non Workflow products mentioned previously here ) and Application server infrastructures. I do not see Workflow and App. servers converging as not everyone needs Workflow. I see Workflow, along with CRM, along with rules Engines, along with lots of other products as being integrated optional pieces that can be plugged into an application server architecture if an organization happens to need them.
Perhaps some of my problem really stems from your title. To me, its fine to say that Workflow uses EAI, but EAI is not Workflow and to choose between J2EE and EAI doesn't really make a whole lot of sense to me these days. Otherwise, what I got from it, is an understanding how Workflow is progressing, at least for IBM and BEA.
We're coming together here. But.... I feel that it's very difficult to use EAI products that don't have workflow features. As soon as someone needs to approve something then you need to kick down to implement it. This is the reason why Vitria (a previously 'pure' EAI product) has added worklists so that they can model a complete business process (human elements also) something that they couldn't do previously. In the same vein, IBM MQ Workflow is working the other way allowing machine agents to process activities automatically with no human intervention (UPES support). So, this is why I'm saying that whilst previously, you had EAI and workflow products, this distinction is now very blurred, at least at the high end. As far as using workflow to tie in legacy systems, I couldn't disagree more, at least with IBM's product line. IBM MQ Workflow is built on MQ series. By placing a message broker such as MQSI between MQWF and your legacy system, it does a good job of integrating them nicely. Simply declare the legacy system as a UPES and the broker takes care of sending an MQ message in the correct format to the connector bridging the legacy system (any system/language that supports MQ). Now, you can wrap your legacy components with an MQ adapter and plug them in to the business process at an appropriate place. You can modify you legacy system and update the bridge to leave the rest of the system unchanged. You can change the business model and again update the bridge to leave the legacy system unchanged. You can replace the legacy system, update the bridge and leave the business process unchanged. It certainly does the job for me.
Just a quick addendum. You used real-time as a criteria. Products like IBM MQ Workflow can now be used in real time systems. You send a message to it to start a process and MQWF sends messages out to agents that do work and send messages back to MQWF. MQWF then figures out which step is next and then sends another message to the next agent. So, this is real-time. The problem with MQWF is that the engine is CPU heavy and latency between a trigger and the event firing can be a problem depending on how much hardware you've got for the WF engine and your business needs. Anyway, the point is you can use it for this stuff. It all comes around to the fact (IMHO) that at least in MQWF's case, it starting to overlap with EAI domains.
Well, I agreed with you in this way from my prior comments: "To me, its fine to say that Workflow uses EAI, but EAI is not Workflow...". Workflow, touching other legacy or even non-legacy business logic, uses EAI. I was about to write "if your stance is as you mentioned in your last comment, that EAI and Workflow are converging then that should be your article title..". Then , I checked your article to re-read a section and noticed you changed it to that very title!! I already like your article better now.
But I have another problem, which I recall reading in your original article but I didn't get a chance just now to double check that the comment was still there. My problem is when you suggest considering Workflow as your entire EAI solution and also considering Workflow or EAI outside an application server framework. The former may be possible if they converge but the latter I do not agree with. My last comment above was "...and to choose between J2EE and EAI doesn't really make a whole lot of sense to me these days". Are you still suggesting that , even a converged EAI/Workflow infrastructure can live outside a (J2EE) application server infrastructure? I mean, I know they can but why suggest it? I mentioned that EAI solutions historcally have lived outside ( case in point Mercator and Level8 ) but are now converging with J2EE application servers. If EAI and Workflow converge ( which by the way I"m in agreement with you. It makes a lot of sense especially when you add real time capabilities to Workflow ) I still view this as one of many possible pieces that fit into an application server framework.
One addendum, the reason I believe that the application server is always in the picture is because I believe ( as I mentioned before ) another convergence is Application servers and EAI ( as I'm sure you noticed ). So, if EAI and Application servers converge ( something that is now happening ) and Workflow converges with EAI, all we have left is an application server. About the only thing that will separate these things eventually will be cost. Vendors will probably price things as options. For example, you'll pay more for an app server with the EAI/Workflow option...that just for the app server...etc..
I say EAI/Workflow without application server only in the sense that the J2EE server may be embedded and therefore not visible to users of the system. The users shouldn't need to care whether the engine used one or not. Why complicate their lives? I believe that a message broker can provide all the connector technology thats needed between agents and the process engine. One connector implemented with the broker would be a J2EE connector but its not the only one, you'd also have the usual list, APPC, IIOP, JMS, MQ, Tibco, ftp, email, SAP etc. This is all I mean. OF course, you can have an embedded J2EE server in the EAI engine and (a possibly different J2EE server) in agents used by the processes. But, the communications between these take place using a message broker. The EAI engine may expose a synchronous API using HTTP/XML or RMI/IIOP of course for administration or job control or for manipulating worklists, checking out workitems etc.
Your comment: "I say EAI/Workflow without application server only in the sense that the J2EE server may be embedded and therefore not visible to users of the system".
Ok, I think we are saying close to same thing then. This is what Mercator for example did when they bought Novera. Novera was a plain vanilla J2EE app server but is now embedded in Mercator.
See the on-line seminar about e-Advantage architectures, including J2EE and EAI.
Let us not rush into "workflow embedded EAI" as the next big solution for all integration headaches. What value is there in introducing yet another technology, more servers, and added complexity? The real value will be if there are flows that are reusable and need to be made explicit and be modifiable by analysts. And then do we have a business case for workflow.
If workflow is hidden in the guts of an app server then I could care less whether it is a workflow engine or somebody's robust event navigation engine or somekind of graph traversing engine. Whether its is hard coded or follows nice FDL based syntax is less important unless there is a business cas ein making it explict.
Furthermore, the real value will be in end-to-end flows. Integration not just between two apps. but stringing many apps. And when we do that a couple of things happen: these engines need to talk to each other and their perforamance/status need to be visible.
So while vendors are busy adding the workflow icing to their stale cake, we need to see if they are providing open solutions or just a marketing ware deeply embedded in their app server.
I currently work in a telecom support system project where integration of workflow and legacy systems plays an important role. We are using basically the architecture mentioned in the article, that is a workflow engine in combination with message broker. The message broker is good at starting workflow processes (both manual and automatic). The workflow products are good at calling legacy systems and notifying users where manual tasks are needed. However, when we want to modify the workflow in response to outside events we immediately get problems (A typical example is cancellation or modification of running "orders"). Typically we want to make different actions depending on where in the process we are. Modelling all exceptions in the process definitions quickly makes the size of them explode and using APIs to dynamically change the workflow generates a lot of bad encapsulated code with bad maintainability. The vision of making business analysts design the workflow is far from reached...
As it is now I regret I didn't model the state machine for outside actions as an EJB and implemented it completely in java (without third party products) and then split up process in smaller parts triggered on state transitions in this EJB. This would still make it hard for a non-programmer to model the process, but at least it would make the system easier to test and maintain.
Does anyone know a good way to solve these problems? Are we perhaps using the wrong third part products? We are using TIBCOs Active Enterprise suite (InConcert, Integration Manager, Message Broker and various adapters). Through my experience these products have the following problems (I have worked with them a couple of years now):
Terrible process design tools (in case of InConcert).
To fine grained API (in case of InConcert) causing severe performance problems if not a lot of time consuming (development time) workarounds like bypassing the API and accessing database directly.
Bad transaction handling (all products) which is really bad for an enterprise product suite.
Does somebody else have any experience of these problems? Do other products (like IBM’s or BEA’s products) better handle these problems?
We might have an elegant solution to your problem. Please drop me an email at edwink at collaxa dot com.
You have hit the state-of-the-art limit for production grade tools. I understand TIBCO claims to have dynamic workflow feature that is you could in principle modify the flows on the fly. But you will still need to anticipate all possible paths at each stage and then find a smart way to create a workflow as you go along. I believe TIBCO claims this is doable but I have not seen it. MQFlow claims there is no magic and they use lazy evaluation so something is possible here.
But there is no quickfix. To build truly smart, flexible workflow we will have to get back to developing the right kind of process models that support such failures, recovery, and exception right at the design level rather than as patch work. You can do that with use of some good rule-engine. But first you will have to find a flow description language that will support it. Which means for a while we are on our own.
"To build truly smart, flexible workflow we will have to get back to developing the right kind of process models that support such failures, recovery, and exception right at the design level rather than as patch work."
The Collaxa ScenarioBean Abstraction is a process model that supports those advanced features.
You can download it and kick the tires at: