JavaWorld published an article about using Message Driven Beans to enable concurrent processing in J2EE. The author discusses the alternatives, why you would want to use MDB, and then some design considerations.
Read: Add Concurrent Processing with Message-Driven Beans
This is a common design pattern these days, but should there also be way to ask for Threads from a containers pool?
-
Article: Add Concurrent Processing with Message-Driven Beans (25 messages)
- Posted by: Dion Almaer
- Posted on: July 22 2003 10:16 EDT
Threaded Messages (25)
- Good article by Lofi Dewanto on July 22 2003 11:09 EDT
- Good article by Steve Lewis on July 22 2003 11:19 EDT
- Why the complication by Bill Burke on July 22 2003 11:28 EDT
-
Use JCA 1.5? by Neil Bartlett on July 22 2003 05:53 EDT
-
"in-VM message broker" by Brian Miller on July 22 2003 06:07 EDT
- "in-VM message broker" by Neil Bartlett on July 23 2003 05:00 EDT
-
Use JCA 1.5? by Viraf Bankwalla on July 26 2003 09:42 EDT
- JCA by Billy Newport on July 28 2003 02:26 EDT
-
"in-VM message broker" by Brian Miller on July 22 2003 06:07 EDT
-
Use JCA 1.5? by Neil Bartlett on July 22 2003 05:53 EDT
- Good article by Bill Burke on July 22 2003 11:31 EDT
-
Good article by Brian Miller on July 22 2003 12:56 EDT
- Async by Lofi Dewanto on July 22 2003 01:28 EDT
-
Good article by Mike Spille on July 22 2003 01:39 EDT
-
JMS is needed by Jason Carreira on July 22 2003 03:11 EDT
- JMS is needed by Mike Spille on July 22 2003 03:30 EDT
-
JMS is needed by Jason Carreira on July 22 2003 03:11 EDT
-
Good article by Brian Miller on July 22 2003 12:56 EDT
- careful with mdb by Manish Sharan on July 22 2003 16:08 EDT
- careful with mdb by Nick Minutello on July 22 2003 17:06 EDT
- WebSphere Enterprise Async Beans by Billy Newport on July 23 2003 13:46 EDT
- WebSphere Enterprise Async Beans by Jin Chun on July 24 2003 09:02 EDT
- WebSphere Enterprise Async Beans by Jin Chun on July 24 2003 09:25 EDT
-
WebSphere Enterprise Async Beans by Mike Spille on July 24 2003 10:31 EDT
-
WebSphere Enterprise Async Beans by Jin Chun on July 24 2003 11:04 EDT
-
WebSphere Enterprise Async Beans by Mike Spille on July 24 2003 01:11 EDT
- WebSphere Enterprise Async Beans by Jin Chun on July 24 2003 01:35 EDT
-
WebSphere Enterprise Async Beans by Mike Spille on July 24 2003 01:11 EDT
-
WebSphere Enterprise Async Beans by Jin Chun on July 24 2003 11:04 EDT
- JMS XA by Billy Newport on July 25 2003 02:52 EDT
- WebSphere Enterprise Async Beans by Jin Chun on July 24 2003 09:02 EDT
-
Good article[ Go to top ]
- Posted by: Lofi Dewanto
- Posted on: July 22 2003 11:09 EDT
- in response to Dion Almaer
A very good article. Thanx. That's why we need EJB! (Please do not mix EJB with Entity Bean ;-)). SB, MDB, TimerBean, Connector are a must to produce a scalable system. I cannot imagine, if I have to implement the same stuff with Thread (the 1. solution) ;-)
<quote>
but should there also be way to ask for Threads from a containers pool?
</quote>
No, I don't think this is a good idea. Single threaded system is easier to understand and to implement (KISS at any layers: presentation and business layer). It is easier to extend the SessionBean with asynchronous type of methods. It would be very nice if I can define some methods of my SB with "asynchronous". Or maybe just adding a third type of SB: Stateless, Statefull, Asynchronous?
Lofi.
http://www.openuss.org -
Good article[ Go to top ]
- Posted by: Steve Lewis
- Posted on: July 22 2003 11:19 EDT
- in response to Lofi Dewanto
<quote>
but should there also be way to ask for Threads from a containers pool?
</quote>
If you look at the JCA 1.5 spec, there will be a standard way for a container to manage all its threads, and pass a thread into a connector. This will keep things single-threaded for a J2EE developer, as long as the container knows what it's doing.
Steve -
Why the complication[ Go to top ]
- Posted by: Bill Burke
- Posted on: July 22 2003 11:28 EDT
- in response to Lofi Dewanto
There are 2 reasons that the EJB spec discourages user threads
1) Transaction and Security contexts are usually stored in a ThreadLocal. Unless the EJB container provides an API to create a thread, it has no way to propagate tx and security contexts. This, IMNSHO, is the real reason the spec discourages user threads. At least in JBoss there is no direct consequence of spawning user threads except that you will lose your tx and security associations.
2) The app server may want to have strict control over threads so that it can manage such a "scarce" resource.
Well, that's all I can think of. Personally, I would just create the threads.
Bill -
Use JCA 1.5?[ Go to top ]
- Posted by: Neil Bartlett
- Posted on: July 22 2003 17:53 EDT
- in response to Bill Burke
The JCA 1.5 draft spec offers a way of doing concurrent processing using its Work Management contracts.
I've been looking recently at doing concurrent processing in much the same way as this article describes, however instead of using JMS queues I have implemented a kind of lightweight "in-VM message broker" which is basically a JCA1.5 connector. Along with EJB2.1 message driven beans which allow you to subscribe to different kinds of asynchronous event sources, it's quicker and more flexible than using JMS.
What do people think of this approach? It's works to the letter of the standards but maybe not their spirit! Also support for JCA1.5 and EJB2.1 (ie, J2EE1.4) is pretty scrappy at the moment and my code is only experimental. Is JBoss going to be supporting this stuff soon?
Neil -
"in-VM message broker"[ Go to top ]
- Posted by: Brian Miller
- Posted on: July 22 2003 18:07 EDT
- in response to Neil Bartlett
I've been looking recently at doing concurrent processing in much the same way as this article describes, however instead of using JMS queues I have implemented a kind of lightweight "in-VM message broker" which is basically a JCA1.5 connector.
Maybe an in-process JavaSpace could serve as a local transactional priority queue. Producers set the priority of their messages and dump them into the space arbitrarily. Consumers match and take from the space according to message type and priority. -
"in-VM message broker"[ Go to top ]
- Posted by: Neil Bartlett
- Posted on: July 23 2003 05:00 EDT
- in response to Brian Miller
That's a nice idea. Also see the stuff JBoss are doing with P2P-style JMS using JavaGroups.
The cool thing (for me at least) about JCA1.5 and EJB2.1 is that MDBs aren't tied to JMS any more, and there are all sorts of other asynchronous messaging models that could be explored.
If only I had a container to play with them in! -
Use JCA 1.5?[ Go to top ]
- Posted by: Viraf Bankwalla
- Posted on: July 26 2003 09:42 EDT
- in response to Neil Bartlett
It would be nice if the work manager was generalized and not tied to the JCA specs. The need to perform asynchronous processing is a general problem, and not limited to connectors.
- viraf -
JCA[ Go to top ]
- Posted by: Billy Newport
- Posted on: July 28 2003 14:26 EDT
- in response to Viraf Bankwalla
Thats what we did in WAS-E 5.0, it uses interfaces similar to the JCA 1.5 spec but in a J2EE friendly way so that it can be used in J2EE applications using the familiar programming model and tooling.
Billy -
Good article[ Go to top ]
- Posted by: Bill Burke
- Posted on: July 22 2003 11:31 EDT
- in response to Lofi Dewanto
It would be very nice if I can define some methods of my SB with "asynchronous". Or maybe just adding a third type of SB: Stateless, Statefull, Asynchronous?
>
Yeah, like oneway in corba.
Bill -
Good article[ Go to top ]
- Posted by: Brian Miller
- Posted on: July 22 2003 12:56 EDT
- in response to Bill Burke
It would be very nice if I can define some methods of my SB with "asynchronous".
...
Yeah, like oneway in corba.
Message driven bean is oneway. No need for another bean type. -
Async[ Go to top ]
- Posted by: Lofi Dewanto
- Posted on: July 22 2003 13:28 EDT
- in response to Brian Miller
I think, it would make EJB easier to use if you simply add an Asynchronous Stateless Session Bean (A-SLSB). Every methods, which are called from this A-SLSB will be executed asynchronously. That's it. MDB is a lot more than that, especially in connection with JCA 1.5 (inbound process).
Many of methods you need to implement are using just a simple async. call and not more than that. Let's think about Statistic Component (e.g.: adding statistics into the discussion thread in TSS). Surely you can implement this with MDB (+ SLSB Facade), no doubt. But it would be a lot more easier just to use A-SLSB ;-)
Lofi. -
Good article[ Go to top ]
- Posted by: Mike Spille
- Posted on: July 22 2003 13:39 EDT
- in response to Brian Miller
\Brian Miller\
Message driven bean is oneway. No need for another bean type.
\Brian Miller\
Compared to a fire-and-forget method call, or an asynchronous call, using an MDB with JMS messaging is _alot_ more work, and requires a fair amount of application level boilerplate code to make it work, most particularly at the invocation site. With a fire-and-forget call, a caller would just say:
myBean.execute (...);
With an MDB, you gotta set up JMS factories/connections/sessions/publishers, which is alot more work.
For truly asynch, it's even more work for JMS, but could be trivial if it was built into the architecture. Something like:
myBean.execute (MyContext ctxt, ...);
otherBean.execute (MyContext ctxt, ...);
MyContext.waitForCompletion (timeout);
....where "MyContext" is used to register async calls, and waitForCompletion() waits for all those asynchronous operations to complete. This sort of model would be trivial for an app developer to code (like the above), and can turn alot of slow, serial operations to generate a web page (or whatever) into a series of parallel tasks.
-Mike -
JMS is needed[ Go to top ]
- Posted by: Jason Carreira
- Posted on: July 22 2003 15:11 EDT
- in response to Mike Spille
If you're not using JMS, what happens if your asynch method throws an error? In JMS the message is re-queued and retried. How would this work with an asynch call to a method? -
JMS is needed[ Go to top ]
- Posted by: Mike Spille
- Posted on: July 22 2003 15:30 EDT
- in response to Jason Carreira
Well, first you have to distinguish between fire-and-forget and asynchronous. Fire and forget is _really_ fire and forget. Once the caller does the method invocation, it's completely out of the loop and will not know or care about any errors that happen. Handling problems would need to happen externally to the caller - in the container, or some other mechanism. The exact semantics could be similar to EJB isolation levels - an async bean could be configured for "guaranteed" or "non-guaranteed" in combination with a fire-and-forget mode, similar to JMS persistent and non-persistent messaging, with a monitoring/admin tool to track and resolve issues.
For async with a later rendezvous (like my cntxt.waitForCompletion() example), errors could be reported by the completion call in almost the same way as synchronous calls, except that the error(s) are deferred until waitForCompletion is called.
-Mike -
careful with mdb[ Go to top ]
- Posted by: Manish Sharan
- Posted on: July 22 2003 16:08 EDT
- in response to Dion Almaer
Good article but it could have included some pointers on resource management.
MDB + SLSB is a great way to get things done in the background . But one place the current MDB container implementations fall short is that they don't allows you to explicitly specify the queue priority and concurrent MDB pool sizes explicitly for each MDB class. I have run into problems where the system was emitting several messages but the less important ones were getting processed before more important messages. And sometimes the number of instances of an MDB would be far more that you would want for that particular queue. You could end up having too many threads corresponding to that MDB eating up resources while others wait. This is what I have observed on Linux which is where I deploy.
But I still think MDB+ SLSB is a great way to process back ground jobs.
Regards
-m -
careful with mdb[ Go to top ]
- Posted by: Nick Minutello
- Posted on: July 22 2003 17:06 EDT
- in response to Manish Sharan
|
| one place the current MDB container implementations fall short is that they
| don't allows you to explicitly specify the queue priority and concurrent MDB
| pool sizes explicitly for each MDB class
|
1) Being able to set pool sizes depends entirely on your container. With WLS you can.
1) The priority is a function of the JMS Message - nothing to do with MDB.
NB: The number of beans in a pool is only useful if you want to throttle execution to below the number of worker threads.
-Nick -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Billy Newport
- Posted on: July 23 2003 13:46 EDT
- in response to Dion Almaer
We added support for application threading in WebSphere 5.0 Enterprise, these APIs were baed on the JCA 1.5 APIs but and completely integrated with the J2EE programming model which an RA is not, i.e. an RA doesn't have ejb-refs, resource-refs, etc.
I agree with Mike and the async beans APIs allow Mike to do what he wants to do. Basically, the administrator can create one of more WorkManagers using the admin, these get bound to JNDI as a specified name.
Applications that require a WorkManager specify a resource-ref to a WorkManager. This resource-ref is bound to one of the actual WorkManagers at deployment time.
When the application is running, it can look up a WorkManager like a DataSource, for example:
InitialContext ic = new InitialContext();
WorkManager wm = (WorkManager)ic.lookup("java:comp/env/wm/myWorkManager");
It then creates an object that implements the Work interface which is basically a Runnable with a release method and then does
Work w = new MyWorkObject(...);
WorkItem wi = wm.startWork(w);
More than one can be started of course. If the user wants to wait for N to finish then the following works.
ArrayList l = new ArrayList();
l.add(wi);
l.add(other work items).
if(wm.join(l, wm.JOIN_AND, 30000)
{
// Works completed.
}
else
{
// timed out after 30 second wait.
}
The WorkItem has methods to allow the application to determine the outcome of the Work.
The system is also architected to allow if a Work is serializable to spray the Works over all the thread pools in the cluster, not just the current JVM. This isn't implemented in 5.0 but may be added in a future release.
Billy
(WAS-E Architect, IBM). -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Jin Chun
- Posted on: July 24 2003 09:02 EDT
- in response to Billy Newport
Hi Bill, just a quick question. I've been using the latest fixpacks for was 4.x for generic jms support with xa through the resource binder, and had a quick question: it seems that the fixpack still requires the ibm mq jars (just guessing as class not found exceptions are thrown for the MQXAResource ). I've worked around that. However, there seems to be inconsistent behavour across websphere mq integration and sonicmq, which we are using. For example, in the developer zone, there is a tutorial with samples to perform 2pc operations where the jms objects (receiver, session, connection) are closed within the scope of a was managed tx with websphere mq without a problem, as the JMS wrappers properly wait to sync with the was tm (as in the datasources/jdbc). But using a generic provider (in this case sonicmq), the actual resources are being closed, so the tm is unable to commit the xaresources of the generic provider (obviously since they are gone :-). I realize that support for this is more mature in was 5.x, however, for those of us stuck in the big corporate world still on 4.x, it would be nice for the integration to be more robust IMHO.
Jin -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Jin Chun
- Posted on: July 24 2003 09:25 EDT
- in response to Jin Chun
Actually, sorry I realize this is a little off topic :-) -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Mike Spille
- Posted on: July 24 2003 10:31 EDT
- in response to Jin Chun
\Byung Chun\
But using a generic provider (in this case sonicmq), the actual resources are being closed, so the tm is unable to commit the xaresources of the generic provider (obviously since they are gone :-). I realize that support for this is more mature in was 5.x, however, for those of us stuck in the big corporate world still on 4.x, it would be nice for the integration to be more robust IMHO.
\Byung Chun\
Based your description, this may be addressed in 4.0.6 fixpack 6:
http://www-1.ibm.com/support/docview.wss?uid=swg1PQ69389
The XA part of the JMS spec is pretty fuzzy in general, and in particular doesn't give complete life cycle flows, and IBM's original interpretation could cause problems with some providers similar to what you describe. The newer stuff defers closes on underlying JMS resources until after the transaction is completed, which I think will fix your problem. You should of course check with IBM for details...
-Mike -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Jin Chun
- Posted on: July 24 2003 11:04 EDT
- in response to Mike Spille
Hi Mike, thanks for that.
I think I was too quick on that post, and have verified it, although it would be nice I think to add functionality that delayed closing the actual physical connection until the tx was completed. This is really the only issue that I have outstanding on this subject.
Jin -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Mike Spille
- Posted on: July 24 2003 13:11 EDT
- in response to Jin Chun
"...although it would be nice I think to add functionality that delayed closing the actual physical connection until the tx was completed."
This is available in some eFix or Fixpack, but I haven't the foggiest idea which.
-Mike -
WebSphere Enterprise Async Beans[ Go to top ]
- Posted by: Jin Chun
- Posted on: July 24 2003 13:35 EDT
- in response to Mike Spille
Hi Mike, if you ever find the fixpack I would appreciate (byung dot chun at aya dot yale dot edu). I'm revved to 4.0.6 with the generic binder fixpack, and from what I can tell this is still an issue.
Thanks again, Jin -
JMS XA[ Go to top ]
- Posted by: Billy Newport
- Posted on: July 25 2003 02:52 EDT
- in response to Jin Chun
Well,
I'm the one who added the generic XA stuff to WAS 4.0. For Mike actually :) and as Mike said, we fixed a problem similar to what you describe in an efix which should be in either 4.0.6 or may have missed the 4.0.6 deadline and hence will be in 4.0.7 or may be available from support as an efix. If you're still having trouble then send me an email at bnewport at us dot n0spam dot ibm dot com
Billy -
JMS XA[ Go to top ]
- Posted by: Jin Chun
- Posted on: July 25 2003 09:13 EDT
- in response to Billy Newport
I just sent you a mail on this. Thanks for your help.
Jin