I'm thinking about message driven beans and wondered if anyone had any pearls of wisdom.
I have an MDB which reads from a topic and
causes 2 database updates to occur.
If the DB is down then the message be rolled back onto the topic and should be redelived later.
If we get an application exception (bad data etc) then we should not roll data back onto the topic but simply log the fact that failure occurred.
I want all 3 options
* read from topic
to be atomic so all succeed or all fail if the db is down.
however if I get a an application error from bad data (i.e. no amount of retrying will solve the problem) I want to commit the read but rollback the database updates and log a error.
Is there any way I can use CMT to acheieve this?
I get the MDB to call a session bean (MDB is Required and SB is Required).
This has the problem of if I mark the TX as rollback then the read is rolled back onto the topic too. I cannot separate the 2.
I get the MDB to call a session bean (MDB is Required and SB is RequiresNew).
This has the problem that the commit on the updates is not independent of the read/rollback on JMS if the container crashes between the RequiresNew tx commiting and the read committing then the message would be redelivered when the container was restarted even though the action has already been taken.
I get the MDB to call a session bean (MDB is Required and SB is Required) with the difference that we hold state in the MDB denoting what happened last time we say messages that failed, so when the message is redelivered we know to take the failure action and can commit this. This depends on us being able to hold state in MDBs between requests. Can this be done in a portable way?
I ran into a simmilar situation sometime back. The way I choose to handle it is to use Bean Managed Transactions in the MDB and make the choice to rollback or commit in it and use a Session bean to perform the Persistance for us. In your case maybe two. The Session beans manage their own transactions and simply return an error on processing which instructs the MDB to rollback, the MDB controls the UserTransaction for the Sessions based on exposed contracts in the beans. Hence the whole transaction management is offloaded to the MDB. This renders the Session beans specific for use in the MDB , however it helps solve the problem.
Agreed its not seem to be the most optimal solution to the problem, but have not been able to think of any other alternative's to solve this.
Thanks for your reply.
The trouble with this though is you have a failure point where the
processing has been completed and the user TX committed, before the JMS read has been committed. if the conainer crashes at this point the message will be redelivered and the processing possible repeated.
Or have ai missed something
You might have to store the data in a way that you can tell if you have already inserted it. That way, the retry scenario wouldn't matter. Does the JMSMessageID stay the same between retries? If so you could stick that in the database and put a unique constraint. Then, you could either check for that before making any calls, or try and catch the unique contraint error and ignore it.
A good point.
I missed mentioning the most important thing here I guess :(.
The JMSMessageID can technically be used to validate if it has been processed or not, you'll have to persist this and ofcourse you are talking in terms of a couple of DB opeartions here.
For us we could take this hit cause our workflow engine which was the MDB used its own internal identifier in the Message to check for redundency and hence we worked around this.
However if the container goes down, in the implementation we've the Beans which do the work expose their Transaction Management interface to the MDB. The MDB performs the Commits on the beans and hence it works out ok. However , if the bean commit happens and just before the MDB Message commit occurs the App server crashes then you do have an issue of reprocessing the message. Thats where the persisted lookup of the ID comes in handy.
Hope this explains it a little better.
I am always torn inbetween what to plan for in J2EE. The fact the services provide fail safety at times is also a pain to deal with and you typically land up adding overheads to prevent it from happening. Most of the times I try to keep the design simple enough to account for duplication in the event it occurs rather than always try to prevent it. It just helps in reducing DB roundtrips. Another option I've thrown around in my head a number of times is to keep an inmemory cache of Messages processed and clean it up regularly at entry points in the system. This way if I get a message in the cache I ignore it if the cache does not exist I try to load a temporary snapshot from the DB limited by a timeinterval of say 3 minutes or so. I never implement this mechanism becuase the overhead in development does not make sense. Maybe I am wrong.
Its certainly interesting to see how others acheive this goal. I'd be very suprised if we where the only two folks in the world facing this problem.
I've been through this stuff a few times and there are some points that you may not have hit yet, particularly if you are trying to be portable across JMS providers.
AaronRobinson67 at hotmail dot com
We'll discuss and then I'll post the solution to the thread.
I personally will be very interested in this discussion, Please do post your converstations.
To make your lives a hell of a lot easier, use CMT is you possibly can. One of the benfits MDBs bring is that they are fully fledged enterprise beans and so can benefit from the container services, in particular in this case, container managed transactions. The immediate benefit is that the message receipt and acknowledgement is part of the transaction. So in Bills case, if the DB fails you need do nothing, as the transaction rollback will prevent the message being acknowledged.
As soon as you start trying to use BMT, you end up with a potential hueristic (sorry guys, just had to get that in) problem between the message acknowledgement and the processing. This is what you guys are fighting with.
Bill, when you get your system exception as a result of the database being down, you are going to allow the message to be redelivered aren't you? The idea being that the database will be available at some point. However, it's not quite as simple as that, the message will be repeatedly redelivered until the database is available, which sounds like what you want until the you realise the overheads the repeated reprocessing of this message incurs, particularly when you may have many MDB instances.
At some point you need to give up, or at least save the message for processing once the database is available. As you've probably realised by now there's no coverage in the J2EE specs for message redelivery other than stating when it should and shouldn't happen.
Any JMS server worth it's salt will support poison/dead messages by sending messages that have been redelivered a certain number of times, to an error queue. WebLogic JMS will allow you to assign an error queue per queue, SonicMQ will only allow an error queue per JMS server (that's my understanding).
If you are using this app server support for redelivery for your system exceptions then you may as well use it for your application level errors. Just allow the data errors to be redelivered, they will fail n times and then get placed on the error queue.
For the problem Bill has, the specs state that you should make your processing idempotent. I can only assume that the processing that Bill is doing on the DB side is actually creating and updating the database to warrant the rollback, but can't the data be validated before updating the database? The obvious argument against this is that it may involve extra database access, but you would probably have to do this to be idempotent.
If none of this is really possible then you have to deal with the heuristic problem as before. We created a databse table keyed by JMSMessageID
You cannot use CMT when using third party JMS providers as the failure of a transaction still results in the message being acknowledged.
One way around this is to use BMT as bill described, other approaches make use of the messaging bridge pattern, where the appservers JMS server is used to drive the MDB, and a bridge is used at thr front to transfer messages from the third party provider to the WebLogic JMS queue.
I feel this is the start of a length thread.