Ask TheServerSide: How many people are using XA transactions?

Home

News: Ask TheServerSide: How many people are using XA transactions?

  1. Gavin King posted on another thread, pondering whether many people have used, and are using, distributed transactions in real applications. This is a good question, so I thought, why not ask the community? So, how many of you have used, are using, or would use if it worked.... XA?

    The Source of the Question

    Gavin King: "It would be very, very interesting to so some actual surveys to find out exactly how many J2EE applications use XA. The impression I get is < 1%. Could be wrong, it's pure speculation."

    Roger Tan: "Not so sure about the number of people actually doing XA in applications, but another question is how many people realize that they are doing XA(or could have relied on XA) behind the scenes i.e. there's lots of questions in the JBoss forum with regards to what the above message means, and that message used to be even more cryptic in earlier versions of JBoss."

    Eric Velazquez: "I'm a former employee of BEA, in the professional services area, and I must say that approx. 80 percent of the projects i worked on were related to system integration, of course WL (Tuxedo on its happy days) in the middle of n legacy / RDBMSs / sync.-async. messaging systems, etc. So for me in that time XA was a need."

    What are your stories? Why did you use XA (if you did)? How did it work out for you?

    NOTE: The Middleware Company's Bruce Martin is actually writing an article on interesting transactional issues, and that will be published soon.

    Threaded Messages (82)

  2. Lots of places where we should but dont[ Go to top ]

    What I typicaly find in the shop I work at is that we are not using XA in places where we should and instead just do two auto-commited insert statements in two different databases that are completly relient on eachother. I cringe everytime i find one of these. The only reason we don't see many errors because of this is the fact that our apps are usualy fairly low volume and our Oracle DB's are very stable in our production envs.

    It also doesn't help that we use OC4J which wants to use Oracle DB Links for XA and our DBAs have outlawed all DB Links in our systems.
  3. Of course[ Go to top ]

    I agree with this 1%, in my enterprise (over 30 java developers) I am the only developer using XA, I need them because I am using a DB an JMS.
    I've been using them for over two years in different platforms and application servers and I have never suffered big problems with them.
  4. EAI and XA[ Go to top ]

    Our project is laying the foundation for a multi-year information architecture implementation by integrating the pivotal applications first. So this involves sending messages and updating a db in one transaction. But since our db is Oracle and our JMS implementation Oracle AQ, this is handled by a single resourcemanager.
    So: No, we don't need XA although our scenario would make one expect otherwise.

    groeten uit Nederland,
    Joost de Vries
  5. RE: EAI and XA[ Go to top ]

    Are you doing this over database links? are all the schemas in the same instance? Are you using Oracle's AQ propogation? Are the Queues and the Tables in the same schema?

    Just curious because we will soon be developing in a similar environment.

    Travis Klotz
  6. Lots of places where we should but dont[ Go to top ]

    Don't know if this is useful for you or not, but the next version of OC4J coordinates recoverable transactions without requiring dblinks.

    Greg
  7. Lots of places where we should but dont[ Go to top ]

    Don't know if this is useful for you or not, but the next version of OC4J coordinates recoverable transactions without requiring dblinks.Greg
    Thats what i'm waiting for. Our head DBA actually gave us a copy of the docs for setting that feature up in the 10.x verions of OC4J and tried to tell us it was a production ready solution. Of course the 10.x versions of OC4J are all still marked as "Developer Preview" and are not going anywhere near my production app servers.
  8. Lots of places where we should but dont[ Go to top ]

    "It also doesn't help that we use OC4J which wants to use Oracle DB Links for XA and our DBAs have outlawed all DB Links in our systems."

    OC4J 10.0.3 does not require DB Links for doing XA. You can access a how-to from OTN at http://www.oracle.com/technology/tech/java/oc4j/1003/how_to.html#tx

    regards
    Debu
  9. Woud use more...[ Go to top ]

    As everyone has mentioned, we have the typical XA/2PC commit scenario -- database save and put a message on a JMS queue. We use it to send out 'transactional-aware' emails.

    However, the current JBoss 3.2 line has not backported the 'Last Resource Gambit' fix yet for our JDBC drivers that does not supprot XA (MySQL).

    I know that 4.0 has it, but would be *very* nice to get it in the 3.2 line.
  10. Woud use more...[ Go to top ]

    Are people using XA without a recoverable transaction manager?
  11. Woud use more...[ Go to top ]

    Greg, is it the transaction manager itself or that one non-XA resource enlisted in the transaction via the last agent optimization that would not support recoverability? Maybe, I didn't read your question right - have still a lot of chapters to work through in your book ;-)

    I have seen quite a few applications though that use this optimization offered by some tx managers.
  12. Woud use more...[ Go to top ]

    It sounded from Andrew's description like he was using XA but not a recoverable TM. I was just wondering if people are doing this in production. Regardless of whether or not the last resource optimization is supported, this combines the worst of both worlds: the cost (including blocking aspects) of 2pc and non-recoverable (hence, no acid guarantees) transactions.

    To answer your other question: the last resource optimization can lead to non-atomic outcomes if a failure occurs during the transfer of control to the one phase resource. It's discussed in depth at the end of chapter 2 in that book you have ;-)

    Greg
  13. Woud use more...[ Go to top ]

    I second this. We use Oracle 8.0.5. The XA support appears beginning in 8i.
  14. Why XA for insert and notify...[ Go to top ]

    <<< As everyone has mentioned, we have the typical XA/2PC commit scenario -- database save and put a message on a JMS queue. We use it to send out 'transactional-aware' emails.
    >>>

    Perhaps, I'm missing something in the above and similar posts. But wouldn't it be easier, not to mention cleaner, to (1) do a transactional database insert and then (2) do a JMS notification of said insert?

    Regards, Mike
  15. Woud use more...[ Go to top ]

    As everyone has mentioned, we have the typical XA/2PC commit scenario -- database save and put a message on a JMS queue. We use it to send out 'transactional-aware' emails.However, the current JBoss 3.2 line has not backported the 'Last Resource Gambit' fix yet for our JDBC drivers that does not supprot XA (MySQL).I know that 4.0 has it, but would be *very* nice to get it in the 3.2 line.
    If you try Arjuna+JBoss in your 3.2 release then you'll get last resource commit optimization.

    Mark.
  16. Real 'enterprise' apps usually need XA[ Go to top ]

    I've spent most of my Java time building enterprise apps: not websites or portals or ecommerce, not small single dept bits, but nasty, big systems that do a lot but usually have a small number of users.

    Most such systems use JMS and if you aren't using XA then you aren't guaranteeing message receipt (actually, you can hack around without XA: when we used iPlanet 6 and XA didn't work we ensured that dbase commits occurred first and made message input idempotent - but that's really a hassle and XA is much better).

    Also the lovely clean architecture where you access only one dbase and it's all yours is lovely in the textbooks, but the messy, unpleasant real world doesn't always work that way. If you're deleting input data from a staging dbase after using it to create objects in a current dbase...

    I think a lot of people don't use XA because the business users don't see it (your functional test will pass just fine), and because it's an implementation detail that only those with hands-on contact with J2EE systems think about it (so an enterprise architect who hasn't actually built J2EE will not think about XA, since they're hung up on the big buzzwords like EJB). These are not good reasons.

    Of course, many people are using java for light-weight applications that don't need XA ('light-weight' is not meant an insult - right tools for right jobs, right?). But the sort of apps that put the EE into J2EE are usually apps where XA is essential.


    Sean
  17. XA for guaranteed delivery[ Go to top ]

    "Most such systems use JMS and if you aren't using XA then you aren't guaranteeing message receipt"

    I thing this is the root of the problem that many people are having. I can only imagine that the term "read only" did not figure in the EJB spec designers vocabulary, because we have the same brute force approach to transactionaility as we have to for entity beans where every access is assumed to update.

    The problem arises because people use a XAT to guarantee delivery when the message is *not* actually performing an update (e.g. when it is used to trigger reference data cache flushes). This is fundamentally the wrong tool for the job and some other means should be used to ensure non-state-affecting message delivery (unfortuntely no standard mechanism is available). State update *notification-only* messages should be issued *after* the actual update transaction is complete - not as an actual part of it IMO.

    Paul.
  18. XA for guaranteed delivery[ Go to top ]

    The problem arises because people use a XAT to guarantee delivery when the message is *not* actually performing an update (e.g. when it is used to trigger reference data cache flushes). This is fundamentally the wrong tool for the job and some other means should be used to ensure non-state-affecting message delivery (unfortuntely no standard mechanism is available). State update *notification-only* messages should be issued *after* the actual update transaction is complete - not as an actual part of it IMO
    Actually when you are sending a message with a quaranty delivery settings you do update, because message gets saved in persistent storage on the JMS server.

    I believe we are talking not about state notification messages but about messages which are part of the business transaction. In this case message should be delivered to the receipent only if a business transaction succeeds over all
  19. XA for guaranteed delivery[ Go to top ]

    Actually when you are sending a message with a quaranty delivery settings you do update, because message gets saved in persistent storage on the JMS server.
    Sorry but you are confusing function with implementation. The jms server persistent queue is an implementation mechanism to achive reliable delivery. Its a means to an end not an end in itself. Thus if you post an update notification message you are not updating the systems overal logical state - you are merely making use of a reliable delivery mechanism.
    I believe we are talking not about state notification messages but about messages which are part of the business transaction. In this case message should be delivered to the receipent only if a business transaction succeeds over all
    Agreed, but I was commenting specifically on the case where the message causes no overall logical state change to the system.

    Paul.
  20. XA for guaranteed delivery[ Go to top ]

    Thus if you post an update notification message you are not updating the systems overal logical state - you are merely making use of a reliable delivery mechanism.
    Just to clarify my own reply: I think part of the confusion is that jms uses the term "queue" which implies a certain implementaion mechanism for reliable delivery. In fact the logical semantics of reliable delivery are equivalent to instant actual delivery and confirmation by the recipient (provided your queue is the only means of communication with the recipient of course). If you think of it this way then it no longer makes sence to think of the delivery mechanism itself being subject to an "update". Whether a logical system update occurs is purely down to the significance of the message to the receiver and nothing to do with the communication mechanism.

    Paul.
  21. My company uses XA transactions all the time, and we can't ignore it. Most systems need to access Adabas data via a non-XA datasource, so we must rely on application server extensions that make this driver the last participant in XA transactions. The other databases involved in the same transactions are DB2 and SQL Server.

    In fact we could not live without XA...
  22. I've had rather terrific experience with XA, when just one distributed transaction being stuck (because of HP JVM bug, by the way) was bringing the whole production system down. With non-XA just one or two users would be stopped.

    I would say in most cases known to me intelligent recovery business scenarios supported by the system's management interface will eliminate a need for XA.
  23. I've had rather terrific experience with XA, when just one distributed transaction being stuck (because of HP JVM bug, by the way) was bringing the whole production system down. With non-XA just one or two users would be stopped. I would say in most cases known to me intelligent recovery business scenarios supported by the system's management interface will eliminate a need for XA.
    This sounds like more a problem with your application than XA. All an XA compliant datasource will do is attempt to guarantee the transaction semantics and if that means that a resource stays unavailable because the coordinator or participant has crashed, then sobeit. What typically happens then is: (i) a failure recovery subsystem of a TPM kicks in and completes the transaction, releasing the resource, (ii) a sys. admin manually releases the resource (could generate heuristics though), or (iii) the backend db is configured so that if an XA transaction doesn't complete in a specific period of time it can automatically rollback the transaction (again, heuristics might result).

    It's definitely wrong to imply that XA is the cause of your failures though. You don't say what your implementation was, so I can't comment on that, but the actual X/Open XA specification is pretty clear on what it does and why. Obviously things might have been lost in translation from paper to code, but if XA wasn't up to the task it wouldn't have lasted as long as it has.

    And it's easy to say "intelligent recovery business scenarios supported by the system's management interface will eliminate a need for XA", but it's a lot harder to do ;-)

    Cheers,

    Mark.
  24. When you are connecting to multiple databases (or multiple schema in one database) you need XA transactions. Also, if you just use one database connection in combination with JMS, you also need XA. Unfortunately, there is no open-source alternative to the likes of Oracle when it comes to *true* XA support. Don't say Postgres supports XA, because it doesn't.
  25. Don't say Postgres supports XA, because it doesn't.
    Which article then goes on to say:
    The only free software solution I've found that supports XA is Firebird.
    I also believe Apache Derby supports XA, useful for embedded applications that access central servers.
  26. Don't say Postgres supports XA, because it doesn't.
    Which article then goes on to say:
    The only free software solution I've found that supports XA is Firebird.
    If you would have cared to read just a tad further, you would have read:
    However, the JDBC driver for JayBird doesn't support XA through JDBC.
    The point is JayBird does not offer an implementation of XADataSource. Instead it only supports XA through JCA. As such, it does not look like an alternative to Oracle.

    Which opensource databases would be a viable alternative to Oracle when it comes to XA support?
  27. The OSS debutant: Ingress. i guess, don't really know but as a former fully featured commercial DB it should support XA, anyone knows it??
  28. The OSS debutant: Ingress. i guess, don't really know but as a former fully featured commercial DB it should support XA, anyone knows it??
    My hopes are also up for Ingress although I have yet to find testimonials on its XA support. Any details on XA support in Ingress are welcome :-)
  29. The OSS debutant: Ingress. i guess, don't really know but as a former fully featured commercial DB it should support XA, anyone knows it??
    My hopes are also up for Ingress although I have yet to find testimonials on its XA support. Any details on XA support in Ingress are welcome :-)
    It looks very interesting, as I understant it is possible to use Ingres as gateway with XA for free.
    (link many databases by different vendors and not nesecary ralational to single Ingres instance). It must be the ideal way, it is the real "transparence" from application point of view and it is not limited to JAVA, you can "map" any stuff to this kind database.

    See "Distributed Transaction Processing User Guide"
  30. I would have thought most applications using XA would be also be using an app-server that supported JCA, making Firebird an option. Depends on your need though really.
  31. I would have thought most applications using XA would be also be using an app-server that supported JCA, making Firebird an option. Depends on your need though really.
    How do you configure a CMP entity bean to retrieve a database connection from JCA? How do you pass a datasource to a Hibernate SessionFactory when your database connectivity resides in JCA? Perhaps this can be done but I don't know how.
  32. We are using XA tx with Oracle, Websphere MQ on Weblogic app server and we are running into Tx timeouts and resource issues. Even if we set Tx timeoout to 6 minutes which is way too high.

    Do you have any suggestions?
  33. We are using XA tx with Oracle, Websphere MQ on Weblogic app server and we are running into Tx timeouts and resource issues. Even if we set Tx timeoout to 6 minutes which is way too high.Do you have any suggestions?
    That's not enough data to even attempt a suggestion. Does it happen at random? Do you get a stack trace? Is the timeout happening during the execution of the code, or during commit processing?

    In any case, search for similar problems on dejanews (now google groups.) It usually helps.

    Guglielmo
  34. EJB and LDAP transaction[ Go to top ]

    In a recent project we needed to update our own Oracle DB within our WebLogic 8.1 server, a remote WebLogic server and Sun's LDAP Directory Server. The latter is non-transactional, and even though it is the last participant, the heuristics are flaky. Considering the widespread use of LDAP, I was surprised I did not come across anyone else using this usace scenario. Anyone out there seen this scenario?

    -- andre
  35. I work on a few different high-volume oltp systems using WebLogic (WL JMS also) and Oracle. I use XA to keep the JMS and Oracle transactions consistent.

    Since the backing store for the JMS server is the same Oracle server that also stores the application data, I _could_ eliminate XA if WebLogic supported the one-phase optimization in this case. The reason is that Oracle would then use a single branch id, for both the jms and data branches of the transaction. I.e. if WLS crashes, Oracle could decide by itself whether it wants to commit or roll back, without loss of consistency.

    So I am using it, but other people with the same architecture may not need to, depending on how the XAResources are implemented.
  36. We do and we don't[ Go to top ]

    Historically I have found that the use of XA depends on a few things, a) Do you need to span transactional boundaries of two distinct transactional systems, b) Are you in the context of an application server, c) is the implementation of remote XAResources sufficiant for your needs.

    In terms of A and B; Its fine if you're living inside of a J2EE container, its all handled for you, its also fine if you're living entirely outside a J2EE container. There are several standalone transaction managers, even good ones that do proper branch recovery. The interesting things occur when you're in the grey area between beans and standalone apps. Jboss services (things deployed in SARs) are an example of this. Its fine if you configure your apps to lookup only XA resources inside the container, but if you try to enlist newly created XAResources in the implicit JBoss transaction you'll have headaches.

    with regards to point C) We don't use XA for certain things because our Message Queueing server performance degrades immensly if you use XA, thats because in their implementation the Queue itself has to be backed by a database, whether you are accessing it in an XA manner or not. Regular JMS Transactions can occur on file based queues and are lightning fast. Also if you need to pop a message from one queue and put 5 messages onto other queues in the scope of one transaction there is no need to enlist a distributed transaction manager.

    I wonder how many people use XA transactions out there and don't really know it or care about the significance. After all, if you configure datasources/JCA adaptors in modern appservers and you're using good infrastructure it "Just Happens". I also wonder how many people get XA enlisted resources when they don't have to and if there applications would perform better if they weren't unnesecairly using XA.

    As a side note, solid XA transactions and recovery of transaction managers knowledge of XIDs and branches have saved my bacon more than once, to me, I couldn't live without them.
  37. Database Historisation[ Go to top ]

    I've been involved in a project in a financial environment that needed historisation of most of the data stored in the database (Oracle 9i btw) and it was actually making sense. The DBA had created triggers on all "history-aware" tables that were basically copying the data sets to another database adding timestamps anytime an insert/update/delete was issued. As more than one transactional resource was involved, XA connections were needed.
  38. Timing Problems with 2PC[ Go to top ]

    I had a very tricky 2PC - problem to solve: In one transaction, I stored some userdata in the DB and published a JMS - Message to inform some backend - systems that userdata had changed. After that, the transaction was commited. Because of the already existing customer IT - infrastructure and customer requirements, two DBs and therefore a distributed transaction had to be used: one for the userdata, one for the persistent store for JMS.

    Here is what happened: After success of Phase 1 of the 2PC, both transactions (userdata and JMS - Message) were committed. Sometimes, the commit on the userdata - DB took quite a bit longer than the commit of the JMS - Message. The message was correctly delivered to a message consumer (some backend - system). The backend - system now tried to read the modified userdata. At that point, the commit on the userdata - DB was still in progress, so the modified data could not be seen by the backend system. After a few more ms, after the commit on the userdata - DB had completed, you could read the modified data. This issue really drove me nuts and it took quite a while to figure out why this problem exactly happened and what to do about it. We ended up with sending timestamps in the JMS - message so the backend system would know if it was reading the old or the modified data, and then maybe retry. Obviously, not a very good solution.

    I read some pretty detailled stuff about XA and 2PC, and IMHO there is no easy solution for such timing problems, while they are horrible to debug. For that (and some other) reason, I try to avoid XA transactions where possible.

    Jochen
  39. Timing Problems with 2PC[ Go to top ]

    I use XA transactions all the time with Oracle 8.1.7.4 and Softwired JMS 4.5.x

    I was having the same exact issues of timing *before* using XA.
    All timing issues disappeared (regardless of the load) as soon as we went XA.

    About our app...
    We built an even-driven system (JMS) on top of JBoss 3.2.x
    I love the DLQ concept even though I the Jboss impl could be better (had to patch a couple of areas to avoid msg loss).
    With XA and the DLQ , we can asynchronously and "transactionally" deal with our server events and easily retry them - event is either processed fully or rolled back and retried N times. If retried too many times, the message is moved to the DLQ.
    As soon as backend error is fixed the message can be moved back to processing queue.
    Again this is only possible because of the strong transactional boundaries in place.

    I would love to upgrade to a more robust transaction manager (open source need to say) that could automatically recover XA transactions with a nice UI if possible to see all the failed yet recoverable transactions (journal).
    I looked at this a while back and couldn't fine any.
    If you guys know of any please let me know...

    In that context (failure & recovery of transactions), the only issue we continue to have is when Oracle starts "going funny" (with its not so nice XA implementation) and generates XA exceptions.
    At that point we usually start getting in-doubt transactions that lock rows/tables in the database (in the JMS console the messages are also stuck).
    We have programmatically detected those in our data layer so we can force rollback those in-doubt transactions.
    For JMS, we have to go to the console and rollback by hand :-(
    For some reason the message continues to be associated with the XA transaction and the message can't even pass through container.
    We're waiting for a fix to come...
  40. Timing Problems with 2PC[ Go to top ]

    I had a very tricky 2PC - problem to solve: In one transaction, I stored some userdata in the DB and published a JMS - Message to inform some backend - systems that userdata had changed. After that, the transaction was commited. Because of the already existing customer IT - infrastructure and customer requirements, two DBs and therefore a distributed transaction had to be used: one for the userdata, one for the persistent store for JMS. Here is what happened: After success of Phase 1 of the 2PC, both transactions (userdata and JMS - Message) were committed. Sometimes, the commit on the userdata - DB took quite a bit longer than the commit of the JMS - Message. The message was correctly delivered to a message consumer (some backend - system). The backend - system now tried to read the modified userdata. At that point, the commit on the userdata - DB was still in progress, so the modified data could not be seen by the backend system.
    Everybody has this problem. We just put a delay on the queue, to give the database a chance to catch up. It's not a perfect solution conceptually, but it works okay in practice.
  41. Timing Problems with 2PC[ Go to top ]

    Yes, we have run into this race condition as well, typically in a JMS message enqueue and DB update/insert scenario from the same transaction. After a JMS message has been enqueued and DB updates/inserts done as part of the same transaction, we would often see the message consumer MDB dequeing the message in a subsequent transaction and expecting to read updated/newly inserted data and failing because the updates/inserts are still being committed. It was easier to get around this by putting in a rerty mechanism in the MDB looking for a newly inserted DB record but gets much harder to do in the DB update scenario as we wouldn't always know what updates the application is dependent on for correct behavior.

    Sophisticated tx co-ordinators send out the commit messages in parallel during the 2nd phase and that only exacerbates the problem. Unless the application has a way to explicitly instruct the co-ordinator to impose an ordering on the resource commits I don't see how this problem can be solved in a genric way.

    I am hoping Mark Little, with his years of Transaction Processing/DTP experience will weigh in on this - btw, Mark (and couple of other folks) has written a wonderful book on Java Transaction Processing with details on JTA, XA, DTP that you will be hard pressed to find in the Java/J2EE literature. Can't recommend it enough.
  42. Timing Problems with 2PC[ Go to top ]

    This is more of an isolation issue than an XA/2PC one - it is just that it is more likely with XA as the commit is asynchronous across the resources.

    I am guessing that the message receiver is reading the database with a isolation level lower than serializable. This allows the receiver to see committed data, which, as the database has only prepared and not committed, is the old version. If you raise the isolation level to Serializable, the read would block until the database completed the previous commit and you would see the updated data.
  43. Timing Problems with 2PC[ Go to top ]

    For updates, raising the serialization level would work in most of the situations but for inserts it probably would not as the message consumer is looking for a specific piece of data and that has not 'shown' up yet and there is nothing to wait on. We had no choice but to build in a retry mechanism.

    Even for updates and reads with serializable iso level we ran into an exotic situation involving BLOB updates in Oracle where the serializable isolation level was not honored due to the way BLOBs are stored out-of-line in Oracle. We had to resort to 'touching' extra non-BLOB columns to gurantee correct serialization behavior and avoid the race condition.

    And then there is the issue of the performance hit for the serializable iso level but that always boils down to a trade-off between consistency and performance. It's just that, as you say, the semantics of the 2PC can make this a fairly common occurence.
  44. Timing Problems with 2PC[ Go to top ]

    Sophisticated tx co-ordinators send out the commit messages in parallel during the 2nd phase and that only exacerbates the problem. Unless the application has a way to explicitly instruct the co-ordinator to impose an ordering on the resource commits I don't see how this problem can be solved in a genric way.
    There are several "legacy" (as in "been around for quite a while") TPMs that do allow you to impose ordering on the participants in the transaction intentions list. Some of the standards for transactions even support this kind of thing too (e.g., the OMG Activity Service allows you to specify a priority for each participant when it's enlisted with the transaction).

    I'd hope that any transaction system that supports parallel commit (and prepare) would allow you to turn that off in some way. It's useful a lot of the time, but as you point out can lead to nightmares if used in the wrong places.
    I am hoping Mark Little, with his years of Transaction Processing/DTP experience will weigh in on this - btw, Mark (and couple of other folks) has written a wonderful book on Java Transaction Processing with details on JTA, XA, DTP that you will be hard pressed to find in the Java/J2EE literature. Can't recommend it enough.
    Hey, glad you like it. Greg, Jon and I tried to write it from a practical standpoint, so I hope we succeeded.

    Cheers,

    Mark.
  45. I saw the same problem with TopLink. TopLink was taking dozen of seconds to merge its cache after a long transaction. It's maybe not part of the 2PC protocol but I believe the database or TopLink should always guaranty a fast Phase 2 when they said phase 1 is a success, at least they should read lock the data during their phase 2.

    Because a message (JMS) by definition is asynchronous and so you don’t care too much about how fast it will be processed, you can use different strategy that guaranty it will not be consumed for a certain time, like 2 minutes.

    Another approach is to send the JMS message inside the afterCompletion of the SessionSynchonization( to be implemented by a Statefull bean ) that are part of the transaction. In this scenario, afterCompletion method is like a hook to do something just after the Phase 2.
  46. We've needed XA on more than half of the projects I've been on. We've needed it on all of the more recent ones. At my current company it's not unusual for an application to communicate with 5 or more backend systems/services which may include databases or CORBA, EJB, or JMS services, etc. I've seen many developers ignore the need because they don't understand transactions. The data just gets cleaned up manually.
  47. I agree very few application developers knowingly use XA. However support of third-party resources such as third-party JMS providers in a transactional context require XA.

    I'm looking for answers for a similar question. Does anyone use transaction interoperability between application servers ?

    Please read Transaction interoperability between J2EE application servers, Is this a real need?

    regards
    Debu Panda
    http://radio.weblogs.com/0135826/
  48. I agree very few application developers knowingly use XA. However support of third-party resources such as third-party JMS providers in a transactional context require XA. I'm looking for answers for a similar question. Does anyone use transaction interoperability between application servers ?Please read Transaction interoperability between J2EE application servers, Is this a real need? regardsDebu Pandahttp://radio.weblogs.com/0135826/
    One scenario I've seen where transaction interoperability is important is for staged migrations from one application server to another. I've also been told that people use OTS like capabilities for disaster recovery protection, but I've not seen architectural details describing how they were achieving this. The need exists, but the main problem is that the industry doesn't have a good interoperability story yet, and mostly has to rely on tighter than desireable integration techniques. Web services may change this: both WS-CAF (WS-TXM in particular) and WS-AtomicTransaction are aimed at providing interoperability protocols that can be more readily leveraged to solve this problem.

    By the way, Mark Little on this thread is co-author of a good book that looks at the whole range of transaction technologies in the Java and Web services environments.

    http://www.amazon.com/exec/obidos/ASIN/013035290X/qid=1094144551/sr=ka-1/ref=pd_ka_1/102-5072102-8616163

    I've read it a few times ;-)

    At Bluestone our old CTO used to argue that transactions were never needed. I started out from that perspective as well but came to a very different view from exposure to customer problems. Bottom line is that, as always, you need a capable architect to make appropriate technology decisions for each solution. I think this thread got kicked off with some fairly silly statements.

    Greg
  49. I'm currently reading Expert One-on-One J2EE Development without EJB by Rod Johnson and I am amaze how he never talk about JMS when he tries to make the same argument as Gavin. Now we have Gavin King adding to the confusion. I worked on many projects based on J2EE and all of them where using database and JMS. All the use cases involving JMS were better to be in the same transaction with the database update.
  50. I don't argue that distributed transactions are bad things: merely that you shouldn't use them unless you really need them.
  51. I don't argue that distributed transactions are bad things: merely that you shouldn't use them unless you really need them.
    Off-topic - can't help mentioning that in the currently popular culture of EJB-bashing (not without reasons, I must add) the value proposition of the MDBs often gets overlooked. I can't imagine designing workflow systems or financial applications without them and it's a pity MDBs get a bad wrap by association even though they are so simple to develop and can be so useful.
  52. I didn't know MDBs had a bad rap, but if so that's too bad, more of ignorance I guess, because as you say they are very useful for just this sort of thing. There seems to be quite a bit of disinformation out there (I'm surprised by whether anyone would wonder if people use XA), you really have to take all what's written in the 'zines and books with a grain of salt, as it might apply to a certain system, but not your own area, i.e. workflow/financial etc. But I suppose for alot of the software out there, written for dot-coms or whatever, perhaps there are other requirements, such as speed (or speedily-built), who cares if a customer order is lost every now and then seems to be the attitude? Also, part of the problem is that MDBs were only recently supported (EJB 2.0, WebSphere 5.0), and people had to come up with various workarounds.
  53. JMS[ Go to top ]

    Actually, I've not forgotten JMS, in fact I'm thinking specifically about the JMS+database case, since it is the most common case for XA.

    (And taking me to task for "adding confusion" is a bit unfair, since I made the suggestion, with much qualification, in a *comment thread*.)

    Regarding JMS.

    * On the message receiver side, I almost *never* need 2PC, as long as I can build my application to handle the (very rare) case of the same message being delivered twice, which is usually incredibly trivial! Indeed, it is quite likely that the app will perform better *without* XA here.

      1. peek the message
      2. perform the db work (checking for dupes)
      3. commit the message

    * On the message sender side, things are a bit more difficult - but again, it is not at all hard to build recovery into the application, if you again assume that the receiver is built to anticipate duplicate delivery. Performance will not necessarily be be better than using 2PC, but might be in certain cases.

    If people don't know the tricks I'm describing here, perhaps I'd better describe them in a blog. But they all depend upon the fact that true atomicity is not actually a requirement, assuming you can handle duplicate delivery. (I've met quite a few people who didn't know these tricks, by the way, and I've never read about this stuff in any book, so I'm not sure exactly how widespread they are.)

    Anyway, regarding the original question, I'm probably wrong - hey, it might be as high as 10% ;-)
  54. JMS[ Go to top ]

    Isn't this almost tautological? If you don't need "true atomicity" of course you probably don't need acid transactions...

    Greg
  55. JMS[ Go to top ]

    The point is that your course-grained "business transaction" can still be truly atomic, without requiring that the individual interactions are themselves atomic.

    No, its not a tautology, I'm describing how you can achieve what you might think you need XA for, without needing XA.
  56. JMS[ Go to top ]

    The point is that your course-grained "business transaction" can still be truly atomic, without requiring that the individual interactions are themselves atomic.No, its not a tautology, I'm describing how you can achieve what you might think you need XA for, without needing XA.
    It is possible to live without XA and with JMS + RDBMS, but "TxId", "status", "xtimestamp", "MsgId" ... fields in database are very noisy.
    BTW doe's somebody trust XA implementations on JAVA app servers ?
  57. JMS[ Go to top ]

    On the message sender side, things are a bit more difficult - but again, it is not at all hard to build recovery into the application, if you again assume that the receiver is built to anticipate duplicate delivery. Performance will not necessarily be be better than using 2PC, but might be in certain cases.If people don't know the tricks I'm describing here, perhaps I'd better describe them in a blog.
    I am using JMS and DB and require XA on the producer side, I am interested in hearing the trick you just described!

    Vincent
  58. JMS[ Go to top ]

    On the message sender side, things are a bit more difficult - but again, it is not at all hard to build recovery into the application, if you again assume that the receiver is built to anticipate duplicate delivery. Performance will not necessarily be be better than using 2PC, but might be in certain cases.If people don't know the tricks I'm describing here, perhaps I'd better describe them in a blog.
    I am using JMS and DB and require XA on the producer side, I am interested in hearing the trick you just described!
    Same for me, I'm curious to learn how these trick work :-)
  59. JMS[ Go to top ]

    I quite agree Gavin on this case. There are 2 distinct cases:

     * multi DB / multi system transaction
    Except for very critical apps, it is usually overkill to use 2 PC tx. If one of the systems fails, it *is* a true nightmare to recover and the critical business is down (although still consistent ;-) ). I'm a 100% pro of application recovery management simply because of the 80%-20% rule (ie in this case 99%-1%)

     * JMS / DB
    This is quite common to ask for consistency and usually I'm not strong enough to fight against the EAI guys and the academic point of view. But again I agree Gavin and I rather do a manual consistency check (even with a evil batch) than have to launch XA drivers etc.

     * Real life experience
    I'm not sure so many people use XA stuffs in production. I had so many problems to make it work on WL 7 / Oracle DB 9i.
  60. JMS[ Go to top ]

    * multi DB / multi system transaction
    Except for very critical apps, it is usually overkill to use 2 PC tx. If one of the systems fails, it *is* a true nightmare to recover and the critical business is down (although still consistent ;-) ). I'm a 100% pro of application recovery management simply because of the 80%-20% rule (ie in this case 99%-1%)
    There is a _huge_ difference between using 2PC and using a recoverable XA protocol. Using 2PC (IMHO and limited experience) is _almost_ free. Using XA is where the expense comes in. When I read a comment such as the above, it really makes me worry about the conclusions.

    To me, the multiple database scenario is at least as important as the JMS + DB scenario, because consistency will get lost without a 2PC protocol, and the potential for a consistency loss is there without XA (e.g. if an app server crashes during 2PC.)
    * JMS / DB
    This is quite common to ask for consistency and usually I'm not strong enough to fight against the EAI guys and the academic point of view. But again I agree Gavin and I rather do a manual consistency check (even with a evil batch) than have to launch XA drivers etc.
    Academic? It's hardly academic. When you have two transactional resources that are in the same transaction, is it worth it to you to have them stay in sync?

    (BTW Gavin, it sounds like there are plenty of votes for you to explain your JMS send-after-commit approach. If you do write it up, please post a link here.)
    * Real life experience
    I'm not sure so many people use XA stuffs in production. I had so many problems to make it work on WL 7 / Oracle DB 9i.
    Yes, here you are right. The first time getting XA stuff to work among combinations of certain products can be a frustrating experience. However, that's why companies have experts who are used to this kind of thing.

    The frustrating thing is that a lot of companies (in my experience) using things like XA don't bother to really test to see if their configurations _work_ .. or have good plans in place to handle when things actually do go wrong. This is what really makes me go off .. when companies invest millions into software that handles all these contingencies, then don't follow through to make sure that the result is achieved.

    Anyhow, it is safe to say that XA is not a "shiny bauble" ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  61. JMS[ Go to top ]

    It sounds like there are plenty of votes for you to explain your JMS send-after-commit approach. If you do write it up, please post a link here.
    Probably I do not understand this problem, but it doe's not look as a problem .

    MyApp -> write -> commit -> send message;

    App2-> recive message -> read -> commit-> acknowledge;

    How XA helps for this notification ? I see no problems without it for this use case.

    Do you need to rollback if "send message" fails ?

    MyApp -> write with message id -> send message -> commit;
    App2-> recive message -> retry read while message id found -> commit-> acknowledge;

    You need to retry, because you can recieve message befor commit in this case and limit iteration count, because "commit" can fail after "send message".

    Is it better to buy some software for this use case ? Doe's it work better in practice ?
  62. JMS[ Go to top ]

    BTW if you have money to buy good database like postgresql then you do not need JMS for this use case, it solves this problem without workarounds (It has native messaging).
  63. JMS[ Go to top ]

    MyApp -> write -> commit -> send message;
    And what happens if:

    1. The server dies before it sends the message
    2. The MOM is down when the server tries to send the message

    Either way, the write was done but the message doesn't get sent. That defeats the whole "all or nothing" aspect of the transaction. What if the "write" was the "take money from one account" and the JMS message was the "post it to the other account." All of a sudden, you're in big trouble.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  64. doing without XA[ Go to top ]

    MyApp -> write -> commit -> send message;
    And what happens if:1. The server dies before it sends the message2. The MOM is down when the server tries to send the messageEither way, the write was done but the message doesn't get sent. That defeats the whole "all or nothing" aspect of the transaction. What if the "write" was the "take money from one account" and the JMS message was the "post it to the other account." All of a sudden, you're in big trouble.
    Well, its is not quite right. Correct is:

    (1) write + commit
    (2) send message
    (3) write again + commit

    Well, actually, (3) is optional if you are expecting an acknowledgement from the server after the transaction is successful, which is an important common case, but let's not get into that now.

    Clearly, you need a recovery phase in case you fall over after (1), which is a bit of a pain to have to implement yourself, but probably not much more difficult than getting XA to work properly. ;-) It's a really simple batch process, a monkey can write it. It can run regularly, or at server re-start.

    Now with XA, I guess you don't usually get recovery until the particular server that failed restarts (ick), whereas here you get it on whatever schedule *you* define.

    (If you fall over between (2) and (3), no problem, the message just gets delivered twice, and you are anticipating that possibility at the other end.)

    This most certainly does not require much noisy stuff in the database as other posters suggest. All it requires is a "sent" flag on the record you inserted in (1).

    Am I saying this is somehow better than XA? No, of course not. All I'm saying is it's not much worse and if, for whatever reason, you don't have or want XA, you can still achieve a truly atomic distributed "application transaction".
    True two-phase commit is much harder to implement, since it enforces the semantic that messages are delivered once and only once, which is simply not necessary in most cases.

    A similar approach can be used with two databases, by the way. Not sure if I would really advocate that however.
  65. doing without XA[ Go to top ]

    We can talk all day about "what if I do this, etc, etc" but isn't one of the value proposition of an app server to insulate a regular monkey from knowing all these details that could have been done with XA. Otherwise all these people could also be writing object pooling, remoting code, ORM, etc, etc. Heck, they might not even need the database recovery log, they could create a better one and see if they have logged successfuly, before calling commit on the database.

    I am not that familiar with Weblogic, but do Weblogic developers also need to worry about all this ?
  66. doing without XA[ Go to top ]

    I am not very familiar with applications servers too, but I found I need to know any tool including tool implementation if I use it. All tools promise to help monkeys, but I am sure monkey can not use Hibernate too. I like open code for this reason, as more I know about implementation as better.
  67. JMS[ Go to top ]

    MyApp -> write -> commit -> send message;
    And what happens if:1. The server dies before it sends the message2. The MOM is down when the server tries to send the messageEither way, the write was done but the message doesn't get sent. That defeats the whole "all or nothing" aspect of the transaction. What if the "write" was the "take money from one account" and the JMS message was the "post it to the other account." All of a sudden, you're in big trouble.Peace,Cameron PurdyTangosol, Inc.Coherence: Shared Memories for J2EE Clusters
    Yes, there are more "complex" use cases and it ends with the same stuff as XA protocol, I maintain "state" and "xid" fields in this case (it is single record in most cases and "xid" can be the same as "order id"). Sometimes transaction fail and fake records go to "history" tables (garbage collection), recovery is trivial too, read all records on startup with state "prepared" and retry. If this stuff does not work then people do it manualy,it is not very common, but it happens too (I am sure it happens with XA too, see problems on this forum).
    XA is fine, but in "complex" cases 3/4 systems are involved and one of them or all do not support XA protocol. It is not because use case is "complex", it because people design it as "complex" for some political reasons.Not everybody trust (or do not want to learn) JAVA application server for this kind of distributed systems too.
  68. JMS[ Go to top ]

    BTW, XA is more common in homepages with single database than in "complex" systems, is not it ? :)
  69. JMS[ Go to top ]

    Academic? It's hardly academic. When you have two transactional resources that are in the same transaction, is it worth it to you to have them stay in sync?
    Academic in the solution, using XA resources is the easiest way. Building a *business* recovery mecanism is much more appropriate IMO for 90% of apps (those non critical ones.
    * Real life experienceI'm not sure so many people use XA stuffs in production. I had so many problems to make it work on WL 7 / Oracle DB 9i.
    Yes, here you are right. The first time getting XA stuff to work among combinations of certain products can be a frustrating experience. However, that's why companies have experts who are used to this kind of thing.
    Well, if I remember well, it was a bug in the XA Oracle driver :-( I was just saying that XA isn't that much used for real if such bugs can still be found.
  70. Asynchronous systems considered harmful[ Go to top ]

    Actually, I've not forgotten JMS, in fact I'm thinking specifically about the JMS+database case,
    <br><br>
    It is funny how there is almost no overlap between different disciplines. In the 1970's, asynchronous digital logic had a brief upswing - until people started to discover all the nasty race conditions that could occur and all the plumbing needed to escape them. Today, asynchronous designs are only seen where there is a large upside to using them. The ARM7TDMI embedded core is, perhaps, the best-known successful example of such a design, where asynchronous elements are used to lower power consumption.
    <br><br>
    Enter the software world. Lately, JMS and other asynchronous ways of writing code have gotten a lot of interest. IBM and others tout MQSeries as the way to go. Integration is a very hot topic when JMS is mentioned, apparently.
    <br><br>
    Nobody has mentioned race conditions until now. I get a warm, fuzzy feeling when I start seeing the exact same problems I worked with in digital design during my introductory digital design courses appearing in the software world.
    <br><br>
    I will stop ranting now.
    <br><br>
    As for transactions, I am happy when people actually use transactions at all, never mind XA. All too many applications out there are designed to run with autocommit turned on.
  71. JMS[ Go to top ]

    Actually, I've not forgotten JMS, in fact I'm thinking specifically about the JMS+database case, since it is the most common case for XA. (And taking me to task for "adding confusion" is a bit unfair, since I made the suggestion, with much qualification, in a *comment thread*.)Regarding JMS. * On the message receiver side, I almost *never* need 2PC, as long as I can build my application to handle the (very rare) case of the same message being delivered twice, which is usually incredibly trivial! Indeed, it is quite likely that the app will perform better *without* XA here. 1. peek the message 2. perform the db work (checking for dupes) 3. commit the message* On the message sender side, things are a bit more difficult - but again, it is not at all hard to build recovery into the application, if you again assume that the receiver is built to anticipate duplicate delivery ...
    If you will build your own recorvery for a JMS sender then it should follow some kind of ACK type protocol in order to find out that message was received and stored in the queue; plus you will need to have some kind of tx log (on sender side not on JMS server!) which is going to store actions on JMS connection (usually application server tx manager has it for XA resources) now you will end up with your own mini recovery manager for JMS which knows how to rollback/commit JDBC connections if sending JMS message fails/succseeds :).
  72. We use XA transactions for migration data from the legacy systems (Informix database) to the new ERP data model (stored on an Oracle DB).

    We use triggers in the Informix DB which inserts data into migration tables so that they are migrated by batches. The batch deletes the data in the Informix migration tables after a successful migration. We don't delete the real legacy data from the legacy system, just the ones in the migration tables.

    Thats why we need XA transactions for it. Otherwise data could remain in the migration tables if the transaction was rolled back on Oracle. It needs to be rolled back on Informix and Oracle in case of a failure.
  73. IU uses XA transactions[ Go to top ]

    We have a workflow service that is currently implemented with EJB session beans and makes extensive use of XA transactions. We also have a number of other cases where XA is required. However, I do agree with Rod, you should only use them if you need them. Spring makes this very easy to implement by simply choosing a different datasource provider in your spring-config.xml. L
  74. It seems that a lot of people are using XA :-) In the last project I've been working in we are using XA data resources for our enterprise integration. It is also needed for JMS and database synchronization. My feeling is that for larger systems XA is needed but for simpler webapplications it's not that necessary.

    /Erik
  75. XA nightmare[ Go to top ]

    we're using (or attempting) to use xa transactions. we have a mdb which sends a few calls to oracle, pulls data from a tuxedo server (an update occurs on the tux side indicating we pulled data), saving a few more things and finally posting the results to another queue for potential final processing. all as one unit of work. all or nothing. the problem is that at the end of the day, we can have 5000 messages sent to the queue, and 4800 messages (give or take a few depending on the location of the moon) committed to the database, and no messages left on the queue. all messages made it off the queue but didn't get committed to the db. this only happens when the system is under load.

    tweaking configuration settings (db pool, db timeout, jvm max memory, etc) can _help_ the situation, but i really feel it's more hiding a problem rather than getting the job done. if you say your going to handle an XA transaction, then handle it all or roll it all back. i don't care how scarce your resources are, or how unresponsive your resources are. if you can't commit the entire block of work, please don't commit any of it. we're using the two industry leaders on this thing, and yes, the db box is very very overworked, but i could care less. all or nothing.
  76. It all depends on what you define XA as.
    XA is typically used for distributed applications, wherein you may have to connect to 2 or more db schemas. In such cases, I have used XA driver with Weblogic app server.
    There was one scenario wherein the user information was in a centralized schema, but there were some user attributes which were application specific and had to be stored in the same schema as the application. The first one could not be altered for any application specific needs. So, in this case whenever user information is to be saved, it has to be persisted across the dbs. Thats where we used XA.
    Another scenario which has been mentioned of frequently in this article is regarding the JMS and DB call being in the same transaction. I don't feel that this actually calls for XA though since most app servers don't support it, the only alternative is an XA driver. I have tried out this scenario in both WLS (Weblogic) and WAS(Websphere) and have found that in WLS, the transaction is maintained even without XA whereas WAS throws an exception. Going through WAS docs/articles, I found out that we need XA if we wanted to achieve what we needed to even though that would cause a performance overhead.

    Coming back to Gavin's question,
    How many people are using XA transactions?
    For the first scenario I mentioned, maybe only 5-10% of applications use XA, but for the second scenario, 90% of the applications with JMS will need XA.
  77. 66.67%[ Go to top ]

    Well in my personal highly unscientific survey of 1 developer/manager and 3 projects 66.67%

    I&#8217;ve used XA on 2 out of the last 3 projects I&#8217;ve worked on with Websphere MQ doing the transaction duties. The one project were this wasn&#8217;t the case was a new internet bank where the client&#8217;s DB of choice of wretched MS SqlServer which, since it wasn&#8217;t (and I imagine still isn&#8217;t) XA compliant, didn't aloow this. For a distributed architecture (Webservice or messaging based) XA is pretty damn useful I think (unless everything is in the one database).
  78. Another case of XA requirement is when you front-end a legacy application (typically CICS or IMS) with a J2EE tier, not presentation but "enhanced business rules" type of stuff.
    It is quite likely that you will need "new data" in a J2EE-controlled DBMS.
    In that case , you need to encapsulate the legacy transaction (via the JCA adapter) and the RDBMS transaction in a global transaction, J2EE being the TM.

    Practically, you encounter some problems because, for example, the IBM JCA CICS adapter implements only "last participant" and not real 2PC, so you have to compromise somehow.

    Jacques TALBOT Teamlog - France
  79. On my current project we use XA with trasactions spanning multiple databases (DB2, Oracle, SQLServer), JMS and JCA resource adapters. We are using Websphere. It all seems to work fine.
  80. Oh, and I found Mike Spille's articles on XA, here on TSS, very useful.
  81. Alternative approach to 2PC[ Go to top ]

    This is from the prevayler wiki:

    http://www.prevayler.org/wiki.jsp?topic=TwoPhaseCommitIsUnnecessary
  82. Alternative approach to 2PC[ Go to top ]

    This is from the prevayler wiki:http://www.prevayler.org/wiki.jsp?topic=TwoPhaseCommitIsUnnecessary
    I read this as being compatible with what others have already said here: in some situations 2PC is overkill. However, I don't read it as "2PC is unnecessary across the board", as might be interpreted from the title ;-)

    Mark.
  83. Alternative approach to 2PC[ Go to top ]

    This is from the prevayler wiki:http://www.prevayler.org/wiki.jsp?topic=TwoPhaseCommitIsUnnecessary
    These are the same guys who say that relational databases are unnecessary :-) It just means they are unnecessary for _them_ !

    Not that I mean to knock Prevayler - it's brilliant, but it's a niche platform.