Discussions

News: Transactions are your friend

  1. Transactions are your friend (25 messages)

    http://weblogs.java.net/blog/marklittle/archive/2006/03/transactions_ar_1.html

    In "Transactions are your friend," Mark Little discusses how transactions provide the knowledge that your interactions with data are valid and your dataset is healthy, along with some of the common perceptions around transactions, such as "transactions" being the same as two-phase commit and others.
    ... "everything I do is within a single VM, so I don't need transactions". This is definitely an education issue, where distributed transactions have become synonymous in the minds of many people with transactions. Most people can see that if they're accessing resources/participants across physically distinct machines or processes, there's a need for transactions to coordinate simultaneous updates to state. In a local (single VM) environment, the need is often overlooked. But it is still there: in many cases, even within the same VM, applications use and modify data from multiple different sources, and in that case, you need the benefits that transaction processing provides. Distribution just makes it more obvious that independent failures can cause problems. But they're still there in the local case; you may just have to look a little harder.

    ... think of transactions like an insurance policy: compared to how much time, money and effort you may lose by not using them, the cost of using them may be well worth it. Obviously there's a tiping point on any graph of cost of using transactions versus advantage they may bring, and that point is going to be very dependent on your application. But consider it nonetheless.

    Threaded Messages (25)

  2. "The intention (mirrored by Microsoft's work with Indigo transactions) is to make transaction implementations so lightweight, with low overhead that they'll become a natural part of the infrastructure (and in the case of Microsoft, that'll mean in the operating system). We've already seen them moving into hardware, so this makes a lot of sense too."

    That is definitely right way to go, but I much rather see transaction support right in programming language.
    Unfortunately it can be done in next generation language.
  3. Transactions are your friend[ Go to top ]

    "The intention (mirrored by Microsoft's work with Indigo transactions) is to make transaction implementations so lightweight, with low overhead that they'll become a natural part of the infrastructure (and in the case of Microsoft, that'll mean in the operating system). We've already seen them moving into hardware, so this makes a lot of sense too."That is definitely right way to go, but I much rather see transaction support right in programming language.Unfortunately it can be done in next generation language.

    Actually we have this kind of thing already here, there are several frameworks out there which cover transactional scopes via annotations or aop.

    @Transactional public void doSomething() is the closest you can get on language level covering transactions.

    And I do not see the problem of not using transactions as distributed transactions being slow metaphor being the fault.
    I personally think the hammering of the MySQL marketing departement and devs who were not able to provide proper transactions support on their DB for several years and branding them as a unnecessary feature did way more damage. We have a population of people having grown up with it who never were exposed to techniques which have been proven worthwile for at least 20 years. Well MySQL has caught up and has decent transaction support, and also distributed transactions probably are not slow anymore (never have used them because I never have run into a cluster situation so far), but the problem of having those issues in many peoples minds still is there.

    All I can say is not using transactions is like scr*** without condom, if you are lucky you can get away with it, if not, then you have a huge problem on your hands.
  4. Transactions are your friend[ Go to top ]

    All I can say is not using transactions is like scr*** without condom, if you are lucky you can get away with it, if not, then you have a huge problem on your hands.

    You are mixing your metaphors ;-).

    Peace.
  5. Transactions are your friend[ Go to top ]

    And I do not see the problem of not using transactions as distributed transactions being slow metaphor being the fault.I personally think the hammering of the MySQL marketing departement and devs who were not able to provide proper transactions support on their DB for several years and branding them as a unnecessary feature did way more damage. We have a population of people having grown up with it who never were exposed to techniques which have been proven worthwile for at least 20 years.

    You're right in that MySQL hasn't helped in recent years, but transactions (and distributed transactions) existed way before MySQL. So maybe in some circles MySQL could be seen as one of the dominant problems, but in a more general situation (and my article was definitely generalising), they are just a blip.

    Mark.
  6. Transactions are your friend[ Go to top ]

    How can you "not use a transaction"? I thought all interaction with a database starts a transaction, which at some point will be either committed or rolled back. In fact sometimes if transactions are not explicitly demarcated you end up with too many and hence too many commits and worse performance.

    E.g. in this kind of scenario:
    <code>
    for (each account) {
      clear();
    }
    </code>
    with a 1000 accounts by failing to think about the transactions you may end up with a 1000 commits, whereas if the whole loop runs in one transaction, this will probably be faster.

    I suppose other resources, such as message queues, etc, can be non-transactional, but can you have non-transactional interaction with a relational db?
  7. Transactions are your friend[ Go to top ]

    I know people here dislike ejbs, but if you are in an ejb most of the time you're in some sort of transaction, although most developers are probably unaware of how they have been configured and just life with the defaults. In fact in some ways ejbs sometimes seem pretty neat when it comes to transactions, though maybe not worth the hassle for this alone....
  8. Transactions are your friend[ Go to top ]

    I know people here dislike ejbs, but if you are in an ejb most of the time you're in some sort of transaction, although most developers are probably unaware of how they have been configured and just life with the defaults. In fact in some ways ejbs sometimes seem pretty neat when it comes to transactions, though maybe not worth the hassle for this alone....
    I couldn't agree more. For a long time people were so busy going blue in the face about entity beans they overlooked that fact that the declarative transaction model provided by EJB was an easy way for junior developers to build transactional applications. (without of course, needing the dreaded entity beans).

    My concern at the time was the fact that by bundling units of work together into a transaction by default, the transactions (and therefore contention window) could grow very large very easily.

    An old thread:
    http://www.theserverside.com/patterns/thread.tss?thread_id=14482

    Now of course spring and/or aspectj give you a far better way - weave in your transaction boundaries.

    Come to think of it - with web services the most popular remoting model*, and the ability to weave in your security and transactional behaviour in any codebase, why do we need EJB again??

    Perhaps the latest EJB entrant, the MDB, is the only one worth keeping :)

    *apologies for the sweeping statement
  9. George,

    You are right that a relational database must use a transaction. At least the non-embedded ones all do. You can not be aware of this: using say the JDBC auto-commit mode. But it is there.

    My point (which I'm not sure that you were responding to) is that not all applications we build use a transactional relational database. Some of us use non-transactional data stores that aren't relational, or can have certain elements of relational semantics but don't have to (e.g. no constraints are enforced, the data set is always sorted in a particular way etc) and even multi-dimensional data structures that are purely read-only and don't have any need for transactions.

    Most people who do java web applications in the enterprise deal with a relational, transactional data store. Spring does a great job of helping you to separate out the transaction demarcation. But the sort of apps that I build are mostly not in this space at all.
  10. Sorry, I'm not arguing, just asking... I posted the reply under the wrong branch, which doesn't help either :)
  11. Sorry, I'm not arguing, just asking... I posted the reply under the wrong branch, which doesn't help either :)

    What you really needed were transactions: then you could have rolled back ;-)

    Mark.
  12. Transactions are your friend[ Go to top ]

    How can you "not use a transaction"? I thought all interaction with a database starts a transaction, which at some point will be either committed or rolled back. In fact sometimes if transactions are not explicitly demarcated you end up with too many and hence too many commits and worse performance.E.g. in this kind of scenario:<code>for (each account) {&nbsp;&nbsp;clear();}</code>with a 1000 accounts by failing to think about the transactions you may end up with a 1000 commits, whereas if the whole loop runs in one transaction, this will probably be faster.I suppose other resources, such as message queues, etc, can be non-transactional, but can you have non-transactional interaction with a relational db?

    Yes and that's fine if you only ever use a single database in the scope of a transaction (if you do that within a transaction managed by an external TM, then you'll implicitly end up using the one-phase commit optimization I mentioned). Those DB transactions are often referred to as local transactions, whereas the TM ones are global.

    However, if you use multiple DBs or, as you mention, a DB and a queue (for example), then the transaction in the database isn't going to help you. Even if both DBs, or the queue, use transactions, they will be unrelated transaction, in the sense that they can commit or abort/roll back independently. That's why you need a global transaction: to manage them consistently and reliably.
  13. How can you "not use a transaction"? I thought all interaction with a database starts a transaction, which at some point will be either committed or rolled back. In fact sometimes if transactions are not explicitly demarcated you end up with too many and hence too many commits and worse performance.E.g. in this kind of scenario:<code>for (each account) {&amp;nbsp;&amp;nbsp;clear();}</code>with a 1000 accounts by failing to think about the transactions you may end up with a 1000 commits, whereas if the whole loop runs in one transaction, this will probably be faster.I suppose other resources, such as message queues, etc, can be non-transactional, but can you have non-transactional interaction with a relational db?
    Yes and that's fine if you only ever use a single database in the scope of a transaction (if you do that within a transaction managed by an external TM, then you'll implicitly end up using the one-phase commit optimization I mentioned). Those DB transactions are often referred to as local transactions, whereas the TM ones are global.However, if you use multiple DBs or, as you mention, a DB and a queue (for example), then the transaction in the database isn't going to help you. Even if both DBs, or the queue, use transactions, they will be unrelated transaction, in the sense that they can commit or abort/roll back independently. That's why you need a global transaction: to manage them consistently and reliably.

    George had made a point with the number of commits: interactions with a database can be faster with proper delimited transactions.

    A DB system does not write all the inserts/updates/deletes directly to disk: it keeps them in memory and puts the modification in a log queue. The log queue is read and written to disk by a parallel thread in the DBMS. When the application issues a commit(), the DB system waits until all modifications have been flushed to disk.

    If you do not delimit transactions, there will be an autocommit after each SQL statement, i.e. you wait for the log writing step to be finished. If you delimit the transactions and group several statements in a transaction, the log writing step is actually performed in parallel and you only wait once, at the end of the transaction.
  14. Tx types[ Go to top ]

    I'd be interested to hear how others have handled the case where an application goes from: no transactions required, to local transactions only, to global transactions. Specifically how it affected their design.

    Most of my apps to date have used local database transaction capability. But recently I ran into a case where a last minute requirement looked like I'd need to use a two-phase commit. To make a long story short, I ended up using a commit across two different schemas in the same database instead -- which did affect my design to a certain extent (specifically my DAO layer). I just wonder how badly my code would have had to change if in fact I had to restructure it around global transactions...
  15. Tx types[ Go to top ]

    I'd be interested to hear how others have handled the case where an application goes from: no transactions required, to local transactions only, to global transactions.
    Spring's transaction abstraction is designed to provide an elegant solution to bridging local and global transactions, either programmatically or declaratively.

    See the transaction chapter from the Spring Reference Manual.

    I have to say that in my experience, as Mark points out, transactions are indeed your friend. "No transactions required" is almost unknown in any real server-side application I've seen.

    Rgds
    Rod
  16. Tx types[ Go to top ]

    I'd be interested to hear how others have handled the case where an application goes from: no transactions required, to local transactions only, to global transactions.
    Spring's transaction abstraction is designed to provide an elegant solution to bridging local and global transactions, either programmatically or declaratively.See the transaction chapter from the Spring Reference Manual.

    Actually in most systems I know of that support distributed transactions, there no special API required to differentiate them from local transactions. OTS and JTA are examples of standards that quite nicely isolate you from that difference, at least as a developer, and implementations like JBossTS, Encina and CICS, are pretty good at it too: transaction inflow and outflow occur naturally as part of your object/service/procedure invocation.
    I have to say that in my experience, as Mark points out, transactions are indeed your friend. "No transactions required" is almost unknown in any real server-side application I've seen.RgdsRod

    You'd be surprised how many banks, telcos and other financial institutions (to pick on just a few verticals) don't, but think they do because "the database has transactions in it". Plus the "local transaction" issue I mentioned doesn't help.

    I would have to say that, after so many years of banging my head off brick walls, less than 50% of the companies I've talked to who need transactions actually use them. On many occasions it's simply because they didn't know they needed them, but on some occasions they knew but simply didn't want to have the overhead (or cost) of using them and were willing to take the risk (and usually on behalf of customers and without telling them).
  17. Tx types[ Go to top ]

    You'd be surprised how many banks, telcos and other financial institutions (to pick on just a few verticals) don't, but think they do because "the database has transactions in it". Plus the "local transaction" issue I mentioned doesn't help.I would have to say that, after so many years of banging my head off brick walls, less than 50% of the companies I've talked to who need transactions actually use them.

    The big companies I visit all use transactions and are aware of the fact that they need them and they do use them. Difficulties arise when you're working with blended architectures that include mainframes, message services, relational databases and J2EE applications connecting to those backends. This is where sometimes people are satisfied with a transaction strategy that covers 99.999% of all possible scenarios. In those situations sometimes it's simply not worth covering the 0.0001% which is left but instead solve things with a manual procedure or something.

    About switching from local to global--to some degree you're right--sometimes people are reluctant to switch to global transactions; largely because of performance reasons, not so much because of increased complexity. At least, that's what I'm seeing.

    regards,
    Alef
  18. Tx types[ Go to top ]

    The big companies I visit all use transactions and are aware of the fact that they need them and they do use them. Difficulties arise when you're working with blended architectures that include mainframes, message services, relational databases and J2EE applications connecting to those backends. This is where sometimes people are satisfied with a transaction strategy that covers 99.999% of all possible scenarios. In those situations sometimes it's simply not worth covering the 0.0001% which is left but instead solve things with a manual procedure or something.About switching from local to global--to some degree you're right--sometimes people are reluctant to switch to global transactions; largely because of performance reasons, not so much because of increased complexity. At least, that's what I'm seeing.regards,Alef

    As they say: your mileage may vary. Although I'm sure there are some verticals out there that may cover what you're seeing, in my experience at developing transaction processing systems, using them and consulting on them to large (often multi-national) companies in many vertical sectors for more years than I care to remember, they're not used as much as they should be, for the reasons I mention in the blog. It's a long hard "sell" to convince people (and I agree, complexity isn't the issue).

    Mark.
  19. Tx types[ Go to top ]

    About switching from local to global--to some degree you're right--sometimes people are reluctant to switch to global transactions; largely because of performance reasons, not so much because of increased complexity. At least, that's what I'm seeing.regards,Alef

    BTW, I didn't mention complexity at all in the blog entry. Where'd you come up with that? I did mention performance (or perceived lack thereof), which is down to a combination of things, all of which (with the exception of distribution) would be necessary in an intra-process situation. So I assume you just mean you agree with me entirely then, not just to a degree ;-)
  20. Tx types[ Go to top ]

    Sorry, it's late here and all of this should have been covered in a single reply.
    This is where sometimes people are satisfied with a transaction strategy that covers 99.999% of all possible scenarios. In those situations sometimes it's simply not worth covering the 0.0001% which is left but instead solve things with a manual procedure or something.

    Yes, and that's the tiping point that I mentioned: presumably (hopefully) these companies took the time to weigh up the probabilities of failures and the cost to them (time, money, effort, etc.) to resolving these failure situations manually. That's fine and I've seen that enough times before to know it works. But it has to be an informed judgement.
  21. banks not using transactions[ Go to top ]

    You'd be surprised how many banks, telcos and other financial institutions (to pick on just a few verticals) don't, but think they do because "the database has transactions in it". Plus the "local transaction" issue I mentioned doesn't help.I would have to say that, after so many years of banging my head off brick walls, less than 50% of the companies I've talked to who need transactions actually use them. On many occasions it's simply because they didn't know they needed them, but on some occasions they knew but simply didn't want to have the overhead (or cost) of using them and were willing to take the risk (and usually on behalf of customers and without telling them).

    Last year I was dumbfounded by an experience with one of Australia's big four banks. I was using their online net banking. I submitted a largish money transfer to an external account and the page came back with an error. I looked at the bank balance - the balance was missing the money, but there was no transaction in the history. The remote account also did not receive the funds.

    The balance was updated but the transaction wasn't recorded? Obviously a non transactional process.

    After many arguments on the phone, the bank admitted the issue, and did a manual fix of the balance. I assumed they would then seriously examine their net banking app and fix the issue.

    This year, I was doing something in the app (not a transfer thankfully), and I got an error page back. The error page had a section saying something like "warning: if you were doing a transaction check your account balance to ensure the money wasn't deducted from your balance".

    I was dumbfounded - faced with a terrible flaw in the system's design, they simply went to the error page and added a note telling customers to check their account balance to ensure their money hadn't vanished into thin air!

    IMHO, this is a symptom of accounting-driven corporations building their outsourced relationships purely on a cheapest available labour strategy.
  22. banks not using transactions[ Go to top ]

    IMHO, this is a symptom of accounting-driven corporations building their outsourced relationships purely on a cheapest available labour strategy.

    I agree. Most customers don't have enough informatic knowledge to understand whether the job is done properly or not.
  23. RE: banks not using transactions[ Go to top ]

    Last year I was dumbfounded by an experience with one of Australia's big four banks. I was using their online net banking.

    The problem is that in the case of internet "transactions" the client (== you) cannot hold (begin, commit, rollback) the transaction. This must be done downstream. Nevertheless the bank must guarantee 'exactly once semantics'.
  24. RE: banks not using transactions[ Go to top ]

    Last year I was dumbfounded by an experience with one of Australia's big four banks. I was using their online net banking.
    The problem is that in the case of internet "transactions" the client (== you) cannot hold (begin, commit, rollback) the transaction. This must be done downstream. Nevertheless the bank must guarantee 'exactly once semantics'.
    Not sure I'm getting what you mean here. If you imagine that this situation was a relational DB, then the bank would be doing an:
    1. update balance (one transaction)
    and then:
    2. insert transaction (next transaction)
    The error presumably occurred in between the two.

    In reality I imagine the bank was like most other banks - a whole heap of huge old legacy systems with crazy stuff like text terminal screen scaper integration.

    This can be made _somewhat_ safer by doing the non-transactional unit of work last. (This falls down if an error occurs on the commit itself).
  25. banks not using transactions[ Go to top ]

    Last year I was dumbfounded by an experience with one of Australia's big four banks. I was using their online net banking. I submitted a largish money transfer to an external account and the page came back with an error. I looked at the bank balance - the balance was missing the money, but there was no transaction in the history. The remote account also did not receive the funds.The balance was updated but the transaction wasn't recorded? Obviously a non transactional process.
    Maybe not. What about an evil heuristic mixed transaction outcome ?
    Apart the error, the evidence of completion on your account might be due to the delay in the counterpart resource commit happening after the commit of the resource of your account ? (Yes, maybe this case requires a direct access to the resource, but...).

    Guido.
  26. Transactions are often overlooked when the number of users is very low and the main point of accessing the data is almost entirely read-only. This is the case with most Business Intelligence tools. I work with a lot of large companies and when they have an Oracle database for their operations, they use transactions. When they want decision support systems they extract the data in a transaction and build a read-only structure (data warehouse, data mart or cubes/tables within these) that doesn't need transactions.

    Having said that, I see increasingly the need for a blending as more and more we write web apps to parameterise the sorts of queries to run against the data warehouses. Those have to run in transactions and that creates a lot of headaches for people who haven't had to deal with the problem domain before.

    Just my 2 cents anyway.