This article is about why ACID is good for you, why ACID doesn't work in long doses, why you shouldn't give up and what concepts, models and technologies you can take in longer doses.

Conclusions

We have discussed why ACID is good, why ACID does not work in long doses, why you should not give up and what concepts, models and technologies you can take in longer doses. We used the simple example of planning a trip to illustrate a long-lived activity. We focused on the simple concept of a Saga as one way of approaching the trip planning activity. But when a Saga is extended, we end up with more generalized workflow models.
We next turned to forthcoming infrastructure to support activities. In particular, we focused on the J2EE Activity Service as defined by JSR-95. We look forward to support for activities in application servers.
Finally, we described how to implement our trip planning activity as a Saga with the J2EE Activity Service. The Activity Service is a more general mechanism than that. Other trip planning behaviors besides atomicity can be achieved using the Activity Service. We leave that as an exercise for you.

Read ACID is Good. Take it in Short Doses

Thanks to Mark Little and Bruce Martin for this well written article which articulates the need for Activity Service Support in J2EE Application Environment.

it would be exciting to see how this goes forward.

-Saroj

The need for a workflow engine emerges...

A thought provoking article. An observation, though: while I know it's not a pure transactional system, it seems to me that the industry (vendors and customers) is converging on a BPEL-based approach to solving many of the scenarios that people have traditionally positioned extended transactions to resolve. One reason may be that many proposed extended transaction models like sagas tend to require the interleaving of application logic with transaction management functions (with sagas, compensators are generally pure application domain logic; the extreme example of this is BTP). It will be interesting to see how things play out over the next few years, but I think application structuring on infrastructure like the Activity service is unlikely to become a popular model. On the other hand, I predict that BPEL engines will be very widely deployed.

It's also worth mentioning that BPEL does build on concepts from extended transaction models including sagas and open nested transactions.

Again, nice work.

Greg

A thought provoking article. An observation, though: while I know it's not a pure transactional system, it seems to me that the industry (vendors and customers) is converging on a BPEL-based approach to solving many of the scenarios that people have traditionally positioned extended transactions to resolve. One reason may be that many proposed extended transaction models like sagas tend to require the interleaving of application logic with transaction management functions (with sagas, compensators are generally pure application domain logic; the extreme example of this is BTP). It will be interesting to see how things play out over the next few years, but I think application structuring on infrastructure like the Activity service is unlikely to become a popular model. On the other hand, I predict that BPEL engines will be very widely deployed.It's also worth mentioning that BPEL does build on concepts from extended transaction models including sagas and open nested transactions.Again, nice work.Greg

I think that for anything other than simple Saga applications, you need something that will look very much like a *reliable* workflow engine. Now, whether that's BPEL or not depends upon the environment you're operating in. The way BPEL is going in OASIS at the moment it's often difficult to see how it'll be useful in Web Services, let alone J2EE ;-)

But I think from a pragmatic stand point, you're right in that BPEL will/should become the standard. It would be good for it to be less Web Services specific though. But perhaps that's a tooling thing anyway.

Mark.

Hi Greg,

A thought provoking article. An observation, though: while I know it's not a pure transactional system, it seems to me that the industry (vendors and customers) is converging on a BPEL-based approach to solving many of the scenarios that people have traditionally positioned extended transactions to resolve.

This is probably going to be more of a matter of business policies than something else, but I think it really depends on the application. Some existing business models already use a two-phase approach (e.g. reserve a flight, then confirm your booking). The BTP or WS-Activity models seem almost a natural fit, better than BPEL IMO.

In addition, 'abusing' workflow (pardon the term) to deal with transactional matters means that the workflow modeler needs to explicity foresee both the compensations and all possible combinations thereof. This doesn't scale well, it is error-prone and difficult to maintain. In the end, it might turn out much more expensive than anything else (linking back to Bruce and Mark's article: it would again mean that errors, exceptions an crashes are resolved by the application).

It will be interesting to see how things play out over the next few years, but I think application structuring on infrastructure like the Activity service is unlikely to become a popular model.

I certainly agree with you here. Powerful as it is, the activity service is probably more something that the BPEL engines will use internally, but I don't see how the regular Java developer would use it. Speaking from our past experience at Atomikos: we have gone out of our way to make the relatively simple JTA API as easy to use as possible, because many people found it cumbersome. And I have yet to meet the first regular Java developer who knows or understands the OTS, to give another example.

Best,

Guy Pardon
Atomikos: The Transaction Management Company

I would like to propose difinition of
"long transaction" as temporary legalization of
high-level inconsistent state.

temporary == within specified time this state
must be transferred into stable state.

legalization == can be defined in business terms
and must be aproved by business process owner.

high-level inconsistent == means inconsistent from
over-all business point of view. No PK-FK violations,
no resources locking.

Now a question, what is more important:

1. define set of "big transactions" spanning
several intermediate states, like flow engines
works, or

2. define set of state transition rules, like
state machine and rule engine.

Of cource, "Time" (cron, timer) is important
player in both cases.

IMHO, option 1 is more frendly for analysis and
use case definition, but option 2 is more scalable
on technical level.

For example, by option 2, we can define:
-- within 15 min state A(car-hotel-no_ticket)
must be tranfered into B(no_car-no_hotel-no_ticket)

According option 1, we have to define several
transactions leading to state A with
[unnessesary different]
scenarios of A-B transition after 15 min.

As usual, thruth is somewhere in between...

Alex V.

JAIN-SLEE (Java API for Integrated Networks, Service Logic Execution Environment), although driven by telecom industry is applicable to any event-oriented environment. It is a completely event-based architecture.

Is there any direct or indirect association between the two?

Read more about JAIN-SLEE at http://jainslee.org or http://java.sun.com/products/jain/

JAIN-SLEE Activity Contexts are conceptually similar. However they are intended for use in environments with transaction volume in the order of 1000s/sec where each individual request is sub 100ms. I would expect that JSR 95 would add a few paragraphs in reference to JSR 22 to acknowledge the commonalities and explain the differences.

If you are further interested in open source SLEE implementations, take a look at mobicents.org.

Ivelin Ivanov

Many databases and some application servers weaken serializability with their so-called isolation levels.

"So-called"? Aren't they defined in SQL standard? SQL2 defined four standard isolation levels more than a decade ago.

The weaker isolation levels are not the same from one database to another.

Actual implementation of SERIALIZABLE isolation level (ok, of SERIALIZABLE keyword) can be different as well, and some databases may not support it at all.

This requires you to reason about using inconsistent data and this is hard. You have to use application knowledge to argue that a transaction reading an inconsistent, possibly to be rolled-back value, doesn’t matter to the correctness of the application.

Unless the transaction isolation level is DIRTY READ, the value is absolutely not "possibly to be rolled-back". If you are talking about inconsistent updates or phantom data, this applies only to the case when the second transaction makes multiple reads. This is important for multiuser desktop applications, but I would say that this does not make a big difference for a web application. Database transaction usually is not kept opened during a whole client session to increase concurrency (this article is exactly about that). So, each HTTP request usually starts a new database transaction and immediately commits it (within possibly the same application transaction, here is where EJB may help). Each transaction usually makes only one read or update. Thus there is no difference would the value change or not, because it is read only once, and this is a committed value unless for some strange reason DBA set isolation level to DIRTY READ.

In the example, if a transaction reserves the last car, a second transaction can observe that fact and conclude there are no cars available. But if later we compensate for the first transaction by canceling the reservation, the second transaction has observed an inconsistent state.

To notice the change in data the second transaction must be long-lived and must reread data. Why would we design a long-lived multiread transaction, if we already decided to go with short ones for update? More realistically, the second transaction will start, observe that there are no cars available and will return to a user with "Nothing available, try again later". Or, if database designer were a little smarter, the second transaction would read "Reserved but not confirmed" flag and would tell a user "Some cars may be available later, try in N minutes", where N = M - K, where M is average time spent by a user from starting reservation to its finish, and K is the length of the current user session.

[By using HLS,] if at any point a transaction rolls back, the CompletionSignalSet is responsible for ensuring that the enclosing activity must fail, triggering any remaining compensation transactions.

This support from application server is nice, but as I see nothing changes on the database level. Which means, that if compensating database transaction fails, system becomes inconsistent. Thus, HLS is just a convenient framework, but it does not solve the original problem.

Actual implementation of SERIALIZABLE isolation level (ok, of SERIALIZABLE keyword) can be different as well, and some databases may not support it at all.

Agreed, but that kind of level isn't suitable for this article. The aim of the article was to bring the issue of ACID transactions and their shortcomings to as wide an audience as possible.

Unless the transaction isolation level is DIRTY READ, the value is absolutely not "possibly to be rolled-back".

Again this is a language issue. If you're looking for a technical paper with all of the details, then take a look at the references. Remember, this paper is intended only to wet the appetite.

If you are talking about inconsistent updates or phantom data, this applies only to the case when the second transaction makes multiple reads. This is important for multiuser desktop applications, but I would say that this does not make a big difference for a web application. Database transaction usually is not kept opened during a whole client session to increase concurrency (this article is exactly about that). So, each HTTP request usually starts a new database transaction and immediately commits it (within possibly the same application transaction, here is where EJB may help). Each transaction usually makes only one read or update.

It really depends on your experiences, which in this case seem counter to the rest of the industry. Take a look at the OASIS WS-CAF work, or the IBM/MSFT/BEA WS-Atomic Transaction or WS-Business Activity specifications. If you disagree with the models on which they are based (and hence the assumptions), then get involved.

To notice the change in data the second transaction must be long-lived and must reread data.

Not true. The example says that the second transaction has terminated and gone and done something based on that (now incorrect) data it read. This is the traditional case of cascade abort.

This support from application server is nice, but as I see nothing changes on the database level. Which means, that if compensating database transaction fails, system becomes inconsistent. Thus, HLS is just a convenient framework, but it does not solve the original problem.

Compensation transactions aren't easy to write and manage, but they do provide suitable functionality in the right case. As we tried to show, it's important to use the right tool for the right job. The Activity Service definitely doesn't solve the problem, but hopefully it provides a good starting point for people to build on.

the article was good, but kinda wished it went further into compensating transactions. though writing about compensating transactions is not easy. especially if you change the scenario from travel to hedge fund.

the new JSR looks like it might make long-lived processes a bit easier to handle. it's nice to have informative stuff in between the fluff pieces like state of java platform.

the article was good, but kinda wished it went further into compensating transactions. though writing about compensating transactions is not easy. especially if you change the scenario from travel to hedge fund.the new JSR looks like it might make long-lived processes a bit easier to handle. it's nice to have informative stuff in between the fluff pieces like state of java platform.

Hi,

Yes, an article focusing on compensation would be nice. I am thinking of writing one myself.

Compensating transactions have been studied in minute detail in the literature. To name an example in the context of web services, there is this research on compensating transactions in which I was involved in a previous life. At Atomikos we're still busy integrating this technology in our commercial product line.

The tricky part is how to expose compensation to the application logic (someone has to provide the compensation logic). This will have to be done in a way that is accepted by the business models for web service partners. At this point I think it is unrealistic to assume that there is a workflow engine (located somewhere on the Internet) that will take care of this. This is because the compensation is closely tied to the integrity of the business data of a particular web service. Consequently, assuming a loose federation of web services, no business will want to 'outsource' that integrity to another web service.

To use the saga example mentioned in Mark and Bruce's article: would one want to trust some other party to invoke all the compensations in the exact reverse order? Probably not.

Best,

Guy Pardon
The Transaction Management Company

Hi,Yes, an article focusing on compensation would be nice. I am thinking of writing one myself.Compensating transactions have been studied in minute detail in the literature. To name an example in the context of web services, there is this research on compensating transactions in which I was involved in a previous life. At Atomikos we're still busy integrating this technology in our commercial product line.The tricky part is how to expose compensation to the application logic (someone has to provide the compensation logic). This will have to be done in a way that is accepted by the business models for web service partners. At this point I think it is unrealistic to assume that there is a workflow engine (located somewhere on the Internet) that will take care of this. This is because the compensation is closely tied to the integrity of the business data of a particular web service. Consequently, assuming a loose federation of web services, no business will want to 'outsource' that integrity to another web service. To use the saga example mentioned in Mark and Bruce's article: would one want to trust some other party to invoke all the compensations in the exact reverse order? Probably not.Best,Guy PardonThe Transaction Management Company

I downloaded the article and will read it. thanks for providing the link.

the article was good, but kinda wished it went further into compensating transactions. though writing about compensating transactions is not easy. especially if you change the scenario from travel to hedge fund.the new JSR looks like it might make long-lived processes a bit easier to handle. it's nice to have informative stuff in between the fluff pieces like state of java platform.

I agree on both counts ;-)

If you're particularly interested in Web Services and compensations, take a look at the following:

http://www.sys-con.com/webservices/articleprint.cfm?id=737
http://www-106.ibm.com/developerworks/webservices/library/ws-comproto/
http://www.sys-con.com/webservices/articleprint.cfm?id=622
http://www.sys-con.com/webservices/articleprint.cfm?id=587
http://www.arjuna.com/library/technical/2004-06-SIGMOD2004-WebServicesTransactionsModels-paper.pdf
http://www.arjuna.com/library/technical/2003-07-HPTS-FullCircleForWebServicesTransactions.pdf