Discussions

News: All you ever wanted to know about Workflow...

  1. Tom Baeyens, in "All you ever wanted to know about Workflow and how it relates to Java, Transactions and Concurrency," offers an explanation of workflow and how it actually works, saying that reading this entry will put you in for a payraise. I don't know if that guarantee is valid, but the text makes some good points about workflow and transactions.
    In summary: Multithreaded computing does not have anything to do with process concurrency. Process executions move from one state to the next in a database transaction. One such transaction can always be calculated in one simple thread. Each signal is handled in one database transaction. So when multiple incoming signals can arrive simultaneous, the database synchronization features (locking and isolation levels) can be leveraged to handle process concurrency.

    If you got this far and still got a clue, MENTION IT ON YOUR NEXT EVALUATION and you can be sure that you're in for a big payraise :). Actually I'm serious ! Knowing how business processes relate to plain java and database transactions is KEY to make the software for your project simple, robust and maintainable.
    What do you think of his explanation? And will you hold him responsible if you don't get a raise after understanding his point?

    Threaded Messages (20)

  2. misleading title[ Go to top ]

    Such a long blog to state the obvious: business transaction != DB transaction

    Looking at the title I expected some in depth comparison of explicit workflow definition in some kind of definition language versus implicit definition throughout an application or several of them.
    Maybe something about compensation transactions strategies ( rollbacks at business level).
    Or maybe summary of Van Der Aalst works in plain English.
    Found none of it.
  3. misleading title[ Go to top ]

    Konstantin, you raise some interesting points.
    in depth comparison of explicit workflow definition in some kind of definition language versus implicit definition throughout an application

    This will be a large part of the talk i'll give at TSSS Barcelona in June.

    In short, without using a process language, a developer has to cut the overall process into request/response operations. Roughly spoken, each request/response will correspond to a state transition in the workflow, typically being one transaction. Inbetween the requests, the developer has to maintain the state in the database 'manually'.
    Maybe something about compensation transactions strategies

    In the blog I try to explain how state transitions in a workflow system map onto the well known transactions in the current Java technologies.

    Suppose the state of a process execution is represented by a pointer to a node in a graph. The nodes are records in the database. In JBoss jBPM we call the pointer to the node a Token. One state transition (= transaction) will be the update of the Token's node reference. So a compensating transaction in workflow would be to update the token back to the original node. This compensating transaction can be calculated from the logs that the process execution has generated in the DB.

    But of course, compensating transactions in the context of workflow suffer from the same pitfalls as any compensating transaction. They can compromise the ACID properties. So you need to consider your application logic very carefully.
    Or maybe summary of Van Der Aalst works in plain English.

    My answer to Van Der Aalst's workflow patterns (which i suppose is the work of Van Der Aalst that you refer to) is Graph Oriented Programming. Basically workflow patterns is the first large scale attempt to categorize all the control flow constructs that you can find in process languages.

    My position is based on 2 arguments.

    1) In Graph Oriented Programming, the behaviour of each node is implemented in plain Java (more precise, any imperative OO programming language). This means that for any desired workflow node behaviour, you can just start writing Java code to implement that behaviour. This is also one of the main ideas behind jBPM's extensibility. If the our jPDL process language doesn't have a workflow construct for the specific behaviour that you want in a node, you can just include your Java programming logic in there.

    2) Different environments require different process languages. A lightweight Java environment is completely different from an enterprise service bus environment. Both environments deserve to have their own process language. So i think the quest for the ultimate process language is on a dead end. A good indication for this is that in the past 20 years, process languages did not show any sign of convergence.

    regards, tom.
  4. I hear it over and over:

    - JBoss is a bunch of braggarts
    - JBoss overpromises and underdelivers

    But then we get fine articles like this, promising complete knowledge of workflows and how to manage them, step by step, and a GUARANTEED salary raise to boot. JUST FOR READING this simple article. Workflow and transactions completely explained in 2 pages of web content. I hope you JBoss critics can finally fess up to how wrong you were.

    Also, make millions working from home on buying property for no money down while doing Tony Little's Life Extension Institute power swinger ab maker 10000 and do your laundry with Oxy-Clean instantaneously.


    BTW, I actually like a lot of what JBoss does, if its proponents weren't in some mortal deathlock chicken contest with Ruby on Rails people to be as arrogant as possible.
  5. Also, make millions working from home on buying property for no money down while doing Tony Little's Life Extension Institute power swinger ab maker 10000 and do your laundry with Oxy-Clean instantaneously.

    I tried to oxyclean stuff. Doesn't work.

    I do buy properties with 0-to-little down though. That does work!

    STAY METAL!
    Roy Russo
    http://jboss.org/jbossBlog/blog/rrusso/
  6. I hear it over and over:- JBoss is a bunch of braggarts- JBoss overpromises and underdeliversBut then we get fine articles like this, promising complete knowledge of workflows and how to manage them, step by step, and a GUARANTEED salary raise to boot. JUST FOR READING this simple article. Workflow and transactions completely explained in 2 pages of web content. I hope you JBoss critics can finally fess up to how wrong you were.Also, make millions working from home on buying property for no money down while doing Tony Little's Life Extension Institute power swinger ab maker 10000 and do your laundry with Oxy-Clean instantaneously.BTW, I actually like a lot of what JBoss does, if its proponents weren't in some mortal deathlock chicken contest with Ruby on Rails people to be as arrogant as possible.

    so much anger it this post... i guess it's been a long time since your last payraise ;-) maybe this can help.

    regards, tom.
  7. so much anger it this post... i guess it's been a long time since your last payraise ;-) maybe this can help.regards, tom.

    Since you are so knowledgeable about transactions and isolation levels, can you explain the difference between READ COMMITTED and SERIALIZABLE as implemented by Oracle? Specifically, under what conditions should you expect rollbacks?

    Guglielmo

    Enjoy the Fastest Known Reliable Multicast Protocol with Total Ordering

    .. or the World's First Pure-Java Terminal Driver
  8. Nice put, but maybe i am missing something here:

    "With Java (or any other imperative programming language) you just don't have a way to program a persistent wait state. Hence you can not express the overall process in Java"

    AND

    "A traditional state machine is not sufficient to express real life processes. Real life processes can have multiple concurrent paths of execution, whereas a traditional state machine only has one"

    To me you just painted a nested statemachine. statemachines in java(even Pascal is you prefer..) is easy. There is even a pattern for it. Combine each statemachine(+sub) with JMS to apply concurrency and throw in a DB and you got exactly the same implemented in java as you do in BPEL.

    The only difference is you dont have a bunch of XML files expressing what you a trying to do.

    I have spent many hours with WPC(IBM workflow) the last year and i am not impressed...

    Please explain to me what i am missing here :)

    Regards Frank
    --
    Frank Bengtsson
    Lenio ApS
  9. Probably graphical representation confuses, Workflow and state machine are different things and there is nothing to compare except graphical stuff (nodes and edges), but state machine can be used to implement internal workflow engine stuff.
  10. To me you just painted a nested statemachine. statemachines in java(even Pascal is you prefer..) is easy. There is even a pattern for it. Combine each statemachine(+sub) with JMS to apply concurrency and throw in a DB and you got exactly the same implemented in java as you do in BPEL.

    I first explained that the basis of every process language is specifying some kind of state machine. I didn't go into details on how to implement that state machine. Many workflow engines are indeed implemented on top of a message queueing system. Graph Oriented Programming is another way of implementing state machines. Graph Oriented Programming is more flexible in the sense that it doesn't require transaction overhead when you don't need it. E.g. If you have 3 sequential nodes in a workflow diagram doing all a very simple operation. Amongst other benefits, Graph Oriented Programming allows you to combine the execution of these 3 in one transaction. Whereas systems based on message queues typically need 3 transactions with a lot of message sending and receiving.
    The only difference is you dont have a bunch of XML files expressing what you a trying to do.I have spent many hours with WPC(IBM workflow) the last year and i am not impressed...

    Software projects are not longer realized by using 1 programming language. More and more Domain Specific Languages found their way into software development. XML is currently the most used host for Domain Specific Languages cause of the availability of tools.

    You should not evaluate a solution on the amount of XML. It is the languages that are offered in XML files that you have to evaluate. It is the content of the languages that should be evaluated. Are they worth their learning curve ?

    In the case of the IBM product, i didn't work with it yet. So I can't comment wether they offered a concise language and an easy embeddable engine.

    Looking forward, i think chances are good that we'll see a decline of XML as the host for Domain Specific Languages in the next few years. With the arrival of Language Workbenches and IDE plugin capabilities, we will probably see that many frameworks will start using a custom syntax for storing the language artifacts on disk.

    regards, tom.
  11. A state machine can never be 'somewere halfway the transition'.
    Isn't that a state in it's own right, which begs to be named properly?
    Real life processes can have multiple concurrent paths of execution, whereas a traditional state machine only has one.
    Isn't that like having two state machines executing at the same time. In the example given there, the billing state machine enters the state of "expect shipping" and when the shipping state machine enters the "shipped" state the billing state machine updates accounts and enters the "finished" state?

    Although I don't actually believe that's a real life example - in real life the billing and accounts are normally complete, and then you have to deal with the shipping company separately, and if something goes wrong and they fail to deliver, you just get a refund.

    Moreover, you are later free to ask for refund, which does not trigger a change in the state of any of the two original machines, but initiates a new process. During all that time there has been and there still is a persistent state.

    I would not hold Tom responsible for not getting a payrise, it's like holding one of the four-thousand-pound-a-week-if-you-buy-my-book guys responsible for not gettin 4k a week. Except he's not even selling.

    I won't be surprised though if someone actually manages to get a rise by selling this to his/her manager.
  12. I know that we are moving up on the abstraction level and that could be a positive thing, but i am a little frustrated over the maturity of the product from IBM. It is simply not ready for production yet. To many errors in the product, it takes to much time to find out why the tool is telling you that something is wrong.

    I my view, a business employee can not use theese tools, to much techical insight is needed. A developer is more concerned with coding rather than drag'n drop. But i guess that's just my opinion.

    Besides subflows, Human interaction is not something that is NOT address in the current BPEL. I think people looking at theese things need to know that every BPM vendor has its own specification of human interaction and subflows.

    Now i am on a project using Oracle's BPM(Collaxa) and that seems to be closer, but only future can tell me.

    Redards, Frank

    Frank Bengtsson
    Lenio ApS
    http://www.lenio.dk
  13. I know that we are moving up on the abstraction level and that could be a positive thing, but i am a little frustrated over the maturity of the product from IBM. It is simply not ready for production yet.
    You can always have a look at JBoss jBPM
    I my view, a business employee can not use theese tools

    We at jBPM take a slightly different (and IMHO more realistic) approach to the relation of business analyst and developer:

    We don't think that it is just moving up on the abstraction level. We especially, don't think that business analysts will be able to specify production ready software. It always takes the involvement of a technical developer to this.

    The process language is a very powerfull tool to create a common language between business analyst and developer. The process diagram is what they both understand and can talk about. They both work on the same process. The business analyst will only see the graphical part. While the developer can see and manipulate the technical details. The nice thing about this approach is that they are always in sync. Unlike UML activity diagrams e.g. that have to be kept in sync manually. It's even more then in sync: It's the software artifact itself that the business analyst is looking at.

    So the traditional idea of business analyst drawing a process and deploying it onto a workflow engine is not realistic, imho.

    Our graphical designer supports the notion that a business analyst and a technical person are working on the same software artifact. The graphical diagram serves as the common language.

    regards, tom.
  14. We at jBPM take a slightly different (and IMHO more realistic) approach to the relation of business analyst and developer:

    We don't think that it is just moving up on the abstraction level. We especially, don't think that business analysts will be able to specify production ready software. It always takes the involvement of a technical developer to this.
    I've been delivering Now BEA's AquaLocic BPM) solutions for several years now, and I agree with your statement. Except that the increased level of abstraction is not to be ignored: With FuegoBPM I've seen many not-so-technical people implementing pretty solid prototypes without much help from hard-core developers.
    The process language is a very powerfull tool to create a common language between business analyst and developer. The process diagram is what they both understand and can talk about. They both work on the same process. The business analyst will only see the graphical part. While the developer can see and manipulate the technical details. The nice thing about this approach is that they are always in sync. Unlike UML activity diagrams e.g. that have to be kept in sync manually. It's even more then in sync: It's the software artifact itself that the business analyst is looking at.
    Agreed again. This is one of the big advantages of implementing business processes with a BPM tool: minimizing the Business vs. IT divide. Fernando
  15. I know that we are moving up on the abstraction level and that could be a positive thing, but i am a little frustrated over the maturity of the product from IBM. It is simply not ready for production yet.
    You can always have a look at JBoss jBPM
    I my view, a business employee can not use theese tools

    We at jBPM take a slightly different (and IMHO more realistic) approach to the relation of business analyst and developer:

    We don't think that it is just moving up on the abstraction level. We especially, don't think that business analysts will be able to specify production ready software. It always takes the involvement of a technical developer to this.

    The process language is a very powerfull tool to create a common language between business analyst and developer. The process diagram is what they both understand and can talk about. They both work on the same process. The business analyst will only see the graphical part. While the developer can see and manipulate the technical details. The nice thing about this approach is that they are always in sync. Unlike UML activity diagrams e.g. that have to be kept in sync manually. It's even more then in sync: It's the software artifact itself that the business analyst is looking at.

    So the traditional idea of business analyst drawing a process and deploying it onto a workflow engine is not realistic, imho.

    Our graphical designer supports the notion that a business analyst and a technical person are working on the same software artifact. The graphical diagram serves as the common language.

    regards, tom.
    The problem here is that you are quickly needing "custom" graphical elements for the graph-oriented-programming, which all are represented by the same graphical representation. You have to move into the technical specs, even as a business analist. If you use BPEL, you end up with the same "graphical" notations, but with a lot of "code-behind" (whether these are webservices, or custom made embedded actions/events).
  16. I think the article is useful in that it does highlight the fact that there is a distinction between a business transaction and resource transaction. I am sure that most developers that visit this website fully understand the concepts and the differences though they would agree that today most automated and manual application testing does not verify that a particular workflow process has been correctly and efficiently mapped to a sequence (or graph) of component interactions and resource transactions.

    Whenever I get the chance to visit and talk with a test team I am always curious how they perform transaction semantic testing. When performing a particular use case do they just compare the state of the database(s) at the end of the business transaction or do they investigate the potential many transient states across multiple requests and resource transactions in completing the test. The answer is usually no because testers (1) have insufficient documentation mapping workflow-to-tx behavior, (2) lack deep knowledge of the issues associated with transaction chopping and concurrency, (3) assume the architect and development teams have done this, (4) not aware that there are tools that can help with providing insight.

    I did create a useful feature in JXInsight to help encourage testers to perform such testing but I have probably not done such a good job in this article in showing it's application to the problem.
    http://www.jinspired.com/products/jxinsight/usertesting.html

    Today we have many articles, magazines, and conferences devoted to security and data protection but yet most enterprise applications and processes could potentially have a higher likelihood of corrupting operational data via incorrect workflow-to-tx mappings than some external attack.


    regards,

    William Louth
    JXInsight Product Architect
    CTO, JInspired

    "J*EE tuning, testing, tracing, and monitoring with Insight"
    http://www.jinspired.com
  17. William,

    For integration testing, this is a cool feature !

    Indeed, testing workflow and BPM processes is typically a nightmare.

    In jBPM, we have the notion of TDD for processes that allows you to create plain JUnit tests for your process. Each test provides the external triggers and takes process through one execution scenario. In that case the jBPM runs in-process. Even without persistence if that would be desired. We believe that unit testing a business process will be a crucial feature in the further adoption of workflow and BPM.

    regards, tom.
  18. Thanks Tom,

    What is really nice about this feature is that it is possible to inject the trace into a client process as well. By injecting it into a client process (Swing Java app) and using our distributed tracing support for JBoss RMI, BEA WLS RMI, CORBA, JMS, HP OpenView ObjectServer, and Visibroker one can aggregate all resource consumption and transaction patterns under this injected trace and its call trace stack. We have seen cluster snapshots from customers that include distributed traces from SwingApp->[CORBA]->Sun AppServer->[RMI]->JBossAS->[ITP]-HP OV ServiceDesk. With the injection mechanism it was easy to record the resource consumption (memory, cpu, thread monitors) and execution patterns (traces and transactions) across the 5 JVM's involved in a single button click on a form.

    It would be great if this kind of functionality could easily be integrated in JBoss jBPM so that we could verify the transactions semantics of a given workflow as well as connecting this to resource metering - great for developers, architects, testers and operations. This would ease of the pain for many groups involved.

    Some additional links:
    http://www.jinspired.com/products/jxinsight/jbosstracing.html
    http://www.jinspired.com/products/jxinsight/corbatracing.html
    http://www.jinspired.com/products/jxinsight/coherencework.html
    http://www.jinspired.com/products/jxinsight/swingtracing.html


    Best regards,

    William Louth
    JXInsight Product Architect
    JInspired

    "J*EE tuning, testing, tracing and monitoring with Insight"
    http://www.jinspired.com
  19. What unit are you trying to test here? Or is it rather an integration test you try to accomplish?
  20. I have seen lots of workflow systems, jBPM is great for java developers, WS-BPEL is good for web service orchestration in SOA but it is still not workflow, although BPEL for People is a great workflow solution it is still a proposal and not implemented yet, WFMC has done great work on interfaces, YAWL and Van Der Aalst's work on workflow patterns are great. All this solutions are great but each in one aspect of the workflow! None of them are as good and as complete as they should be! I see some thing very dangerous here: Lots of business logic of a system will be in the workflow definitions, the different approaches and lack of accepted standard is harming developers. Most of these products (including jBPM) are forcing workflow designers (which are usually not developers) to do some programming and hard coding work to get their simple business process to run, isn’t really a way to avoid this? I am happy that BPMN is merging into OMG, May be it is time to get all these people around one table and make an accepted standard of them. Which is a complete correct solution for end users?
  21. jBPM is great for java developers
    thanks :-)
    All this solutions are great but each in one aspect of the workflow! None of them are as good and as complete as they should be!
    yes. We at JBoss jBPM believe there are multiple process languages with a 'raison d'être'. BPEL will be the standard language for service orchestration. I hope jPDL will be the de facto standard for workflow in Java. And there will be a few (but not many) more. That is why we put so much emphasis on Graph Oriented Programming. It is a single foundation that serves for all of these process languages. Once you get the simple ideas of Graph Oriented Programming you will understand a new workflow language that is based on it in no time. If your collegue knows the relational model and ACID properties, it's easier to explaining SQL and transactions. Without this knowledge of relational model. You have a problem. It's similar with process languages and Graph Oriented Programming. regards, tom.