Discussions

News: Hammurapi Rules supports JSR-94

  1. Hammurapi Rules supports JSR-94 (23 messages)

    Hammurapi Group has announced the latest release of its rules engine product, Hammurapi Rules. This release adds support for JSR-94. Hammurapi Rules is an open source (LGPL) rules engine, with Java being the primary language for rules authoring. Featured highlights include:
    • Hammurapi Rules leverages Java type systems and several naming conventions to build its Rete network, allowing Java developers to quickly get familiar with rules authoring
      • Debug rules in your Java IDE
      • Compile time checks vs. runtime exceptions
      • Rules can natively access underlying Java application
    • Flexibility
      • Rule sets are defined in XML
      • Rules can be parameterized at assembly time and registration time
      • Specialized rules languages can be added if needed
    • Scalability
      • Hammurapi rules supports multithreaded inference
      • Working memory can be stored to disk or to database and as such survive JVM shutdown
      • Inference can be distributed across multiple machines
    • Rules are assembled into rule sets and parameterized at runtime by means of XML - Hammurapi Rules can load rule sets from different sources including file, url and classloader resources
    • Negations support
    • Logical loops detection
    • Autodetection of updates
    • Rules engine builds derivation trees to help in debugging rules logic - derivation trees can be dumped to XML or visualized in Swing GUI
    Access Hammurapi Rules' user guide and online documentation. What do you think of Hammurapi as a rules engine?

    Threaded Messages (23)

  2. It's not RETE or even forward chaining[ Go to top ]

    I looked at Hammurapi last week and I have to say it's not forward chaining or RETE. Just look at the javadoc for the Rule object and the sample rule. http://www.hammurapi.biz/products/hammurapirules/doc/api/index.html http://wiki.hammurapi.biz/index.php?title=Hammurapi_rules#Writing_rules One critical and defining feature of rule chaining is that the engine doesn't iterate over the rules and pass the facts to each rule. Instead, the facts are asserted to the working memory and the matches are fired. Passing objects to a rule is a procedural approach. What hammurapi is doing is using memories to improve performance. Basically, by remember what facts a rule has evaluated, the engine can quickly look at what changed and re-evaluate. Rule chaining means chaining the conditions, not chaining the actions of the rules as Hammurapi's document states. Hammurapi also confuses distributed rule engine with service coreography. Telling a rule engine some rules are on another system doesn't make it distributed. The approach has been shown to not scale well as dataset increases. Hammurapi should correct its documentation and retract the claim it implements forward chaining. There's plenty of open source rule engines out there which show how chaining is implemented. Drools3 documentation has a decent explanation of how it's done for those who want to learn more. I should state that I helped write the RETE explanation in drools3 docs, so I'm totally bias. peter
  3. JSR-94 specification, section 6.1, the first paragraph on page 6:
    One of the most common classes of rule engines is the forward-chaining rule engine. Forward-chaining rule engines implement an execution cycle that allows the action of one rule to cause the condition of other rules to become met. In this way, a cascade of rules may become activated and each rule action executed. Forward-chaining rule engines are suitable for problems that require drawing higher-level conclusions from simple input facts.
    In Hammurapi rules an action of one rule, posting of a conclusion to the object bus, triggers other rules which wait for a conclusion of this type. Higher-level conclusions are drawn from lower-level conclusions. You can run tutorial and browse derivation tree to see in detail how higher-level conclusions are drawn from lower-level ones. If we take this definition
    A Rete-based expert system builds a network of nodes, where each node (except the root) corresponds to a pattern occurring in the left-hand-side of a rule. The path from the root node to a leaf node defines a complete rule left-hand-side. Each node has a memory of facts which satisfy that pattern.
    from the Wikipeida article on the Rete algorithm then Hammurapi rule also complies. Object's class is a pattern, invocation handlers are nodes. Each node has a memory of facts, stored in the collection manager, which satisfy that pattern. The important thing to understand is that patterns/conditions are also objects as well as facts. --- Regards, Pavel.
  4. Actually, the description of RETE on wikipedia is bad and inaccurate. I have a detailed explanation of why the description of RETE on wikipedia is wrong http://woolfel.blogspot.com/2006/10/wikipedia-has-been-updated.html. I would strongly suggest Forgy's RETE paper and doorenbos RETE-UL paper. I took a look at the Rule class and it most definitely is not forward chaining. if you want to see how chaining really works, look at JBossRules (aka drools3), CLIPS or Sumatra (http://www.sourceforge.net/projects/ruleml-dev). I'm the author of sumatra, which is my port of CLIPS. Having spent the last 6 years obsessed with pattern matching algorithms and implementing rule engines, the implementation I see does not meet the definition of forward chaining. I could be wrong, and misread the code. Forward chaining means the facts are evaluated from the object perspective. By that I mean, when an object is added for evaluation, it matches all the patterns for that object. Hammurapi does not appear to be doing that. If it isn't doing that, it is most definitely not doing forward chaining. peter
  5. Peter, I expected you to challenge the definitions. You probably have the right to do so, taking into account your experience with rule engines. If we take a laymen definition of the Rete algorithm - there is a network, objects (facts, conclusions, conditions) are not presented to all rules, but only to those which are interested in them, i.e. are in appropriate nodes in the network - then Hammurapi rules fits. I wouldn't be surprised if Drools isn't RETE according to some of definitions of the Rete algorithm in the Internet ;-) Regarding forward chaining, the Rule class doesn't do chaining. The chaining logic resides in Dispatcher and its subclass QueuingDispatcher. I stick with the definition of forward chaining found in the JSR-94 specification. --- Regards, Pavel.
  6. Peter, I expected you to challenge the definitions. You probably have the right to do so, taking into account your experience with rule engines. If we take a laymen definition of the Rete algorithm - there is a network, objects (facts, conclusions, conditions) are not presented to all rules, but only to those which are interested in them, i.e. are in appropriate nodes in the network - then Hammurapi rules fits. I wouldn't be surprised if Drools isn't RETE according to some of definitions of the Rete algorithm in the Internet ;-) Regarding forward chaining, the Rule class doesn't do chaining. The chaining logic resides in Dispatcher and its subclass QueuingDispatcher. I stick with the definition of forward chaining found in the JSR-94 specification. --- Regards, Pavel.
    right, laymen definitions of RETE could mean anything, which is why i say it's wrong. At the end of the day, what matters is whether or not the rule engine scales like RETE. By scale, i mean as the number of rules, object types and facts increase, does it grind to a halt or perform well like OPS5. The defintion of forward chaining in the JSR94 like wise is bad and frankly worthless. I'm bias against JSR94 and think it's kinda pointless. It's funny you say someone might think drools isn't RETE. A few years back I took time to show why drools2 wasn't RETE. In 2005 I donated a clean implementation of RETE to drools, which influenced drools3. You'll see on my blog that drools3 scales as Dr. Forgy's RETE describes. It's performance is comparable to some commercial RETE engines. In my opinion Drools3 does implement RETE plus additional runtime optimizations. Sumatra is still in alpha, but the current code base is comparable to commercial RETE engines. You shouldn't take my word for it. Look at CLIPS codebase to see for yourself. Or look at the description ernest has for JESS http://www.jessrules.com/jess/docs/61/rete.html. if you want to test the scalability of hammurapi, I have samples in CVS and a description of rule benchmark suite here (http://woolfel.blogspot.com/2006/08/benchmark-suite-take-4.html). I was a bit harsh in my criticism, but RETE most of the explanations of RETE are poor or wrong. i would encourage you to read the RETE explanation on JBossRules website http://labs.jboss.com/file-access/default/members/jbossrules/freezone/docs/3.0.4/html_single/index.html. Peter lin
  7. oops, i should proof read[ Go to top ]

    meant to say I was a bit harsh in my criticism, but most of the explanations of RETE are poor or wrong. peter
  8. Drools is not Rete????[ Go to top ]

    I'm afraid to disappoint you but Drools is very much Rete, infact its ReteOO :) an enhanced version of Rete. I've looked over your code. What you have written is an unscalable forward chaining pojo scripting framework with descrimination at the ObjectType level. That is the simplest part of Rete, there is much more. I wrote a similar engine in about 20 minutes to demonstrate forward chaining and cross products - I could make that JSR94 compliant in a few hours. This doesn't show the strength of the code, just the shallowness of the spec. Forward chaining does not a Rete Network make :) You'll need to implement alpha nodes and beta nodes and have chaining of those beta nodes, along with alpha and beta memory. You'll also need an Agenda for conflict resolution, a truth maintenance system and make sure that you can handle first order logic as well as propositional logic. Your current framework will make it impossible to handle first order logic in an scalable or meaningful manner. Drools 2, which wasn't full rete, attempted a Pojo approach - it seems you are making many of the same mistakes - you'll find this approach very limiting. It's nice to see the lengths you have gone to document what you have done, you've obviously invested a lot of time and have plenty of energy. Don't be discouraged, its an extremely difficult tech domain to get right, companies have invested millions in writing engines that ammount to a pile of dog turd. You can read my documentation openining paragraph to try and understand the different between a Production Rules System and a scripting framework: http://labs.jboss.com/file-access/default/members/jbossrules/freezone/docs/3.0.4/html_single/index.html#d0e45 It's important we educate the public on the difference otherwise we are in danger of a backlash from the deluge of products like hummurapi rules. Although I'm currently re-thinking the Turing Complete part, the definition is too weak to have real bite. Why not get involved in the Drools effort and learn about real production rules sytems - it's fascinating stuff and you'll learn loads from our team. You're welcome to pop over to irc.freenode.net #drools anytime and join in :) Peter is often there too. Mark
  9. I've taken the liberty of redefining what inference and chaining means, since many of the definitions out there aren't useful. In fact many definitions make no distinction between inference and chaining. http://woolfel.blogspot.com/2006/10/inference-vs-chaining.html Hopefully, this will help clarify what inference and chaining mean in the current state of art. A quick google revealed there's lots of old definitions for both terms that date back 3-4 decades when procedural inference engines were the norm. Given that RETE changed the AI world in 1979, it's time to update the terms so they reflect changes that have occurred the last 2 decades. peter
  10. What changed in the last two decades[ Go to top ]

    Peter, Probably it's a bit off the topic, and I know I'm opening a whole new can of worms, but ... nonetheless ... because I enjoy our discussion, do you know what changed the most in the last two decades? The software dollar vs. hardware dollar ratio. Nowadays hardware is fast and cheap, but good professionals are expensive and scarce. Java won the world not because it is faster or more featureful than C++, it is not even more portable. Java is SIMPLE and, as a corollary, more outsourceable and offshoreable. The Occam's razor principle should substantiate my point. Those old days smart folks spent years in ivory towers squeezing performance out of slow machines by polishing algorithms. Nowadays you get offshore contractors and a couple of weeks of time to implements rules in your application. Some time ago I participated in a project with 5 offshore folks and 20 simple rules to implement. Those folks knew Java and servlet/JSP programming. We really wanted to use a rule engine and did a little looking around but ended up hard-coding those rules because rule engine's specialized languages were too cryptic for regular Java guys. Maybe it's time to recognize that scalability and performance are not the only qualities and there are such qualities as usability, maintainability, time-to-market? And that there can be a spectrum of solutions of the same problem with different qualities. Different people would choose the mix of qualities that is appropriate for them depending on the problem at hand, skillset, time constraints, ...? And that JSR-94 allows to change mind down the road without having to rewrite the whole application, but only rules (though it still can be a daunting task)? Remember, every good quality is noxious if unmixed. --- Regards, Pavel.
  11. Peter, Probably it's a bit off the topic, and I know I'm opening a whole new can of worms, but ... nonetheless ... because I enjoy our discussion, do you know what changed the most in the last two decades? The software dollar vs. hardware dollar ratio. Nowadays hardware is fast and cheap, but good professionals are expensive and scarce. Java won the world not because it is faster or more featureful than C++, it is not even more portable. Java is SIMPLE and, as a corollary, more outsourceable and offshoreable. The Occam's razor principle should substantiate my point. Those old days smart folks spent years in ivory towers squeezing performance out of slow machines by polishing algorithms. Nowadays you get offshore contractors and a couple of weeks of time to implements rules in your application. Some time ago I participated in a project with 5 offshore folks and 20 simple rules to implement. Those folks knew Java and servlet/JSP programming. We really wanted to use a rule engine and did a little looking around but ended up hard-coding those rules because rule engine's specialized languages were too cryptic for regular Java guys. Maybe it's time to recognize that scalability and performance are not the only qualities and there are such qualities as usability, maintainability, time-to-market? And that there can be a spectrum of solutions of the same problem with different qualities. Different people would choose the mix of qualities that is appropriate for them depending on the problem at hand, skillset, time constraints, ...? And that JSR-94 allows to change mind down the road without having to rewrite the whole application, but only rules (though it still can be a daunting task)? Remember, every good quality is noxious if unmixed. --- Regards, Pavel.
    Pavel, Good questions. I'll do my best to answer. If someone wants to write a dozen rules, honestly it's easier to just hard code it. Would I do it? It depends. If I am working with entry level engineers, who just want to write rules in java, it makes complete sense to just use a simple framework. It would be silly and a waste of time to force the developers to learn S-Expression syntax and write rules in CLIPS. In contrast, if I'm writing business rules to manage mortgage lending and the rules are complex, it would be crazy to use a simple POJO framework. Simply put, it won't scale and I'd probably get fired when the system grinds to a halt. I definitely don't recommend everyone use a full blown RETE rule engine for every single situation. My primary concern is when rule engines claim to implement RETE when it doesn't. That ends up confusing developers and gives RETE a bad name. Google for drools2 and you'll see some people complained it had difficulty scaling beyond 100 rules. The performance issue wasn't RETE's fault. It was drools2 implementation. Now that drools3 is RETE, those problems have disappeared and performance is good. Engines like Jena2 and pychinko like wise claim RETE but they do not scale like RETE. This gives semantic web people a feeling that RETE doesn't scale. It's not RETE fault Jena2 and pychinko aren't RETE. It's a lack of integrity and due diligence on the part of pychinko and jena2 developers. I agree that hardware is getting cheaper and good engineers are hard to find. But that's a strawman argument. If a system has to handle pre-trade compliance for real-time trading systems, a simple engine won't be able to respond fast enough. Would a simple POJO approach handle 1000 transactions per second 24x7? The deciding factor should be based on the business requirements. If there's going to be 4,000 rules, what kind of engine would you use? I would use a mature RETE engine. I would encourage you to read Dr. Forgy's paper. This domain is deep and vast. I've been at this for almost 6 years and i'm only scratching the surface. Implementing RETE is very hard, but it's a great way to sharpen one's skills. Once you go through the process of understanding the algorithm and implementing it, you'll have gone deep into compiler theory, pattern matching, relational theory, and language design. I consider the experience invaluable. peter
  12. Peter, Thanks for the answer. Now I understand that you care very much about the good name of RETE. Let's do comparative manners benchmarking, it will give ground for further discussion. --- Regards, Pavel.
  13. A bunch of quoting and thoughts[ Go to top ]

    Mark,
    I'm afraid to disappoint you but Drools is very much Rete, infact its ReteOO :) an enhanced version of Rete
    I didn't say that Drools is not Rete. I said that there are so many definitions of Rete in the Net that you can surely find one which Drools doesn't comply with. Similarly, I found one on Wikipedia, that Hammurapi rules complies with. If it is not the definition you like, so be it. I could say that Hammurapi rules implements so much enhanced Rete that alpha and beta nodes are not required anymore; but I wouldn't :-) Most people don't care about alpha and beta nodes. They need a tool which does the job. I mentioned "the mix of qualities" in my previous post. We'll see how benchmarking is going if Peter agrees with the proposal.
    I wrote a similar engine in about 20 minutes to demonstrate forward chaining ...
    At least (or at last?) you agree with forward chaining!!!
    It's important we educate the public on the difference otherwise we are in danger of a backlash from the deluge of products like hummurapi rules.
    I'm sure C++ pundits wrote something like this about Java a decade ago ;-)
    It's nice to see the lengths you have gone to document what you have done, you've obviously invested a lot of time and have plenty of energy. Don't be discouraged, its an extremely difficult tech domain to get right
    We need to define what is "right". For me a significant part of the "right" is that a regular Java guy can write meaningful rules in an hour or so. This is, by the way, why I "have gone the lengths" to document the product. Another significant part of the "right" is compliance with standards.
    companies have invested millions in writing engines ...
    After participating in a rule engine selection process, I'd say that millions are invested in the salesforce, PowerPoint presentations, nice-looking GUI and support of the myth that business people can write rules without developers involvement. I also got an impression that things are intentionally kept too complicated so companies can sell training and professional services. --- Regards, Pavel.
  14. Rete Definition[ Go to top ]

    There is one established definition of Rete - Charles Forgey's paper. There is another definitive document on Rete enhancements the Rete/UL paper. Surely your bright enough to not accept everything you read on the web as truth? You'll find someone saying just about everything, you can't then quote that as a justifcation for your comment without looking a bit silly. It's not just about scalablity, but also engine capabilities and expressiveness - this goes hand in hand with declarative programming. Your developers and business analysts will be able to do far more with a powerful declarative platform than a limited and verbose projo scripting framework. Trust me, I've been there with early Drools work. Where is your first order logic support support? Your Truth Maintenance System? How do you handle Conflict Resolution? Where is your data indexing to provide performance? If all you want is a better chain of command type framework, then thats fine. you can see why the word "rule engine" is bad, we need to differentiate between limited hammarupi rules scripting frameworks and proper production rule systems. You'll be surprised that even with todays hardward with the sheer number of rules and facts that some companies deal with it will gring to a halt if not implemented properly. We aren't just talking 2 or 3 times faster, we are talking the difference between running in a 10 seconds or hours.
  15. Wikipedia updated...again[ Go to top ]

    As the person responsible for writing most of the Wikipedia Rete article, I'd just like to register that I have posted a substantial re-write at http://en.wikipedia.org/wiki/Rete_algorithm. While I didn't accept all of Peter's original criticisms (you can see our discussions in the feedback to the article he mentions on his blog site at http://woolfel.blogspot.com/2006/10/wikipedia-has-been-updated.html), I certainly agreed that it could be both clearer and more comprehensive. Hopefully the new version of the article provides a better description. I plan, at some point, to add some further diagrams, and I (or someone) also needs to go through and add links to other articles, etc.
  16. There's an easy way to prove Hammurapi implements RETE. Simply run manners benchmark and post the full details for the world to see. There's examples of manners benchmark in drools subversion and sumatra CVS. good luck. peter
  17. Manners benchmark[ Go to top ]

    Peter, You know, it is a good idea. I can implement a benchmarking harness similar to JSR-94 TCK so any JSR-94 compliant rule engine can be benchmarked. Input facts will be read from manners_XXX.dat files from Drools examples. I will try to reuse the model classes AS-IS. But before I invest my time into this exercise we should come to an agreement that: - manners_XXX.dat files are not bad and pointless. - Once the harness is released, you will prepare a Drools benchmark with the harness and will make it freely available so everybody can download it and run by invoking a command file without having to go through reams of documentation and multi-step installation. --- Regards, Pavel.
  18. Manners[ Go to top ]

    You won't be able to implement manners in any meaningful sense as you have no support for first order logic. We made this mistsake in drools 2, so we had a lighttning fast manners test - but basically we were cheating and missing the point of manners, as we did not have a 'not' conditional element.
  19. Re: Manners[ Go to top ]

    Peter, So basically you are saying that manners benchmark is useless because it can be cheated? My understanding of benchmarking process is that it is a black-box testing with time measurements. Steps: - you give input to a component and start timer - the component processes the input and produces output - you stop timer and validate output. If the output is valid then the benchmark is a success and you publish results. If the component is "cheating", it means that your input is not goog enough and you should provide better input so the component cannot cheat. Users of the component care about observable results, i.e. component's output for a given input. If the component produces valid results with all inputs then it means it solves the problem at hand and the internal structure of the component is irrelevant. --- Regards, Pavel.
  20. Re: Manners[ Go to top ]

    Peter, So basically you are saying that manners benchmark is useless because it can be cheated? My understanding of benchmarking process is that it is a black-box testing with time measurements. Steps: - you give input to a component and start timer - the component processes the input and produces output - you stop timer and validate output. If the output is valid then the benchmark is a success and you publish results. If the component is "cheating", it means that your input is not goog enough and you should provide better input so the component cannot cheat. Users of the component care about observable results, i.e. component's output for a given input. If the component produces valid results with all inputs then it means it solves the problem at hand and the internal structure of the component is irrelevant. --- Regards, Pavel.
    Good question. Manners benchmark is a stress test, which attempts to put the rule engine under extreme load. It isn't a benchmark in the sense of a micro-benchmark. Because Manners is a stress test, one can short circuit manners or not perform full cross product. Full cross product in this case means comparing the facts from both sides of a join and propogating all partial matches down the network. All complete matches are then added to the agenda. One easy way to short circuit is avoiding full cross product. This is basically what LEAPS backward chaining algorithm does. In the case of LEAPS, it avoids doing a cross product on all joins, and it has no agenda. Since LEAPS algorithm doesn't do cross product, it isn't appropriate for cases where the application needs to find all matches before deciding which match is best. Having said that, short circuiting the runtime isn't really cheating. It's just a form of optimization, which works in certain cases. The downside is when companies claim to beat other engines, but fail to mention the runtime isn't calculating the full cross product. The important distinction here is disclosure. When a company says they are faster on manners because the runtime is running in LEAPS mode, but fails to tell users, it's deceptive marketing. On the other hand, if a company reports their results and clearly explains the additional optimizations, users can make well informed decisions. For example, I implemented no-agenda for Sumatra as a way to tell the engine, fire the rule immediately. On my blog, I attempt to explain the pros/cons of doing that and when an user might consider using it. Hope that answers your question. For a more detailed explanation, I would suggest reading Drools3 documentation on jboss website. I also have several dozen entries on my blog that explain the low level bits of how it works under the covers. peter
  21. Peter, Please note that "Hammurapi" is a code review tool. "Hammurapi rules" is a ... JSR-94 compliant piece of software. --- Regards, Pavel.
  22. Peter, Hammurapi rules fits your definition of RETE algorithm. Once I get hold of the original Dr Forgy's paper I'll add a section to the Hammurapi rules online documentation and to the Hammurapi Rules User Guide describing how RETE algorithm is implemented. --- Regards, Pavel.
  23. you can find copies of manners in drools SVN, or you can find it on the university of texas ftp site. peter
  24. I have added information on RETE to Hammurapi rules' online documentation and user guide. In a nutshell, the assertion part of Hammurapi is RETE "as the doctor (Forgy) ordered". The retraction part is not RETE for a reason, see the docs for more details. Many thanks to Peter Lin for helping to get to the source.