Velocity: A template engine, Rule engine, or both?

Discussions

News: Velocity: A template engine, Rule engine, or both?

  1. Velocity was a pioneering template engine for Java, but it turns out that it is also a great Rule engine. Amit Kumar writes about how he used a combination of properties, toolbox and parameters to implement a powerful rule engine for an entitlement application using Velocity.

    Read the article.

    Threaded Messages (80)

  2. I admit that Velocity is indeed a great thing
    But I always used it for what it is supposed to be used, namely, as a template engine: be it user interfaces, unit test code or other template-based automatically generated chunks of texts.

    If, as I've understood it, the idea of using Velocity as a rule engine was to decouple rules from the code, so changing rule != changing code, than I would suggest to look at BSF and plug-in any scripting language that you love most, e.g. Rhino(JS) or Jython. Then you'll get much more horsepower just for the same efforts.
  3. Rule engine?[ Go to top ]

    While I'm very pleased that rule engines are getting threads on TSS - the definition is certainly a little slippery!

    There are a few things missing from this approach that I like to see in a rules based approach:

    - "What, not How" -- that is, a declarative approach that distinguishes between what needs to be done, versus how it is done (method calls etc.)

    - an "engine" of some sort -- the rule engine should be doing some work to schedule, orchestrate and fire the rules. Often this involves some form of inferencing.

    - (optional) non-developers editing rules -- tooling that not only moves the rules outside of the Java code development lifecyle, but enables non programmers to interact with the rules.

    - (optional) a rules editing lifecycle, distinct from the code -- can you edit your underlying Java code without invalidating all your rules, for example?

    Perhaps the author could also take a look at the OSS rule engines, such as Drools? There are several dozen rule engines listed on http://www.javarules.org (see the web links).
  4. I am not that new “guy in the town” in application security and I also have worked on entitlement systems. Rules engine is a hot stuff there. Somebody event mentioned that Rules engines is a more mature and popular technology than RBAC.

    Maybe. However, in my experience, Rules engines I am aware of are one of the most dangerous things in security. Why ? Because Rules engines are supposed to implement business rules in the most of cases. Rules are ambiguous and Business Analysts construct rules for their tasks without concerns about other tasks. They expect the Engine controls what it has to control. That is, access to the resources.

    Have you seen such Rules Engine that controls rules conflicts ? I mean, if a rule says that a user may not access the resource, other sets of rules or its combinations may grant this access, i.e. the rule based on such combination gets into the conflict with the first rule. I have not seen such rules engine.

    Please, tell me that I am wrong and point me to the one secured Rules Engine.

     Thank you,
    - Michael
  5. not rule engine's job[ Go to top ]

    I am not that new “guy in the town” in application security and I also have worked on entitlement systems. Rules engine is a hot stuff there. Somebody event mentioned that Rules engines is a more mature and popular technology than RBAC.Maybe. However, in my experience, Rules engines I am aware of are one of the most dangerous things in security. Why ? Because Rules engines are supposed to implement business rules in the most of cases. Rules are ambiguous and Business Analysts construct rules for their tasks without concerns about other tasks. They expect the Engine controls what it has to control. That is, access to the resources. Have you seen such Rules Engine that controls rules conflicts ? I mean, if a rule says that a user may not access the resource, other sets of rules or its combinations may grant this access, i.e. the rule based on such combination gets into the conflict with the first rule. I have not seen such rules engine. Please, tell me that I am wrong and point me to the one secured Rules Engine. 
    Thank you,- Michael

    I have to say it is very easy to screw things up with a rule engine. by screw up I mean this. Analyst Joe knows his domain and is given a nice natural language rule editor. Joe decides to make a change and put it directly into production. Joe by-passes validation, QA, and staging. The end result is the rules in production may have contradictions, or worse yet incorrectly allow users to do things they are not suppose to.

    These things do happen, but to me they shouldn't be the rule engine's problem. A rule engine is designed to run efficiently and provide flexibility. If the user stupidly does things without thinking it through, it is user error. The primary difference I see is this. When the business logic is hardcoded in C/C++, C#, Java or the language of choice, people expect to go through a formal deployment process. The same isn't true of rule based applications. This is largely due to the sales pitch. Many sales people will pitch the solution as "something rule analysts can use without a programmer." When the customer asks, "so that means I can change a rule and deploy it immediately."

    It's not uncommon for a sales guy to answer "yes" just to close the deal. the end result is the customer gets a false sense of how a rule application would work in practice. Using a rule based approach can have rapid turn around, but it doesn't mean by-passing QA, validation and staging. The benefit of using rules is you can handle part of that using automated process. It's trivial to generate rules that will trigger the new rule and see what happens with the full set of production rules. Beyond that, many errors can be caught using decision tables and decision trees. It's not possible to catch all potential conflicts in the most complex cases.

    the type of cases where automated validation will fail is rules with multiple "or" disjunctions that create indeterminant state or infinite loop. These classes of problems are solvable, but that discussion is rather long and depends on the specific situation.

    peter
  6. doh error[ Go to top ]

    It's trivial to generate rules that will trigger the new rule and see what happens with the full set of production rules.

    what I meant to say is, it's trivial to generate facts (ie data) that will trigger the new rule and make sure it reaches the desired result.

    peter
  7. not rule engine's job[ Go to top ]

    Authentication and authorization are not a tasks for a rule engine -- I agree. Surprisingly people usually think (and I did too) that rule engine is something like server, web server, database, j2ee container, COM/DCOM etc. with associated security realm or some kind of authentication infrastructure with groups, roles, credentials. It's not true. In case of ILOG JRules, the rule engine is just a java object. It is instantiated and called within the context of the executing method and inherits credentials of the caller of that method. Rule engine is POJO, plain old java obect. If you call rule engine from let's say EJB bean -- use J2EE security.

    Having said that I think it's possible. Although I admit it's a bad design and against rule engine theory,in ILOG JRule you can pass you custom security manager and user credentials to ILOG as input parameters and use it in the condition part. I for example pass Logger (log4j) to rule engine and use it within rules: ?context.logger.info(...). Why not to pass security? But again, personally I'd consider such design as bad one. Very awkward to see authorization in the condition part of a rule... Besides, performance-wise it's bad too. Want to have different rules for different roles -- create different rule sets.

    Glen
  8. SSO[ Go to top ]

    The kinds of apps i'm used to usually have Single-Sign-On, so it really doesn't make sense to have security or authentication in the rule engine, unless you're talking about dynamic security, which is a different topic completely. Back when I worked on wireless applications, we had a lot of dynamic security/privacy related issues and scenarios.

    peter
  9. are you sure?[ Go to top ]

    Authentication and authorization are not a tasks for a rule engine -- I agree... Glen

    Rules engines and security

    I agree that rules engines may not be suited for authentication, after all it seems that most authentication is user+password = user+password. Seems static and simple, couldn’t imagine a justification for a rule engine here.

    I think that a rules engine is not ideally suited for non-repudiation security concerns.

    Authorization, however, seems to be a challenge that a rules engine fits into nicely for some situations. Follow me here into the hypothetical for a second. Let’s say the organization wants to build a virtual case file with data elements from a variety of different sources and a variety of different formats. The case file itself can belong to a user who can choose to share a variety of rights on the logical groupings of data to a number of other users, the user can delegate rights under rules defined by the enterprise, department, and workgroup level. Authorization consists of many to many challenges, within the context of a particular data set. Speed is also a concern.

    A hierarchical structure such as LDAP doesn’t seem to fit. Securing application class at method/class signatures doesn’t solve the problem either. The authorization need is based on instance of a particular data set. The data set spans the boundaries of multiple data stores, so relational DB security doesn’t seem to fit either.

    If the logical entities of the case file, the users, organization, legacy systems are all facts. Then a rule engine can provide and instance-oriented declarations about these facts including authorization (as the author calls it entitlement).

    True?
  10. are you sure?[ Go to top ]

    Rules engines and security I agree that rules engines may not be suited for authentication, after all it seems that most authentication is user+password = user+password.

    Seems static and simple, couldn't imagine a justification for a rule engine here.I think that a rules engine is not ideally suited for non-repudiation security concerns.

    Authorization, however, seems to be a challenge that a rules engine fits into nicely for some situations. Follow me here into the hypothetical for a second. Let's say the organization wants to build a virtual case file with data elements from a variety of different sources and a variety of different formats. The case file itself can belong to a user who can choose to share a variety of rights on the logical groupings of data to a number of other users, the user can delegate rights under rules defined by the enterprise, department, and workgroup level. Authorization consists of many to many challenges, within the context of a particular data set. Speed is also a concern.A hierarchical structure such as LDAP doesn’t seem to fit. Securing application class at method/class signatures doesn’t solve the problem either. The authorization need is based on instance of a particular data set. The data set spans the boundaries of multiple data stores, so relational DB security doesn’t seem to fit either.If the logical entities of the case file, the users, organization, legacy systems are all facts. Then a rule engine can provide and instance-oriented declarations about these facts including authorization (as the author calls it entitlement).
    True?

    Good points. I like your definition of authentication vs authorization. To put it another way.

    authentication == who you are
    authorization == what you are allowed to do.

    I'm not a fan of W3C, but the policy rules initiative is attempting to address the authorization aspects using rules. many of the scenarios that are being considered fit directly into the authoriztion space. In my bias thinking, one could use a rule engine to manage authorization if the policies complex and/or change rapidly. In many larger systems, a process may span multiple systems, but the user's access level changes depending on the process and system.

    In these situations, the problem is complex. Some of the scenarios I've been hearing about including dynamically add/removing rules at runtime. The challenge with that is how do you manage conflicts. Using a procedural rule engine in these cases ends up failing miserably. I'll use an example using JESS

    (defrule policy1
      (rule_metadata
         (name "policy1")
         (active true)
       )
      (session
         (active true)
         (groupid ?gr)
         (id ?id)
       )
      (group
         (id ?gr)
       )
       ;; more conditions that do something
    =>
       ;; some actions
    )

    In a procedural rule engine, how does the system make sure all current processes stop and all matches get cancelled immediately. Using a RETE rule engine, all I would have to do is turn off the rule with

    (assert (rule_metadata (name "policy1")(active false) ) )

    the rule engine will automatically remove all matches in the agenda and resolve all conflicts. For a procedural rule engine to handle these cases, it would have start the evaluation process from the beginning. Obviously, that would not scale very well and would crash in a real-time environment.

    peter
  11. How about relational graphs[ Go to top ]

    I don’t consider such a thing as a procedural engine. To me procedural and declarative are orthogonal conventions of thought.

    Peter, if I might pick your brain some more, what do you consider of this…

    I believe rule engine can provide solutions problems that manifest in relational models as well, particularly if you are designing generic meta-structures to solve a wider domain of business problems with smaller data structures. A simple example being a data model for defining human relationships. Human has many Associations with many Humans. Human entity with recursive man-to-many resolution entity. Now take for example a family tree. You could capture lineage by linking parents, children, husbands, wives, cousins etc. Problem with the relational model is that as you go deeper into this graph, you have more and more self joins, complicating the queries and degrading the performance. With a rule engine I do not care how deep the graph is. As long as the facts are there I can say, am I a decendent of?... and go back as far as the facts will let me, with no joins. I can also define other types of human to human relationships: friends, neighbors, etc. All with the same two tables.

    I find the recursive relationship quite often in business data models, but I have also found the folks that are raised on relational don’t cross the chasm. Many times data modelers seek to match entities on business terms, but then the business changes.
     
    Not so sure why rules engines seem to be bounded by “what they should NOT be used for”.
  12. Some history[ Go to top ]

    For those who haven't spent years learning and researching this stuff. The original paper on RETE by Dr. Charles Forgy was in 1978/79. I haven't read the whole thesis myself and have only skimmed a few pages. It's a rather long paper and very dense. Anyways, the original description of RETE drew on database theory. Classic RETE is considered by many in the field to be "tuple-based". Follow up papers on RETE in the 80's moved towards more object centric approach. In fact, this is mirrored in the development of CLIPS rule engine. COOL (clips object oriented language) was added around version 3, if I remember correctly. In both the 79 and 85 paper by Forgy's he calls the working memory a "database." These two fields rule engines and relational database are closely related. In fact, there are researchers that have worked on translators to convert clips rules to sql and back.
    I don't consider such a thing as a procedural engine. To me procedural and declarative are orthogonal conventions of thought.
    Peter, if I might pick your brain some more, what do you consider of this. I believe rule engine can provide solutions problems that manifest in relational models as well, particularly if you are designing generic meta-structures to solve a wider domain of business problems with smaller data structures. A simple example being a data model for defining human relationships. Human has many Associations with many Humans. Human entity with recursive man-to-many resolution entity. Now take for example a family tree. You could capture lineage by linking parents, children, husbands, wives, cousins etc. Problem with the relational model is that as you go deeper into this graph, you have more and more self joins, complicating the queries and degrading the performance. With a rule engine I do not care how deep the graph is. As long as the facts are there I can say, am I a decendent of?... and go back as far as the facts will let me, with no joins. I can also define other types of human to human relationships: friends, neighbors, etc.

    what you're describing here is the classic pattern matching problem. For example, the classic mis manners benchmark can be executed in a database, though it runs much much slower.
    All with the same two tables. I find the recursive relationship quite often in business data models, but I have also found the folks that are raised on relational don’t cross the chasm. Many times data modelers seek to match entities on business terms, but then the business changes. Not so sure why rules engines seem to be bounded by what they should NOT be used for.

    I suspect it's easier for people to think in terms of "what rule engines are not good for," because the learning curve is rather steep. Before I started using JESS, I was fascinated with game AI and tried to learn as much as possible. It wasn't until I started using RETE engines that it became apparent how much there is to learn in this field. My 4 short years of working with rule engines is just the tip of the iceberg. Someone told me once, "I want to become a rule expert." My response was, "be prepared to spend the next 20 years learning."

    peter
  13. Not new & "sales pitch"[ Go to top ]

    Well, Peter, I did not get it - did you agree with me or not ? Do you know a Rule Engine secured enough to be used for Authorization and or Entitlement ?

    I do not warry much about "sales pitch" because my own business people require me to provide a Rule Engine where a Business Analist "drives the ball". The only one way to allow such Rule Engine for Entitlemet is to provide rule compatibility and conflict resolution automatically for all new rules.

    Getting back to the article we discuss, I think the author did not realise the difference between Personalization and Entitlement. To me, the difference is in a small word - "security". If a Rule Engine is secured, I would use it in Entitlement system even if it is VELOCITY or JSP Tag Library... ILOG JRules is not secured enough to me because they have only one part - user's credentials but they do not have secured rule validation.
  14. honestly I don't know[ Go to top ]

    Well, Peter, I did not get it - did you agree with me or not ? Do you know a Rule Engine secured enough to be used for Authorization and or Entitlement ?I do not warry much about "sales pitch" because my own business people require me to provide a Rule Engine where a Business Analist "drives the ball". The only one way to allow such Rule Engine for Entitlemet is to provide rule compatibility and conflict resolution automatically for all new rules. Getting back to the article we discuss, I think the author did not realise the difference between Personalization and Entitlement. To me, the difference is in a small word - "security". If a Rule Engine is secured, I would use it in Entitlement system even if it is VELOCITY or JSP Tag Library... ILOG JRules is not secured enough to me because they have only one part - user's credentials but they do not have secured rule validation.

    if by secure rule validation you're referring to automated validation of the rules at deployment time, than my answer is it requires writing your own custom component to handle it. if the rules fit into a few templates (ie structure), than it is feasible to write a rule validation component to handle this with a high degree of certainty.

    the long answer is this. there's still an on going debate about proof vs theorem for validation. If the domain (ie topic) is sufficiently focused and narrow, both approaches have been shown to be effective. I'm not an expert in these areas, but one approach I've explored is this.

    1. use the rule to generate data which triggers the rule by itself. if the rule reaches the desired result, move to next step in validation.

    2. load all the rules including the new rule and assert facts which trigger the new rule. if the rule reaches the correct result proceed.

    3. analyze the rulebase to see if any of the conditions match the action of other rules. In other words. Say my new rule has a condition "name=albert einstein". I search the action of the rules to check and see if any of them assert or modify the fact.

    4. if a rule contains a matching action, add it to a list of rules that may potentially conflict.

    5. build a decision table or tree to see if there is any conflict.

    6. analyze the data in the database and assert enough entries to prove the new rule does not result in indeterminancy.

    this is just one approach. One could build a decision table first if the rulebase is small and the rules are not complex. If the rulebase is large and each rule reasons over several objects, it may take a long time to generate a decision table. I favor the approach above because it allows me to filter out rules that don't apply, which makes it easier to build a decision table. Once I have a decision table, it's easier to figure all the potential states that affect those rules.

    of course, that is not a complete proof though. To prove it conclusively for the given rulebase and database, one would have to run through all possible combinations, which is much too intensive. The approach I take is to create a set of rule templates, which satisfy the business requirements, and avoid potential conflicts/contradictions.

    any of the modern rule engines are secure enough in my mind. The catch is you have to have either:

    1. a clear deployment/validation process in place with unit tests and QA people

    2.write a custom rule validation component to check all new rules before deployment.

    3. do 1 and 2

    I don't think there is a generic rule validation engine that can cover arbitrary domains. If that were possible, a lot of people would be out of a job. I'm not sure I answered your question. If not, feel free to email me directly woolfel AT gmail DOT com.

    peter
  15. Wrong tool for the job[ Go to top ]

    I think the author recently found out that velocity is very cool, but now tries to use it for every problem he sees.

    The author should take a look at rule engines, or maybe rhino for scripting..

    Velocity is not meant for scripting, but for templating.
  16. Those aren't rule engines![ Go to top ]

    Rule Engines? The alternative "engines" listed (BeanShell, EL, JEL) are scripting or expression languages, not rule engines. Certainly you can use a scripting language to build a rule engine, but they doesn't make BeanShell a rule engine any more than Java or Ruby or C++ is.

    Did you look into Drools (http://drools.org/)? There's a nice list of actual rule engines over at java-source (http://java-source.net/open-source/rule-engines).

    Velocity is a great template tool and I find this application interesting, but I'm not convinced the author properly compared this approach to actual rule engine libraries.
  17. Interesting[ Go to top ]

    Although the author of the article has his terms mixed up, there is a rule engine there. More specifically it is the RuleEngineVelocityImpl class, which implements his RuleEngine interface.

    The previous posts are accurate in their assessment of beanshell, jel, el and velocity. The sample looks like a very primitive map based procedural rule engine. In essence what the author has done is made his own rule markup, base interface and concrete class to process rules. The limitation of this approach is scalability and runtime performance for the following factors:

    1. number of rules increase
    2. number of conditions per rule increases
    3. complexity of rule. Measured by the number of objects a rule involves
    4. complexity of the rule as object attributes are compared to each other. in others joining object attributes
    5. selective firing of rules and grouping of rules
    6. resolution of potential comflicts

    A rule engine need not do any inferencing by producing new facts or modifying facts. Others may disagree, but I would classify the example closer to a flow or routing engine than a general purpose rule engine.

    I say that because the execution semantics do not include concepts like updating the working dataset. I wonder why the author didn't look at mandarax http://java-source.net/open-source/rule-engines or other general purpose rule engines. From my experience with financial applications, the approach described in the article would be inappropriate for a real trading system.

    the kinds of rules that I am aware of are far more complex and require features like joins, truth maintenance and rule chaining.

    peter
  18. I am really impressed with the uses of velocity as rule engine and article.

    NO SINGLE LINE RULE....Wow!!
  19. Brilliant![ Go to top ]

    It’s scary to think of that management team that thought they saved money by getting such an eloquent interpretation of “rules engine”. Too bad for Jess, and to think of all that R&D, when all they needed was a template engine.

    No knock on Velocity, but Rule Engine… c’mon. Is TSS this starved for content?
  20. don't worry, JESS is fine[ Go to top ]

    JESS is used in a ton of places. Most of them don't want it known they use JESS. Even though Sandia doesn't actively market JESS, it has a healthy customer base. Anyone doing real work with rule engines won't fall for "velocity is a rule engine" hype. I've also heard some people are laughing at the biztalk rule engine, since MS is claiming it's an inference engine. When they try it out, they realize very quickly it's not an inference engine since the biztalk docs call fact modification "side effects". Right there it violates truth maintenance and what an inference engine is suppose to do.

    i suspect as BPML pickup more hype, more companies will claim to have a rule engine when they don't. or worse, they'll claim to beat RETE when they don't even support inferencing.

    peter
  21. TSS and Rule Engines[ Go to top ]

    Although TSS is one of my favorite J2EE sites and I read it daily, it hardly can be named an authority in Rule Engines. The article's author (apparently a newbie in Rule Engines) got overexcited over his idea of using "rules" along with templates in Velocity and wrote an article -- which is fine. Unfortunately the article was posted here by the TSS moderator/watchdog with comments like "great rule engine" -- that's scary. Velocity is not a rule engine. And rule engines are not for programmers -- they are for business people a.k.a policy analysts a.k.a rule authors.

    What I saw in real life (Fannie Mae, Freddie Mac and a big government agency that give away millions as cash benefits to needy people) -- nobody needs a "rule engine". They need a full-blown BRMS (business rule/process management system) -- they have analysts/ rule authors, managers who approve rules and deployers. They have strict change management procedures controlled by SEC. Everything is traceable, rules are in repository and have versions and history. They want to see rules in English or pseudo-English in a powerful tool. Business people hate compiled code -- it's a black box for them and they do like rule engines because they can understand what they see and can change.

    There are very few such systems -- notably, ILOG JRules and Blaze Advisor. People who are talking about JESS or any of those open source Java rule engines probably never designed and developed manageable critical applications. What is crucial is rule management process (and of course execution) not rule engine.

    Glen
  22. TSS and Rule Engines[ Go to top ]

    Although TSS is one of my favorite J2EE sites and I read it daily, it hardly can be named an authority in Rule Engines. The article's author (apparently a newbie in Rule Engines) got overexcited over his idea of using "rules" along with templates in Velocity and wrote an article -- which is fine. Unfortunately the article was posted here by the TSS moderator/watchdog with comments like "great rule engine" -- that's scary. Velocity is not a rule engine. And rule engines are not for programmers -- they are for business people a.k.a policy analysts a.k.a rule authors. What I saw in real life (Fannie Mae, Freddie Mac and a big government agency that give away millions as cash benefits to needy people) -- nobody needs a "rule engine". They need a full-blown BRMS (business rule/process management system) -- they have analysts/ rule authors, managers who approve rules and deployers. They have strict change management procedures controlled by SEC. Everything is traceable, rules are in repository and have versions and history. They want to see rules in English or pseudo-English in a powerful tool. Business people hate compiled code -- it's a black box for them and they do like rule engines because they can understand what they see and can change. There are very few such systems -- notably, ILOG JRules and Blaze Advisor. People who are talking about JESS or any of those open source Java rule engines probably never designed and developed manageable critical applications. What is crucial is rule management process (and of course execution) not rule engine. Glen

    historically, that's one of the biggest limitations of JESS. It didn't have a full blown rule IDE for analysts. Those using JESS often have their own GUI's already, so it's a different category of development/deployment.

    shops that want a complete solution generally go with FairIssacs Blaze stack or iLog BR Studio stack. Mindbox is in there too, but they focus on a specific domain with their tools. The rule consulting world is small, so who is using what tends to be common knowledge to those in the industry :)

    Obviously, you know what Fannie Mae is using, so I won't bother mentioning it.

    peter
  23. TSS and Rule Engines[ Go to top ]

    Every software vendor proudly brags about big clients. Here in the Wash DC area ILOG had a tech day last summer and invited their big clients -- Fannie Mae, Freddie Mac, Equifax, eBay -- so it's not a secret. To my surprise, eBay uses JRules to analyze auctions. They told that they changed and deployed rules in 10 minutes after they received a call from FBI to stop selling pieces of the crashed shuttle.

    Peter
    You mentioned Mindbox - probably I'll have to evaluate them. What kind of industry are they focusing on? Is it "too financial"? My client is a government agency and they have to evaluate several vendors and produce a document why they selected one vendor among others. MindBox brags about proprietary "case based" engine in addition to rule based engine. Do you know what it is? Did you ever evaluate / work on MindBox? Any good rule engine forum on the Internet?
  24. TSS and Rule Engines[ Go to top ]

    Every software vendor proudly brags about big clients. Here in the Wash DC area ILOG had a tech day last summer and invited their big clients -- Fannie Mae, Freddie Mac, Equifax, eBay -- so it's not a secret. To my surprise, eBay uses JRules to analyze auctions. They told that they changed and deployed rules in 10 minutes after they received a call from FBI to stop selling pieces of the crashed shuttle. Peter You mentioned Mindbox - probably I'll have to evaluate them. What kind of industry are they focusing on? Is it "too financial"? My client is a government agency and they have to evaluate several vendors and produce a document why they selected one vendor among others. MindBox brags about proprietary "case based" engine in addition to rule based engine. Do you know what it is? Did you ever evaluate / work on MindBox? Any good rule engine forum on the Internet?

    I could be wrong, but in the past they focus on mortgage and financial. Yeah, it's not really secret or anything. But people outside the industry typically don't know about it. Mindbox likes CBR. I haven't used it, so I can't really say how it works.

    I know a lot of gov agencies use JESS because Sandia is government funded. Government agencies get a special deal on JESS license :) iLog and FairIssacs do like to brag. Sandia is the opposite, they rarely ever say anything.

    the power of rule engines is you can change the rule on the fly. The danger is when some one stupidly by passes the deployment process and a ends up crashing the production servers.

    peter
  25. Velocity as a Rules engine? Not even close.

    Take a look at ILog JRules or Blaze, now that's a REAL Rules Engine.

    I don't like it when people use the wrong tool to hack a job.
  26. give the poor guy a break[ Go to top ]

    It's obvious the author is just getting into rule engines. Calling Velocity a rule engine is wrong, but we all make mistakes.

    peter
  27. many definitions but one of the is the following: A tool that is pure rules, great usability for business people, decoupled from java services, fast, it will not kill your server's memory, and so on. For the case of jrules , blaze or harley , I think if you use a rule engine as a platform for many things other then rules then you should be careful. Also putting all your business logic into their proprietary scripts is dangerous. These are the things we kept in mind when we designed Rulesharp. Have a look for yourself www.rulesharp.com randy
  28. You've got to be kidding me, right?[ Go to top ]

    That’s a problem in human nature. We can’t come out from our mindsets so easily. Yes!! Everybody knows, Velocity is a great Template engine but I compiled its features and uses it as rule engine successfully.

    If you have something in your pocket and you can use it to solve your problem. It’s not foolishness to go out to try to find something else to solve that problem.
  29. You've got to be kidding me, right?[ Go to top ]

    That’s a problem in human nature. We can’t come out from our mindsets so easily. Yes!! Everybody knows, Velocity is a great Template engine but I compiled its features and uses it as rule engine successfully. If you have something in your pocket and you can use it to solve your problem. It’s not foolishness to go out to try to find something else to solve that problem.

    like I said in a earlier post, the two classes you implement constitute a rule engine to me. it would appear others do not agree. Though one could just as easily create a RuleEngine interface and concrete implemetation without velocity at all. I could just as easily use beanshell to take java-ish syntax and compile it. My interpretation is that rule engines mean something very specific to them.

    Therefore calling velocity a rule engine doesn't sit well. Perhaps the title should be different, more like "using velocity to build your own rule engine."

    peter
  30. You've got to be kidding me, right?[ Go to top ]

    You may be right peter but what is more important an “envelope” or the “letter and message” inside it?

    Most of the people are not able to digest the uses of Velocity as rule processor but let it come closer to you and break you mind sets and then open your CMD prompt write few code to try on it and then I won’t need to say anything.

    I saw few comments of the code sample as well. As I said now, it’s code SAMPLE. It’s not actual code. If I might have added the actual code then you people must have been cursing me for the complexity of the sample code.:)
     
    My aim was to describe the concept, not to build up a BRMS product.
  31. You've got to be kidding me, right?[ Go to top ]

    You may be right peter but what is more important an “envelope” or the “letter and message” inside it?Most of the people are not able to digest the uses of Velocity as rule processor but let it come closer to you and break you mind sets and then open your CMD prompt write few code to try on it and then I won’t need to say anything.I saw few comments of the code sample as well. As I said now, it’s code SAMPLE. It’s not actual code. If I might have added the actual code then you people must have been cursing me for the complexity of the sample code.:) My aim was to describe the concept, not to build up a BRMS product.

    This is just my opinion, so take it with a big grain of salt. Since I work with rule engines as an application developer and write rule engines, writing a robust, scalable rule engine is a difficult task. Although it may appear simple to take velocity and write a rule engine re-using some features, that is far from a robust flexible rule engine. Ignoring BPM and BRMS for now.

    I have zero experience with velocity, but I doubt a templating language is sufficient. My reasoning is this.
    #if( $creditLimit > 1000 )
        Over limit. You can't complete this transaction.
    ##OR just a Boolean value like false##
    #else
        #set ( $creditLimit = 1000 - $creditLimit )
        Now, your balance is ${creditLimit }.
    ##OR just a Boolean value like true##
    #end

    I assume $creditLimit resolves to a variable. Based on this simple syntax, one of the fundamental designs of modern rule languages is to avoid (if, then, else). There are a couple of reasons for this. When rules are written in a (if, then, else) format, it is harder to compile the rule such that evaluation does not require checking all conditions. doing so incures a major performance and scalabilty cost as rule complexity increases. I'll use an example.

    if
      A && B && C && D
    then
      set A == 2, set B == 1, set C == 0, set D == 10
    else
      set A == B

    In practice, a (if, then, else) rule should be written as two rules, or simplified as one rule. I could rewrite the rule this way:

    if
      A && B && C && D && (A != B)
    then
      set A == 2, set B == 1, set C == 0, set D == 10

    It's not always possible to rewrite the rule in this fashion. It really depends on what each atomic condition is and what the intent of the rule is. In the example, it's pretty simple and only involves 1 variable. What happens if I need to compare 1 attribute from 3 objects? How will it look like in the pseudo template language? How does the system then process the rule for optimal runtime performance?

    It's very easy and trivial to come up with a simple rule API and framework, but once a developer starts using it to build an application, the limitations become painfully apparent.

    peter
  32. You've got to be kidding me, right?[ Go to top ]

    <blockquoteThis is just my opinion, so take it with a big grain of salt. Since I work with rule engines as an application developer and write rule engines, writing a robust, scalable rule engine is a difficult task. Although it may appear simple to take velocity and write a rule engine re-using some features, that is far from a robust flexible rule engine. Ignoring BPM and BRMS for now.I have zero experience with velocity, but I doubt a templating language is sufficient. My reasoning is this.
    #if( $creditLimit > 1000 )&nbsp;&nbsp;&nbsp; Over limit. You can't complete this transaction.##OR just a Boolean value like false###else&nbsp;&nbsp;&nbsp; #set ( $creditLimit = 1000 - $creditLimit )&nbsp;&nbsp;&nbsp; Now, your balance is ${creditLimit }.##OR just a Boolean value like true###end
    I assume $creditLimit resolves to a variable. Based on this simple syntax, one of the fundamental designs of modern rule languages is to avoid (if, then, else). There are a couple of reasons for this. When rules are written in a (if, then, else) format, it is harder to compile the rule such that evaluation does not require checking all conditions. doing so incures a major performance and scalabilty cost as rule complexity increases. I'll use an example.if&nbsp;&nbsp;A &amp;&amp; B &amp;&amp; C &amp;&amp; Dthen&nbsp;&nbsp;set A == 2, set B == 1, set C == 0, set D == 10else&nbsp;&nbsp;set A == BIn practice, a (if, then, else) rule should be written as two rules, or simplified as one rule. I could rewrite the rule this way:if&nbsp;&nbsp;A &amp;&amp; B &amp;&amp; C &amp;&amp; D &amp;&amp; (A != B)then&nbsp;&nbsp;set A == 2, set B == 1, set C == 0, set D == 10It's not always possible to rewrite the rule in this fashion. It really depends on what each atomic condition is and what the intent of the rule is. In the example, it's pretty simple and only involves 1 variable. What happens if I need to compare 1 attribute from 3 objects? How will it look like in the pseudo template language? How does the system then process the rule for optimal runtime performance?It's very easy and trivial to come up with a simple rule API and framework, but once a developer starts using it to build an application, the limitations become painfully apparent.peter
    II don’t think, using the velocity to execute the rule will cause some kind of performance problem. Many of the high volume web applications are using velocity to throw their dynamic output. And most of the genius around recommends velocity to do so.

    So if we generate the complete page using velocity templates but if we ask our application to execute 1 - 10 more lines using the same tool then we can have performance problem? Common…. It can’t be!!!

    Practically speaking, most of the rules in real word applications don’t involve more then 2-3 variables but if somebody wants to do something like Fourier series calculation or like sorting and then comparing some objects then he is free to write a tool which is a pluggable thing and he can use it like a single call in his rule template.
  33. You've got to be kidding me, right?[ Go to top ]

    I don't think, using the velocity to execute the rule will cause some kind of performance problem. Many of the high volume web applications are using velocity to throw their dynamic output. And most of the genius around recommends velocity to do so. So if we generate the complete page using velocity templates but if we ask our application to execute 1 - 10 more lines using the same tool then we can have performance problem? Come on. It can't be!!! Practically speaking, most of the rules in real word applications don't involve more then 2-3 variables but if somebody wants to do something like Fourier series calculation or like sorting and then comparing some objects then he is free to write a tool which is a pluggable thing and he can use it like a single call in his rule template.

    I don't know what kind of rules or application you build, but the stuff I work on involve large datasets like millions of rows, hundreds of rules, and rules with quite a bit of joins.

    for something simple like flow or routing rules, you're far better off using a simple tuple based rule engine. I'll put it another way. If you're given the task of building an application that must process 500 transactions per second, how fast do the rules need to evaluate if each transaction may involve 20-100 rules?

    try running your simple example where you need to run real-time analytics like OLAP multi-dimensional queries, or complex policy rules which look at how long a process take and then adjusts how much to charge the customer. There's a lot more going on in the rule world than simple rules with 2-3 conditions. The history of rule engines is over 3 decades, so it's had plenty of time to mature.

    go look at some of the proposals for W3C policy rules and then see how the velocity approach would handle those cases. on the otherhand, if the requirements are that simple, than I see no reason to do anything more complicated than the example in the article.

    peter
  34. You've got to be kidding me, right?[ Go to top ]

    Give it up. Variables are just a simplifed example to suit your simplified example. Your idea doesn't fit well on any business rule engine semantics. Chaining is where you will inevitably paint your templates into a corner.

    There a whole lot of mess in between the utility of what you present and what you claim it not to be (Fourier series calculation).

    Put a better example on the table
  35. You've got to be kidding me, right?[ Go to top ]

    Give it up. Variables are just a simplifed example to suit your simplified example. Your idea doesn't fit well on any business rule engine semantics. Chaining is where you will inevitably paint your templates into a corner.There a whole lot of mess in between the utility of what you present and what you claim it not to be (Fourier series calculation). Put a better example on the table

    Some perspective here. The examples provided with biztalk are just as simple as the article and that's being actively marketed by Microsoft. It would be interesting to see a more complicated example like this:

    if
      user.member is current
      user.group is equal to admin
      group.id is equal to user.groupid
      group.access is equal allowed
    then
      set session.access to true

    if
      session.access is equal to true
      content.lock is equal to false
    then
      allow editing

    this is a very basic example. want to give it a try with velocity and templates?

    peter
  36. You've got to be kidding me, right?[ Go to top ]

    <blockquoteSome perspective here. The examples provided with biztalk are just as simple as the article and that's being actively marketed by Microsoft. It would be interesting to see a more complicated example like this:if&nbsp;&nbsp;user.member is current&nbsp;&nbsp;user.group is equal to admin&nbsp;&nbsp;group.id is equal to user.groupid&nbsp;&nbsp;group.access is equal allowedthen&nbsp;&nbsp;set session.access to trueif&nbsp;&nbsp;session.access is equal to true&nbsp;&nbsp;content.lock is equal to falsethen&nbsp;&nbsp;allow editingthis is a very basic example. want to give it a try with velocity and templates?peter

    Oh yes :)
    It can be done very easily. Rule/template for your examples may look like as follows:

    #if( $null.member == "current"
         && $null.group == "admin"
         && ${group}.id == $null.groupid
         && ${group}.access == "allowed" )

      #set ( session.access = "true" )

    #elseif( ${session}.access == "true"
    && ${content}.lock == "false" )

    allow editing

    #end


    Where I assume that user, group, session and content are references to corresponding objects and will be available to Velocity Context as per your application anatomy.
  37. You've got to be kidding me, right?[ Go to top ]

    Oh yes :)It can be done very easily. Rule/template for your examples may look like as follows:#if( $null.member == "current"&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp; $null.group == "admin"&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp; ${group}.id == $null.groupid&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&amp;&amp; ${group}.access == "allowed" )&nbsp;&nbsp;#set ( session.access = "true" )#elseif( ${session}.access == "true" &amp;&amp; ${content}.lock == "false" )allow editing#endWhere I assume that user, group, session and content are references to corresponding objects and will be available to Velocity Context as per your application anatomy.

    cool. now here is the next question. How many times will each attribute be evaluated for a single evaluation of all the necessary data and the given rules?

    peter
  38. a little change in the rules[ Go to top ]

    Here is another scenario for you to think about.

    if
      session.active
      user.member is current
      user.group is equal to staff
      group.id is equal to user.groupid
      group.access is equal allowed
    then
      set session.access to true

    if
      session.active
      session.access is equal to true
      content.lock is equal to false
      user.group is equal to staff
    then
      allow editing

    if
      session.active
      content.lock is true
      user.group is equal admin
    then
      allow unlock content


    with this subtle change, the rules aren't that much more complicated, but now it affects the runtime performance. How many times would the velocity approach evaluate session.active? would it A) do it 3 times or b) do it once for all three rules?

    some food for thought. what appears simple isn't always simple. even minor changes in a rule can have dramatic effects.

    peter
  39. A spanner in the works![ Go to top ]

    Here is another scenario for you to think about.if&nbsp;&nbsp;session.active&nbsp;&nbsp;user.member is current&nbsp;&nbsp;user.group is equal to staff&nbsp;&nbsp;group.id is equal to user.groupid&nbsp;&nbsp;group.access is equal allowedthen&nbsp;&nbsp;set session.access to trueif&nbsp;&nbsp;session.active&nbsp;&nbsp;session.access is equal to true&nbsp;&nbsp;content.lock is equal to false&nbsp;&nbsp;user.group is equal to staffthen&nbsp;&nbsp;allow editingif&nbsp;&nbsp;session.active&nbsp;&nbsp;content.lock is true&nbsp;&nbsp;user.group is equal adminthen&nbsp;&nbsp;allow unlock contentwith this subtle change, the rules aren't that much more complicated, but now it affects the runtime performance. How many times would the velocity approach evaluate session.active? would it A) do it 3 times or b) do it once for all three rules?some food for thought. what appears simple isn't always simple. even minor changes in a rule can have dramatic effects.peter

    Peter,

    I have very little experience in rule engines but a great deal in EL and content rule-like engines, If I modified
    your example:

    if
      session.active
      user.member is current
      user.group is equal to staff
      group.id is equal to user.groupid
      group.access is equal allowed
    then
      set session.active to true

    if
      session.active
      session.access is equal to true
      content.lock is equal to false
      user.group is equal to staff
    then
      allow editing

    if
      session.active
      content.lock is true
      user.group is equal admin
    then
      allow unlock content


    then how many times should 1) a rule engine evaluate
    session.active, and 2) velocity evaluate it?

    Food for thought...

    - Gary
  40. that depends[ Go to top ]

    if
      session.active
      user.member is current
      user.group is equal to staff
      group.id is equal to user.groupid
      group.access is equal allowed
    then
      set session.active to true

    if
      session.active
      session.access is equal to true
      content.lock is equal to false
      user.group is equal to staff
    then
      allow editing

    if
      session.active
      content.lock is true
      user.group is equal admin
    then
      allow unlock content
    then how many times should 1) a rule engine evaluate
    session.active, and 2) velocity evaluate it?

    Food for thought...

    - Gary

    If it is a RETE rule engine, session.active will only evaluate once for all rules. This assumes the rule engine implements node sharing. If it doesn't, then it depends how closely the rule engien follows Forgy's description. In forgy's 1980's paper, RETE builds a sequence of 1-input nodes (aka alpha node).

    I have no idea how velocity works, but I'm guessing it's procedural, which means 3 times. Unless this velocity approach is able to chain the conditions of the rules, it will be 3 times per evaluation cycle. In JESS the rules would look like this

    (defrule rul1
      (session
        (active true)
        (OBJECT ?sess)
      )
      (user
        (member "current")
        (group "staff")
        (groupid ?gid)
      )
      (group
        (id ?gid)
        (access "allowed")
      )
    =>
      (call setActive ?sess true)
    )

    (defrule rule2
      (session
        (active true)
        (access true)
      )
      (content
         (lock false)
      )
      (user
        (group "staff")
      )
    =>
      ;; allow editing
    )

    (defrule rule3
      (session
        (active true)
      )
      (content
         (lock true)
      )
      (user
        (group "admin")
      )
    =>
      ;; allow unlock content
    )

    The number of times a single session instance would be evaluated in this set of rules is twice. This is because rule2 evaluates 2 attributes.

    In clips and jess variable bindings are preceded by "?" quetion mark. I won't bother describing the RETE network, since it's easy enough to download JESS, load the rules and execute (view) command. So generally speaking, a forward chaining rule engine that implements node sharing will evaluate the fields of the objects fewer times.

    enjoy

    peter
  41. A spanner in the works![ Go to top ]

    by the way, that is a trivial example. A single instance of JESS can easily handle all the session within a single servlet container. whereas it's not at all clear how this velocity approach would handle it.

    hopefully my answer was to your satisfaction.

    peter
  42. the real test[ Go to top ]

    If you really want to show the validity of the velocity approach, I would suggest running both manners and waltz benchmark.

    http://www.cs.utexas.edu/ftp/pub/ops5-benchmark-suite/

    Run a simple example with 8 guests and see how long your approach takes. If it ends up taking more than 30 minutes, don't be surprised. if 16 guests ends up taking more 2 hours don't be surprised either. My guess is for 32 guests, it might take your velocity approach 12 or more hours.

    peter
  43. This is ridiculous[ Go to top ]

    can I just write some if-else conditional logic and call it a Rules engine? then I can just use some JSP tags, use JSTL or a case switch statement and now I've created a Rules engine?

    Come on this is just ridiculous.

    Velocity is a Template Engine, Period. End of Discussion.

    Now JESS, I can say that is a Rules Engine, but it's still not a fully-blown BRMS like ILog or Blaze.

    Does the author of this article even know what a Rules Engine is? Are you familiar with the RETE algorithm that ILog supports?
  44. for those interested[ Go to top ]

    there's some resources on drools about RETE and rule engines. Here are some more resources on rule engines, forward chainging and other topics in this domain.

    http://drools.codehaus.org/
    http://www.ruleml.org/
    http://www.cs.berkeley.edu/~wilensky/cs188/lectures/practical-deduction6topage.pdf
    http://www.csd.abdn.ac.uk/~cmellish/teaching/CS5538/lectures/
    http://www.igda.org/ai/report-2003/aiisc_rule_based_systems_report_2003.html
    http://portal.acm.org/citation.cfm?id=17360&coll=GUIDE&dl=GUIDE&CFID=39845790&CFTOKEN=8328330
    http://www.bizrules.com/us/page/articles.htm

    I can understand the appeal of writing a quick and dirty rule engine to meet immediate needs, but as history has shown, it only works in cases where scalability and performance are not needed.

    peter
  45. The solution might work for the author; however he performed a clumsy research regarding the issue at hand. I don’t know how on earth his investigation on rule engines end up having "JEL ,Bean Shell and EL" as rule engines???
    We may call these non-standard tricks: like this one or using XSLT for implementing rules as "rule patterns" and not rule engines.
  46. Granted the “envelope” opened a can of worms not worth continuing, what make the jLog and Blaze worth the $? Is it the GUI? The repository? The syntax? Can’t convince myself to take the leap. Jess has a good documentation, seems easy to use. I wish it were on the radar for Spring.
  47. some reasons[ Go to top ]

    My experienec with iLog BR Studio is limited, but from what I hear in the industry their natural language editing capabilities is the primary draw for customers. It's unrealistic to demand business analysts learn how to write clips or in a programming language. Therefore, a natural language like interface is desirable. It's basically equivalent to Domain Specific Language (DSL). Blaze also has a good suite of IDE's for writing rules.

    the hard part though is understanding the business case and figuring how where and how to optimize the business process. once you know that, it's 100x easier to write the rules or design a DSL for that specific domain.

    peter
  48. some more reasons[ Go to top ]

    Some more reasons -- just to name a few

    - debugging in ILOG is very powerful. It saves a lot of development time.
    - ILOG has decision tables -- our analysts love them. ILOG 5.0 supports copy/paste from MS Excel. Every row in the decision table gets translated to a rule so think about DT as a set of rules.
    - ILOG has plug-in for Eclipse. Some of our developers work in Eclipse and we save on Rule Builder licenses.
    - have web interface, so you can updates some rules over the Internet
    - have very good graphical rule flow editor. Looks like UML state diagram.
    - if you have time constrains for rules - like "this rule should work from Dec 12 to Dec 25" you have "travel in time" feature to debug those rules.

    Agree, to define a good and manageable set of rules is the most complicated part.

    JESS and ILOG -- like Linux 8 years ago and now. 8 years ago only hard-core geeks could install and work on it. Usually policy analysts are not geeks and do not want to be ones. Now it's simple and user-friendly. I know about a project that was designed and prototyped in ILOG and once they finalized rules they wrote rules in JESS -- client was a government and already had JESS.
  49. Thanks[ Go to top ]

    Good to get your perspectives. ILog sounds like next step for me.
  50. Thanks[ Go to top ]

    Try to get evaluation version of JRules. Plug-in for Eclipse is not enough to understand the whole power and beauty of ILOG.

    One more thing -- ILOG is going to support both .net and Java i.e. the same rule file (basically it's a text file or xml) can be read and used by both C# and Java. For my client it's very important...
  51. I thought it already did[ Go to top ]

    the .NET version of JRules was released last year, so I believe the same rule file currently works regardless of the platform you use. there are some differenecs though. the .NET version uses custom attributes, where as the older version of JRules doesn't.

    peter
  52. some more reasons[ Go to top ]

    +1

    Just to elaborate a bit more from my own experience. One of the hardest parts about rule engineering is translating from human knowledge to executable rules. In the old days, this was sometimes call "knowledge engineering." These tasks were done by specialists who understood rules, rule engines and had the communication skills to sit down with the domain expert and ask the right questions.

    After several months of hard work, the knowledge engineer would produce a functional spec with sufficient details to produce rule templates and rules. Rule templates are similar to word templates. The structure of the rule stays the same, but the values change.

    As the industry moves forward with combining decision tables, decision trees, domain specific language and modeling techniques, it becomes easier for analysts to write rules. The pitfall is that often analysts don't think in mathematic algorithms. That results in rules that may appear correct, but in practice are not. Decision tables/trees are good tools for exposing these issues, but it still requires the analyst work closely with the engineers. But like all things, the key is cooperation and communication.

    I think the new iLog BR Studio has better DSL (domain specific language) features, but I haven't had a chance to play with it yet. Have you used them glen?
  53. some more reasons[ Go to top ]

    We use ILOG 4.6 and ILOG 5.0 is not that much different except Decision Tables -- they improved them a lot.

    My current project is quite interesting and quite different from what I saw before. We use a lot of collections / grouping / relations -- i.e. operators collect, in, exists, not in, etc. and we have to use our own collector classes (i.e. implement IlrCollection interface). Condition part for some rules is quite complicated. On contrary the action part is very simple and consists of just one method call. I can deal with it but the only option for me is technical rule language (trl) -- similar to Java. If you have collect in the condition part it's very difficult or impossible to write rule in Business Application Language (BAL) -- pseudo-English language. I had a consultant from ILOG for a couple days -- he could not help me much.

    Example? Among other things we find eligibility of people for government benefits (money, food stamps, health insurance). We have to divide a household on groups to find the best possible combination of benefits. Group A gets Benefit X, group B gets Benefit Y -- benefit is for a group. It's possible that a father or spouse are in some other group but we have to count their income in order to find eligibility for a child. So our rules look like "for a group, if there is a child in the group and there are father/mother in some other groups who does not get Benefit Z and there are no caretakers for that child in other groups and child's brothers/sisters if any have benefit X or better then give benefit X to the group" -- I have to use two "collect" and one "not exists" and call a single method grantBenefit. No way to come up with good BAL.

    Two things - 1. BAL has some limitation -- not every rule can be written in BAL and 2. rule structure states the same you change only values -- sometime the left hand part gets really complicated so I agree and do not agree with you that later on you just change values. In my case, the left hand part is very twisted and what they change is the left part.
  54. some more reasons[ Go to top ]

    We use ILOG 4.6 and ILOG 5.0 is not that much different except Decision Tables -- they improved them a lot. My current project is quite interesting and quite different from what I saw before. We use a lot of collections / grouping / relations -- i.e. operators collect, in, exists, not in, etc. and we have to use our own collector classes (i.e. implement IlrCollection interface). Condition part for some rules is quite complicated. On contrary the action part is very simple and consists of just one method call. I can deal with it but the only option for me is technical rule language (trl) -- similar to Java. If you have collect in the condition part it's very difficult or impossible to write rule in Business Application Language (BAL) -- pseudo-English language. I had a consultant from ILOG for a couple days -- he could not help me much. Example? Among other things we find eligibility of people for government benefits (money, food stamps, health insurance).

    We have to divide a household on groups to find the best possible combination of benefits. Group A gets Benefit X, group B gets Benefit Y -- benefit is for a group. It's possible that a father or spouse are in some other group but we have to count their income in order to find eligibility for a child. So our rules look like "for a group, if there is a child in the group and there are father/mother in some other groups who does not get Benefit Z and there are no caretakers for that child in other groups and child's brothers/sisters if any have benefit X or better then give benefit X to the group" -- I have to use two "collect" and one "not exists" and call a single method grantBenefit. No way to come up with good BAL. Two things -

    1. BAL has some limitation -- not every rule can be written in BAL and

    2. rule structure states the same you change only values -- sometime the left hand part gets really complicated so I agree and do not agree with you that later on you just change values. In my case, the left hand part is very twisted and what they change is the left part.

    All good points. I've also seen rules where the left-hand side (conditions) gets rather complex and can't fit nicely into a rule template. Some people will disagree with this statement, but in those cases you need a rule expert. Someone who has tried to implement the edge cases and figured out the hard way what not to do.

    The benefits example sounds like the classic pattern matching problem, which solved in a non RETE approach ends up being 100-1000x slower than RETE. In CLIPS terms, the collections/groups is equivalent to multi-slot. Having worked on that class of problems, I'd agree with you. Many people make mistakes with multi-slots on jess mailing list. God knows I did when I first stared using jess in 2000.

    I'm bias, but I think BAL/DSL can be effective if the scope is narrow and clear. Writing a BAL to cover 80% is a good approach to take. Often it's impractical and impossible to design a BAL that covers 95%. You end up with a BAL that is difficult to compile and produces less than optimal rules.

    peter
  55. some more reasons[ Go to top ]

    Peter,

    Can you please email me at magadan594 at yahoo dot com ? Want to discuss you something that is out of this "Velocity" topic :-).

    Thank you
    Glen
  56. BAL in ILog, New Decision Tree[ Go to top ]

    Peter or Glen, Have you guys worked with 5.0 at all with the new Decision Tree. I know you mentioned Decision Table, but I wanted to know if the Decision Tree was Business-Analyst friendly.

    Also, the BAL in ILog is just "OK". I think it can be a lot better, but I can see how hard it is to link up the regular english language with IRL code. I often find that the BAL alone does not suffice, and you'll have to create a lot of custom stuff by writing IRL code.

    I'm hoping that in the future JSR-94 will take off, and more vendors will support it. This way, we may get an open-source IDE to support a project like JESS.

    Starting a new project on Apache to create RETE would be too much work. Like Peter said, why duplicate all that work that's already out there.

    Velocity is old technology. It's a decent template engine, but I still disagree its use as a rule processing engine. There is just no way anyone should use Velocity to interpret rules. That's like writing an HTML page using JavaScript, or using JSPs to directly connect to a DB using JDBC. Why would anyone even think of doing that? I'm usually one to think out of the box, but in this case, I have to disagree.
  57. BAL in ILog, New Decision Tree[ Go to top ]

    I've only played with BR Studio a little. I haven't had the opportunity to use it on a project. A lot of the work that I've done in the past involved writing custom IDE's for specific domains.

    As to whether decision tree could be business-analyst friendly, i would say that depends on the analyst. I can think of several cases where decision tables and decision trees don't help. here is an example from 2A7
    C. Share Price Calculations. The current price per share, for purposes of distribution, redemption and repurchase, of any redeemable security issued by any registered investment company ("money market fund" or "fund"), notwithstanding the requirements of section 2(a)(41) of the Act and of Rule 2a-4 and Rule 22c-1 thereunder, may be computed by use of the Amortized Cost Method or the Penny-Rounding Method; Provided, however, that:

    # Board Findings. The board of directors of the money market fund shall determine, in good faith, that it is in the best interests of the fund and its shareholders to maintain a stable net asset value per share or stable price per share, by virtue of either the Amortized Cost Method or the Penny- Rounding Method, and that the money market fund will continue to use such method only so long as the board of directors believes that it fairly reflects the market-based net asset value per share.

    # Portfolio Maturity. The money market fund shall maintain a dollar- weighted average portfolio maturity appropriate to its objective of maintaining a stable net asset value per share or price per share; Provided, however, that the money market fund will not:

       1. Except as provided in paragraph (c)(2)(ii) of this section, Acquire any instrument with a remaining maturity of greater than 397 calendar days; or

       2. In the case of a money market fund not using the Amortized Cost Method, Acquire a Government Security with a remaining maturity of greater than 762 calendar days; or

       3. Maintain a dollar-weighted average portfolio maturity that exceeds ninety days.
    http://www.law.uc.edu/CCL/InvCoRls/rule2a-7.html

    I didn't paste the entire section C of the regulation, since it's rather long. In this example, there are multiple ways of calculating "share price", but the thing is the "or" in the regulation means calculating using both methods and then picking one with least risk. In cases like these, the decision table may show contradictions in some cases, since it depends on the actual fixed income security. For other securities it may not have any contradictions.

    Therefore viewing it in a decision table isn't helpful. In fact, it can lead to a false sense of compliance. Although some people claim you can capture these complexities with a decision table, it is not possible. By not possible I mean this. It may be feasible to test a limited set of data and prove it always reached the desired conclusion. In practice that's not possible, because a tradign firm will buy and sell securities. Even if it is true today, there's no gaurantee tomorrow it is still true.

    In those cases, a decision table ends up costing a company a lot of fines. Regulatory compliance is specifically written to have redundancies to make it harder for people to get around them. A common example is measuring exposure to a security through multiple dimensions like security rating, issuer rating, maturity, duration and combination of one or more dimension.

    ok, I'm probably boring everyone to death so I'll shut up now.

    peter
  58. BAL in ILog, New Decision Tree[ Go to top ]

    1. Totally agree with you on Velocity. It's not a Rule Engine and I do not even bother discussing it. Please see #2 and #3.
    2. Also agree with Peter -- leave the poor guy, the author, alone -- we all make mistakes
    3. Leave the poor guy, article submitter and "content starving" poor TSS, alone also -- we all make mistakes... I wish TSS could be more "rule engine" friendly -- so far, I did not find good rule engine place. But I guess it depends on us too.
    4. Decision tree -- I do not know in general, but for our analysts it's too low-level. We have too many rules (several hundred) to draw decision trees -- the original agency policy manual has 1,200 pages. Usually we discuss the rule flow and sometime the content of a package -- under package I mean set of rules that constitute a node on the rule flow graph. When we were evaluating rule engines almost two years ago we had vendor presentations and Blaze boasted that ILOG does not have the decision trees. I personally think that ILOG introduced Decision Tree just to keep up with Blaze. I am skeptical about decision trees -- just another representation of a rule. Do not buy much.
    5. ISR-94 -- I am VERY skeptical about JSR-94, total waste of time -- it buys nothing, much ado about nothing. It's standard for interface, not for rule engine itself. Interface with ILOG is just four lines - create context, assign rule file, assert some objects and invoke execute -- that's it. Hey, to switch to Blaze you have to rewrite and retest several hundred rules, redraw rule flow, replace decision tables with rules, redraw decision trees, but the good news -- you do not have to rewrite those 5 lines because they meet jsr-94! Congratulations! You saved 10 min of precious development time.
    6. About Apache... I have a friend who is Apache member, not just contributor. He always asks why people think that they can easily start a project in Apache? It's not a SourceForge. Apache is a serious established brand. Can you go to Nike with lousy shoes design and ask them to produce them under Nike's brand? Usually for a project it is a greatest accomplishment to get transferred to Apache. Usually a project is viable and already has established community. Or a vendors like BEA, Sun or IBM donate code to Apache. There are many good projects that I doubt will ever get under Apache. So either join DRools or start a brand new on the SorceForge.
  59. 2. Also agree with Peter -- leave the poor guy, the author, alone -- we all make mistakes

    Sure, we all make mistake. However, not many people have the gust to admit their mistake. This naive author does not seem to think that he has made any mistaken. There are no mentioning of that Velocity is far being a rule engine.
    I wish TSS could be more "rule engine" friendly

    Me too.
  60. BRLDF framework in ILOG[ Go to top ]

    Also, the BAL in ILog is just "OK". I think it can be a lot better, but I can see how hard it is to link up the regular english language with IRL code. I often find that the BAL alone does not suffice, and you'll have to create a lot of custom stuff by writing IRL code.

    We are using ILOG at Aceva Technologies since JRules 3.0 days. We always had our own domain language BAL was never enough. With ealier versions (prior to 4.0) we had to write tons of very complex code expressing grammar using Ilr classes. But with introductoin of BRLDF (business rules language definition framework) creating custom language has gotten much easier. With this new framwork you can specify your grammer in BNF form using xsd and you are half way there. This gives you a very powerful tool to create custom lanauge. But then again you can not use decision tree or decision tables or templating functionality with custom language, oOnly BAL is supported with these functionality. Hope that helps.
  61. BAL / IRL / TRL / BRLDF[ Go to top ]

    Peter,

    Agreed, the out of the box BAL does not always cover 100% of cases. Of course we keep improving it -- so it is getting more and more expressive. We are careful however as we don't want it to get too close to a programming language.

    We support other languages however; TRL and the "native" IRL, and we have our Business Rule Language Definition Framework -- which you can use to create your own language. BAL is defined using BRLDF for example. In addition you can write code to customize the translation of a BAL rule to IRL... so plenty of flexibility! :-)

    Dan
  62. BAL / IRL / TRL / BRLDF[ Go to top ]

    do you mean you work for ILOG? Did ILOG started mail list as they promissed on the tech day?
  63. BAL / IRL / TRL / BRLDF[ Go to top ]

    Peter,Agreed, the out of the box BAL does not always cover 100% of cases. Of course we keep improving it -- so it is getting more and more expressive. We are careful however as we don't want it to get too close to a programming language.We support other languages however; TRL and the "native" IRL, and we have our Business Rule Language Definition Framework -- which you can use to create your own language. BAL is defined using BRLDF for example. In addition you can write code to customize the translation of a BAL rule to IRL... so plenty of flexibility! :-)Dan

    I'm hopefully that over time with lots of hardwork, the industry can make it easier. I could be overly optimistic, but I think in 20-25 years with dedication and focus, it feasible to make it much easier to build rule applications. I'd love a full license of BR Studio to play with instead of the trial version :)

    peter
  64. some reasons[ Go to top ]

    "most of the rules in real word applications don’t involve more then 2-3 variables"

    Can you tell me your experience regarding this? I have worked with enterprise level rules and 2-3 variables is nowhere even close. Most of the rules I have seen start with 10 variables at a minimum.
  65. some reasons[ Go to top ]

    "most of the rules in real word applications don’t involve more then 2-3 variables"Can you tell me your experience regarding this? I have worked with enterprise level rules and 2-3 variables is nowhere even close. Most of the rules I have seen start with 10 variables at a minimum.

    10 sounds about right from what I've worked on. 2-3 is *cough* trivial. using the velocity approach, once the number of rules doubles or triples, it's going to see 4-8x increase in evaluation time. At that point, there will be a huge bottleneck and it will have to be thrown out. The history of procedural rule engines hitting the scalability wall is well known in this field. Even today, there are some major software vendors in the financial space struggling with scalability because their rule engines work procedurally.

    peter
  66. Thanks :)[ Go to top ]

    JRule is cool but the problem is – it’s not free. Anyway,

    Thanks for reading the article and for investing your valuable time in posts. I really appreciate the comments/compliments from all of you guys. I hope it was really a nice learning experience for us in the context of Rule Dilemma.

    But one thing is for sure, we are lacking a STANDARD open source Rule Engine/ BRMS.

    What about starting a new OPEN SOURCE project under some umbrella like Apache? :)

    Thanks again.
  67. there's already drools[ Go to top ]

    drools already exists, so there's not much point in starting a new rule engine project. writing a full RETE rule engine is hard enough that having duplicate efforts would probably be counter intuitive. Once the engine is done, there's still all the IDE stuff that is needed, so my best guess is it will take several years at minimum. having worked on rule engines, written rule compilers and built applications with rules, it takes a lot of time to gain a good understanding.

    peter
  68. No Thanks To You[ Go to top ]

    Yes, people times are very valuable. They should not be wasting their times reading your un-inform article.

    You don't even know anything about rule engine.

    TSS is really suck lately, posting these inaccurate articles.
  69. ... please go to Discussions/ Feedback and join me
    Thank you
    Glen
  70. We'd also love to have more discussion over at http://www.javarules.org!

    Dan
  71. Check out the Prova rule-based scripting language at http://comas.soi.city.ac.uk/prova. It merges Java with Prolog-style rules and distributed agents. For starters, I suggest having a look at the presentation at http://comas.soi.city.ac.uk/prova/etc/ProvaPresentationSep2004.ppt and a reviewer's guide at http://comas.soi.city.ac.uk/prova/etc/ProvaReviewersGuide_1.7.doc.

    Alex
  72. Not new[ Go to top ]

    Check out the Prova rule-based scripting language at http://comas.soi.city.ac.uk/prova. It merges Java with Prolog-style rules and distributed agents. For starters, I suggest having a look at the presentation at http://comas.soi.city.ac.uk/prova/etc/ProvaPresentationSep2004.ppt and a reviewer's guide at http://comas.soi.city.ac.uk/prova/etc/ProvaReviewersGuide_1.7.doc.Alex

    I read through the second powerpoint. Although it looks interesting, it's not new. People have been building rule applications using messaging for over 8 years. According to the powerpoint, Prova is built on Mandarax. If we go back further to ART and CLIPS, people have been using messaging + rule engines in financial compliance for over a decade. Actually, it goes back to late 80's when ART was still in business.

    I haven't used prova, but the features in JESS provide the similar functionality. For those curious about the benefits of backward vs forward chaining, Dr. Forgy has a good article on the topic.

    http://www.rulespower.com/contents/article1.html

    I don't see the point of having a "scripting language" that combines imperative and declarative to be honest. When you start introducing imperative language with declarative, it can lead to some really bad practices. Generally, once the rules start mixing procedural code with declarative rules, you end up making the rules a mess and increases the cost of maintenance. In my mind, this is a horrible thing to do. The approach that most modern engines take is to provide a "built-in" API so that you can write small functions, which you can then call in the rule. This way, the rule remains declarative and the procedural bits are in discrete functions.

    I realize the topic of rule engine is rather large and it's hard to keep track, but the powerpoint presentation needs more research. Many of the examples and claims look terrible to my bias eyes. ART and CLIPS both provided a simple way to write functions, and call them from rules. Anything a backward chaining rule engine can do, can be done more efficiently using forward chaining with better scalability. The catch is it requires an experience rule consultant. those who say backward chaining is better often claim it is easier to understand. that might be true, but scalability and performance aren't achieved through laziness.

    peter
  73. I believe that the author has no clue to what rule engine is. A rule engines is a systems that can make reason or inference either in a forward manner (forward chaining) or in a backward manner (back chaining). Velocity does not have either. The author should download JESS and play with it to understand what rule engine is. JESS use the RETE engine to do its reasoning.

    http://herzberg.ca.sandia.gov/jess/

    JESS is Reference Implementation for JSR-94 (Java Specification for Rule Engine). There are other Java rule engines , such as JRules and more... The exciting fact about JESS is it has a large user-base and developers from everywhere develops extension to JESS. One extension is FuzzyJ which extends JESS to reason with uncertainty (fuzzy-reasoning). Other users are doing neuro-fuzzy rules systems, that is combining neural network with fuzzy logic (via FuzzyJ) and accomodate JESS to achieve a powerful LEARNING rule engines. This is fascinating area as neural network discovers the rules on the fly from the database and loads those rules to the working memory of the JESS RETE engine. In this case the rules are not pre-determined, that is they are dynamical rules.

    Cheers,
    sione.
  74. that's way beyond most users[ Go to top ]

    I think you might have lost a lot of people there. FuzzyJ, fuzzying reasoning and dynamic rules are way beyond the average rule application. But on the topic of dynamic rules, there's been research the last few years into pi-calculus for mobile scenarios. Once you get away from mainstream BPML, policy rules or compliance, there's all sorts of crazy stuff one can do with RETE rule engines.

    One of the most powerful features of JESS is the ability to dynamically write new rules and add them to the rule engine. There are some people doing this type of work in the fuzzy reasoning realm, but it's pretty hard to apply these to business applications.

    peter
  75. that's way beyond most users[ Go to top ]

    I think you might have lost a lot of people there.

    Perhaps , I might lost a lot of people there, but my intension not to do that. I tried to describe beyond the static rules in comparison to state of the art dynamical rules systems. There is a huge trend at the moment to this area both in academic & industries. Static rules is fine in you know your rules. If you don't , then static rules is useless. There are situations where rules are not static, that is why I mentioned JESS with many extension that its users have developed. Other commercial rule engines lack this extensibility of JESS. I will give you an example:

    What do you think that 'Amazon.com' is deploying? A static rule engines or a dynamic rules engines ? Buyers behaviour changes all the time very rapidly and it would be crazy for a business analyst to edit the rule knowledge-base every 2 to 3 hours, because initial rules would not apply to new situations that has just arose. They have to be pruned, and a human needs to do that job manually.

    In static rules systems, not only the business analyst do not known in advance of the buying behaviour of all online customers and therefore cannot hard-code the rule knowledgebase. This is when dynamical rules ,as I mentioned in my earlier posts is all about. Let the machine learning part (neural network, support vector machine, decision trees , and so on) or data mining engine discover the rules on the fly and then load to a RETE engine for deductive inferences. Rules can be pruned dynamically if they do not fit patterns in the current situations and new rules are added automatically if the mining engine detects a new rule-patterns appearing in the database. Well, there is no human to sit and watch the rule change overs, it is all the machine that does this. If you think you can try out a static rule-engines something like JRules for product recommendation systems such as 'Amazon.com' , the systems will become useless in just one day of trading. 'Amazon.com' uses text-mining when potential customers type in key-words when they are searching for book titles. If you use a static rule engines, you will suffer the curse of dimensionality for rule explosions in the thousands or even millions because search terms are not known in advance for online customers looking for books. 'Amazon.com' regularly mines the titles, key-words (similar words) , content of the book database and index them. This indexed database is incrementally change (is changes very fast) every minutes/hours or so, since everyone in every second logs on to 'Amazon.com' to search for a book title. When there is new data is indexed, it means that there is potential new rule that the mining engine can extract and load to the RETE engine.

    There are many applications from finance to control engineering and so on about dynamic rule engines being deployed but 'Amazon.com' example is the widely known one for dynamical rule engines.

    Cheers,
    Sione.
  76. agreed[ Go to top ]

    That is one of the most prominant examples of using a rule engine + data mining (aka knowledge discovery in databases) to generate rules and remove rules on the fly. Using static rules for a recommendation engine would reach its limits very quickly. this is especially true of products that go through fads and trends very quickly. Often it is not practical for the analysts to dig through 1TB of data just to figure out how they should change the recommendation rules.

    Very few companies have reached that level of sophistication. There are even fewer rule consultants who know Knowledge base reasoning/ case base reasoning well enough to build such applications. That probably why Amazon has been successful. They've been able to combine their expertise in data mining with rule technology to produce a system that is pretty effective. I've only explored recommendation engine scenarios a little, but it is complex topic.

    On a side note. I remember a talk by an Oracle instructor who interviewed at Amazon. They asked him, "how do you defragment a 1TB database?" That was back in 99. I'm sure Amazon's database is considerably larger than that now.

    peter
  77. DFA vs rule engine[ Go to top ]

    The author of this article describes a solution to a real problem. However, he seems to fail not in respect to the solution he provided, but in respect to the claim he makes that this is a "rule engine".

    I believe that if he had described the problem as a DFA, where a state comprises of the varibles, authorisations, resources etc, and any other contextual information that the author describes, and the transitions are the so-caller "rules", then this article would have gone down much more smoothly.

    I have recently encountered a similar problem, of regulating screen flows according to complex and ever-changing corporate rules, and built a DFA based on:
    *Shark JaWe (jawe.jar) to draw the DFAs.
    *Shark XPDL parser (sharkxpdl-standard.jar), to parse the generated XPDL and to take from it the process definition information.
    *BeanShell to evaluate the transition conditions.
    *5 classes I wrote to create a POJO representation of a running DFA.

    With this approach I made a small DFA engine in a few days, that can be used to regulate screen flow and to set get state info, that is independent of the underlying GUI approach. I am currently using this inside an applet that accordingly request the next jsp to load.

    I know that one can blame me that JaWe produces XPDL that describes business processes and not DFAs. Maybe I had spent too much time in the workflow area, and I try to give a workflow flavor to all the solutions I give. I am aware of the possible critisism, but I like the shark editor very much, it is LGPL, very stable and very smart.

    Others could critisize that I should be looking at UML statecharts, and I would appreciate feedback on this direction, if anybody knows of a really good statechart approach with:
    *LGPL licence.
    *a nice editor.
    *a ready made parser API.
    *higher level abstractions of "superstates", like XPDL "subflows" and "block states" to break down my DFA, and parameter passing semantics on calling these "superstates".

    I would appreciate feedback in this direction.
  78. Velocity allows you to write delicious scripts that really shine. Concluding Velocity is both a dessert topping and floor waxing solution based on those obscure observations would be asinine but no more so than this bizarre suggestiong that it's a rule engine. That's really quite asinine.
  79. A Better Question[ Go to top ]

    How does nonsense like this get posted on TheServerSide as a legitimate "article"? Isn't there some sort of sanity check?
  80. Hi All,

    Just want to know if there is anyhting similar to TRL in ILOG rules for .net.

    some of my req are :

    1.I have List and I need to Loop thru tht
    2.Create new instances inside rule engine and send them to the app.

    Some rules involving complex business logic had to be written in ILOG rules for .net rather simple if else.

    I know tht its possible with TRL and IRL in ILOG JRules.But donno know abt ILOG rules for .net.

    Anyone pls help me out.

    This is very urgent.

    Vineela
  81. I am quite new to architecturing Business layers.
    Can somebody guide me on the followings.

    1.What are the available strategies for creating business layer(I presume one is hard coded and others using rule and workflow engine).please illustrate in depth if there can be any other way of seperating the business layer strategies

    2.Do biztalk server 2004 supports compile time or runtime workflow/rule
    3.What are the common algorithms(like rete etc) used in rule engines and their advantage features.
    4. which is the most popular rule engine for dotnet and why?
    5. can ilog for dotnet only be used with Biztalk server

    thanks in advance