Business Rules in the SOI

One of the architectural topics that caused our team more than a bit of thinking was our approach to implementing rules.

The original idea for handling rules as noted in the high-level architecture diagram below (to the left of the Business Action) was to implement a common rules engine that would be responsible for the storage and execution of rules. This approach would allow for one-stop shopping for Process and Entity Services as well as a common repository for easily changing rules.



Initially three different options were discussed:

• BizTalk Rules Engine (BRE). Since BizTalk is a major player in the Compassion SOA and includes a rules engine, our first thought was to leverage this engine to store and execute rules.
  
  While the engine is versatile and can be called from inside .NET assemblies (such as the business actions we’ll be developing), the major impediment to using the engine was twofold: 1) The BRE does not include an easy to use interface for managing the rules. As a result, other organizations have created their own interfaces using tools like Microsoft Excel to manage the rules. Without doing something like this rules could not be managed by business user, 2) when using the BRE a BizTalk installation is required with the appropriate licensing. This model may not be desirable if services are deployed in a partner country since it would force the adoption of BizTalk at those locations.
  
  From a previous concall with Microsoft it is also not anticipated that Microsoft will ship the BRE as a separate product in the near future.



• NxBRE. A second approach that was considered was adopting the open-source rules engine for the .NET Platform called NxBRE, which is a port of the JxBRE engine available at SourceForge. This engine use rules defined in RuleML and allows for rules to be defined using Visio stencils. One of our consultants performed an evaluation of NxBRE to determine how it could be invoked from within business actions.
  
  After evaluating this option it was decided that although it overcomes the licensing restrictions implicit in the use of the BizTalk BRE it still does not provide a simple way for business users to easily change the rules. Further, as with the BizTalk BRE, NxBRE is an inference engine that works off the concepts of Facts, Queries, and Implications. This structure would work well for simple data validation but would be unwieldy when trying to represent data modification rules since a large number of facts would have to be loaded into the engine and the return data from the engine would be complex (for example to represent rules regarding how an entity’s attributes are set based on all of the data associated with the entity).



• Ultimus 7.1. Version 7.1 of the BPM suite includes a rules engine called Ultimus Director that can be used from within Ultmus workflows to represent rules. It is designed with end users in mind and so can be used by business users to change and update rules without developer intervention. However, the engine is not meant to be invoked from outside Ultimus and it is assumed that licensing restrictions would apply. As a result, this option does not appear to be a general solution.

In order to clarify Compassion’s position regarding the implementation of business rules in the SOA architecture a conference call with Gartner analyst Jim Sinur was held. He provided the following clarifications and best practices.

1. The need for and use of BREs should be determined based on business rule volatility

• That tipping point is usually in the range of 25-35% of the rules to be processed
• Compassion should rank business rules by; change frequently, hardly ever change, and middle-of-the-road, that is, perform a volatility analysis
• BREs may be cost effective for managing the rules that change frequently
• Compassion may have few business rules that change frequently and a BRE would be overkill

2. BREs are expensive and there are currently no solid candidates in a .NET architecture

• Virgin Atlantic has created a custom spreadsheet interface to BizTalk for volatile business rules, this allows business analysts to change these rules without developer assistance
• Microsoft may add a similar interface to BizTalk, however, this is just something Compassion should watch and not count on

3. Create guidelines for business rule placement

• "Flow-Related" business rules should be placed in BizTalk or Ultimus
• Business rules should be parameterized whenever possible - especially the more volatile rules
• Avoid placing business rules within custom business framework services - especially any "flow-related" rules
• Create and use a mechanism for business rule management
• Report on what's there
• Keep pulse of how often the rules are changed
• Change the strategy if you discover a high degree of change
• Create rules so they can be tailored to meet global requirements

Approach
As a result of this analysis and the Gartner discussion the decision was made not to pursue adopting a BRE as a single repository for all rules within our service-oriented infratructure. The decision was based primarily on two factors. First, although a volatility analysis has not been done it is highly unlikely that Compassion meets the 25-35% threshold where implementing a BRE would be cost effective. Secondly, within our technology scope there doesn’t appear to be a BRE that could be employed without creating a custom UI and/or incurring licensing costs. As a result, the direction we’re following is to employ the native rules engines within BizTalk and Ultimus (Process Services) and then implement custom rule development within Entity and Infrastructure Services.

In following this approach the following rules matrix was developed in order to provide guidance as to how and where rules would be implemented.

Process Rules
What?
Rules related to the flow of a system to system process or system to human process, for example, a rule that changes the routing of a form within Ultimus based on partner country

Where?
Implemented in the BizTalk BRE or Ultimus Director when available

Data Integrity Rules
What?
Rules related to how data is ultimately stored in a data store such as Compass or an XML document including FK relationships and link tables, for example, a rule that dictates that when an email is updated the shared email addresses are also updated

Where?
Typically implemented in stored procedures that persist the data and invoked from business actions. If the data is persisted in XML documents, then XSDs can be used to implement these rules

Data Modification Rules
What?
Rules related to how data can be inserted, updated, or deleted. Typically, these rules are dependant on the state of the persisted data in a data store, for example, a rules that specifies that in the US only two email addresses are stored

Where?
Implemented directly in business actions that perform inserts, updates, or deletes.

Data Validation RulesWhat?
Rules related to the format of data, for example, the requirement that an email address conform to a regular expression

Where?
Implemented directly in business actions that perform validation typically prior to inserts or updates


In following with the best practices outlined by Gartner the Data Modification and Data Validation rules include support for externalization. Specifically, this support includes:

The ability for a business action to be notified as to which rules to process based on a configurable value (such as the partner country) and on which client is invoking the service

The ability for the business action to use specific parameters for a rule based on a configurable value (such as the partner country)

We implemented this support using a custom EDAF handler as shown in the following diagram.



As indicated in the diagram the rules handler is invoked as a part of the service implementation pipeline for specific business actions. The handler passes configuration information indicating the local value along with the service action name to the rules store. The rules store returns two data structures, one that is a collection of rule identifiers (GUIDs) that should be processed for the business action, and a second that includes a collection of country specific values to use when processing the rules. Both data structures are placed into the EDAF Context. The Rules Handler caches the data structures once retrieved for performance reasons.

The business action then uses the data structures in the Context to determine which rules to process and what values to use when processing. The semantics of the rules themselves are encoded in C# or VB .NET code, typically in private helper methods that make them easy to maintain. The collection of rule ids are used only to indicate which rules to process and so the presence of the id in the context triggers the execution of the rule.

Because the two collections are not related, multiple rules can use the same local value in its processing.

When rules are retrieved the handler queries against these two tables. Given the LocalValue, ServiceActionName and the client making the request (From information encapsulated in the SOAP header), a set of Active RuleIDs would be returned. If the the client doesn’t appear in the BusinessActionClientRules table, the Active flag from the BusinessActionRules table applies. Notice that the syntax of the rule is not included in the table, only the id and whether it is active for the business action and client. This data would then be placed in a hashtable within the EDAF Context object by the Rules Handler.

Inside a business action, particularly the BusinessActionBase class, we've written code to check the hashtable for the RuleID before executing the rule.

We've also built an ASP.NET user interface to manage the rule data.

Overall, this solution has workd quite nicely and in production we use these rules both to provide specific values to use when evaluating a rule and to turn rules on and off for specific clients.