Rules are created and added to Audits to be run together with other rules, or can be run individually. They can be scheduled or run ad-hoc. With DvSum, you define a data audit rule once, which can then be run repeatedly.
Rules in DvSum can be created from a number of different locations within the tool, covered in this section. They can also be added from the Rules Library, or if you have existing rules for your data, they can also be upload to DvSum.
When a Rule is created it is assigned a unique ID, such as DQ-000167. This links to the Rule page which includes the Rule Definition, Summary, Exceptions from the latest Audit run, and trends.
Standard Rule Creation
The majority of Rules are created through the Rules Wizards provided within the tool. From the left navigation pane, access Audit > Manage Rules, and click “+ Add Test.”
Completing Rules Wizards
For each of the Rules wizards, information is completed within the following sections. Depending on the type of Rule, options will differ. If the Rule is applicable to a single data source, Reference Inputs will not apply. Doc-matching rules will also require different information.
For some rule types, under Basic Inputs, the Variance Check check-box indicates this is a rule that is comparing to a previous run and you will be asked to provide a Variance Threshold or %.
Creating Rules from within Profiling – Data Quality
When in Profiling, by clicking the Data Quality tab, you have the ability to create Rules and Rulesets for the data you are reviewing.
You must first click “Enable Ruleset,” complete the required information and click “Save” before defining the specific checks.
Rules created from the Data Quality tab within Profiling are limited to Completeness and Value Range checks. For example, by selecting the “Blanks Check” check box, the system automatically creates a rule to find and return blanks for that selected field.
You also have the opportunity to establish Filter conditions, to eliminate the scenarios where a blank might be an acceptable value.
Within the same Ruleset, or as a standalone Rule, you can also define Value Range Checks. These can be defined based on a Min/Max Threshold, or based on select values from a Picklist within the Field or Reference Dictionary.
Based on what you define for the Blank Checks and Value Range Checks, this is automatically saved as a Rule, and can be viewed and edited under Audit>Manage Rules.
Creating Rules from within Profiling – Data Analysis
More complex, data combination Rules can be created from within Profiling, under the Data Analysis tab. When exploring data through the filtering analysis, you may identify combinations of data that you want to define as Exceptions or that you want to create a query for to review the record details.
For example if the combination is invalid and should not exist, a Rule can be created from this view. As seen below, for a Product/Item Category of Accessories (Accy) sourced from Asia, a user may know that the Plan Level (planning category or priority) should not be “4.” By clicking “Add Business Rule,” entering a Description, and clicking Create, a Rule template (or shell) is created, enabling you to create the specific rule.
Clicking Create takes you Audit>Manage Rules with the Rule template created. From here you can Edit, add detail, Action Items, Create a Workflow, etc. before finalizing the Rule.
You may also simply choose to click, “Add Rule” which takes you to the Rule Editor. Here you will need to recreate the filtering conditions for the Rule you want to create and click, “Add Rule” which notates the condition under the “Exceptions Rule List.”
Here you can specify whether the conditions are valid or invalid. if Valid, the rule will show under "Valid Rule List". It will consider the condition as valid and bring all other records as exceptions. If Invalid, the rule will bring only those records as exception which meet the condition. This can be selected from top right corner.
When creating Rules from this view, you are able to create multiple filtering conditions, within the selected Attributes, and also add them as Rules. This creates a compound Rule that returns all qualifying Exceptions or records within the query, and allows you to view them together. At the bottom, there is Cleanse Column list. Here you can check the columns which you want to make cleanse-able and unchecked column with only serve as a filter.
In the cleanse workbench, the filtered columns with be selected by default and only cleansable column will be showing exceptions to be fixed.
Alternatively, DAE rules can also be created from Audit -> Manage Rules tab as well.