How do You Audit Your Data?

With DCM Audit you can audit actual data in your ServiceNow instance. Audit functionality reports data compliancy against the planned blueprints and helps you to identify any deviations & fix the data issues.

Audit analyses each record and link starting from a Root record. Each step in the analysis generates audit results and messages which tells if a record or a link was compliant against the blueprint. Audit will also find, if required data is missing or if it doesn?t match with given conditions.

In this article I will:

• give an example blueprint for ?User data model?
• show examples of audit results against that blueprint
• discuss about blueprint elements and unique audit results
• discuss how to use audit results

In order to run an audit, you first need to have a published Blueprint. You can read more about how to create blueprints from our wiki.

Example Blueprint

Example Blueprint – User Data Model

In the example blueprint above, we have a ?User? table selected as a root container (or class) on the left. User container has three links defined:

1. Mandatory Department reference will connect a user to organizational structures
2. Mandatory Manager link define another user as a manager (used for approvals and reporting)
3. Optional Company link for directly connecting users to companies. This is optional, since a User is already connected to a Company via Department and Business Unit structures.

This blueprint includes User and Company classes multiple times in different parts of the blueprint. In DCM, we use a concept called ?blueprint element? which means a unique part of any given blueprint. In the example blueprint, all four Company containers are just one blueprint element. And the same element can exist in multiple other blueprints as well. Links are blueprint elements too and a combination of source class, target class and a ?link type? creates a single blueprint element.

One could draw this blueprint in many different ways, but for the sake of an example, this should be easy to understand and will create meaningful data when we talk more about Audit results and Unique audit messages.

Running an Audit

Now that we have a blueprint ready and published, we can use that to run an audit. You can read more about how to run an audit from our wiki pages.

Once an audit run has completed, we should have a new Audit instance record available which includes all the findings from that particular audit run.

Audit instances basically include three different types of records:

• Audit results are created for each ?root record? that was part of the audit.
• Audit messages will collect all the information about every audited record in comparison to containers, links and filter conditions defined in the blueprint.
• Unique audit messages are unique versions of the audit messages. This will group all duplicate audit messages under one record.

Here is an example of Audit Result view:

Example audit result

In the example audit we had only six ?audited records? (also know as root record). So, audit was limited to six records from the root class which was the User class in this case. Quick overview of the audit tells us that:

• Only one record was fully Compliant with the blueprint. Meaning that every detail was in place, including optional links and matching filter conditions.
• One record had a Note level deviation. Meaning that optional information was missing.
• Four records had Critical deviations. Meaning at least one mandatory piece of information was missing.

In brief, 17% of our User data is Fully compliant while 83% has some issues to be addressed.

Now, for each Audited record (six of them) we can have a look at the more detailed Audit messages to see what we should fix. Let?s take ?Joakim Sourcehill? as an example.

In brief, 17% of our User data is Fully compliant while 83% has some issues to be addressed.

Example audit messages – user data audit

From the audit details window, starting from Joakim as the root record, we can see:

• Six compliant messages meaning data defined according to the blueprint. Since the total number of messages happens to be 10, this represents 60% of the ?required data? related Joakim as root record.
• Two note messages indicating that optional data is missing. When taking a closer look, it?s actually the same message twice, but found from different parts of the blueprint.
• Two critical deviations indicating which mandatory details were missing. In this case, the ?Consulting? department is not connected to a Business Unit which also means that mandatory Company reference is missing.

In this view we see all audit messages generated by a single audit instance. And we saw that some messages were duplicated even related to a single root record within a single audit instance. Naturally more duplicates are found when auditing more Users related to same organizational structures and other users (managers). More duplicate audit messages also appear when same audit in ran again without fixing the issues in between.

In order to get a clear view into issues we need to address, the Unique audit messages will show each finding only once, but use the original audit messages records to set a weight for each message to indicate an impact of that deviation within the big picture. This way, audit messages can be considered as ?events? that are then correlated into stateful unique audit message to be addressed.

From the above example, the ?Optional reference missing? message for Joakim not referring to a Company would get a weight value 2. And if Joakim would be a manager for other users, auditing those users also would increase the weight of this audit message.

With this information, we can focus on correcting the issues with the most impact first.

“Unique audit messages” is a new feature in DCM planned to be released in DCM R4.0. We will create another article about unique audit messages once generally available.

Another View

The DCM application also includes a module called Data Content Planner. You can read more about the Planner from our wiki pages, but in this context I just wanted to show what a single root records looks like when opened into the Data Content Planner.

Example data content – user data model

The idea with the Data Content Planner is to use a blueprint in the background and allow users to update and create data on top of the data model.

The Data Population Wizard Preview will show the same information as non-compliant audit messages in the earlier examples. DCM user can search, drag & drop and modify records in the Planner. And then populate all changes at once to actual tables, when done.

Back to Audit Results

Now we have seen the details related to an audit instance and a single root record, but what about the overall data quality or compliance trends?

With all the detailed information readily available, it?s quite easy to see the big picture also by creating a summary level reports and trends.

We can look at audit results on different levels and from different perspectives. Different levels of details for audit results are:

• Audit instance
• Audit result (root record)
• Unique Audit Message (new in DCM R4.0)
• Audit message

With different perspectives I mean different filters and conditions used to select audit data to be reviewed. For example:

• All audit instances for current month, year etc.
• Audits using same blueprint
• Audit results for a particular class or audited record
• Unique audit messages related to certain class or single record
  etc.

Next, let?s have a look at what information is available on different levels.

Audit Instances

Audit instances are used to collect all audit results and messages related to a single audit run. From this level, we can find metrics like:

Audit result counts and percentages per deviation level (Critical, minor, note, compliant)
Compliant vs. Non-compliant message counts and percentages
Messages counts and percentages per deviation level

These can be very useful when following up the overall data quality and trends for different data domains or the ServiceNow system as a whole.

Audit instance records and related metrics are meant to be kept from six to twelve months for trend reporting.

Audit Results

Audit results in this context refer to summary of audit messages per audited record. An audit result record will indicate the completeness of data from a root record?s perspective against the whole data model.

If we take ?Joakim? as an example user again. The missing Business Unit reference for a Department that Joakim belongs to is reported as a deviation for ?Joakim? when looking at the Audit results record. While the actual record missing the reference is ?Consulting? Department.

From Audit result level, we can find following metrics:

• Overall deviation level of the Audited record (=worst deviation level of any related audit message)
• Compliant vs. Non-compliant message counts and percentages
• Message counts and percentages per deviation level (Critical, minor, note, compliant)

These can be very useful when following up the data quality and trends for particular records or collection of records. This level of information is targeted for people responsible for the data on a daily basis (for example service, application and process managers).

Audit result records and related metrics are meant to be kept from six to twelve months for trend reporting.

Audit Messages

Audit messages represent the most detailed level of audit data. Each audit message is related to root record which is the starting point for the audit and the ?real record? which is the actual record that generated the message.

When using ?Joakim? again as an example, the messages related to ?Mandatory Business Unit reference missing? is connected to ?Joakim? as the root record and ?Consulting? department as the real record.

From audit message level, we can find following data:

• Audit result type (categorization of audit messages)
• Deviation level (note that audit also includes compliant messages)
• Related root record
• Related real record (or closest real record)
• Related blueprint element
• Related unique message (starting in DCM R4.0)
• Previous audit message (indicating the path)
• Audit message text

This level of detail is very useful when communicating what needs to fixed or otherwise addressed. From reporting point of view, we can create different lists and summary graphs on what kind of issues we have. From remediation point of view non-compliant audit messages can be used to trigger events, create tasks or send notification for example.

Different Dashboards

The DCM application includes an example dashboard showing the audit results for the system as a whole. As such, it can be very useful for people responsible for the platform. For others, it?s more of an example on what kind of reports and widgets are available based on audit data, using out-of-the-box reporting capabilities.

What I would recommend is to create a dashboard per ?data domain?. You can read more about the data domains for example from this previous article.

Below is one example of a data domain specific dashboard for ?Business Applications?:

Example dashboard – Business application data domain

This dashboard is using audit data related to three different blueprints, all related to business applications:

• Tier minimum ? Service frontend: example of a minimum required data model for multiple root classes. This blueprint could define the record level ownership for a particular domain.
• Target data ? Business Applications: example of a long term target data model for all business applications.
• Critical data ? Business Applications: example of critical data required for all applications OR could be required data model for critical business applications also.

The point here is that with different blueprints we can have a bit different requirements for the same data, or same classes at least. And by having different blueprints we can also focus and follow bigger trends at the same time.

Remediation Tasks

When digging into a record level and (unique) audit message details, a report or dashboard may not be the best way to communicate deviations.

In the Business Application dashboard example above, there is one report showing ?deviations per Application manager?. And for these managers, it could be better to create a task and use the audit messages to describe what needs to be checked and fixed as part of that task.

This type of automatic task creation is already available using DCM Add-ons. And we?re planning to include it as a standard feature in a later DCM release.

What do you think of all this? How do you report, maintain or improve your data quality on a regular basis?

Note: Screenshots in this article have been taken from DCM version 3.3.2 and views may have changed since.