| While data analytics serves as a powerful tool used in organization to gain insight into operations, they may also be a source of risk. |
Feasibility to reveal secrets hidden in voluminous data makes use of the data analytics difficult to resist. There are tools enabling the organization to bring efficiencies in operations, improve decision, enhance agility, identify market potential etc.
Internal audit is regular user of data analytics during the ordinary course of discharging its responsibilities to the organization. Internal audit uses data analytics for its own use as well reviews the data analytics done by functional units.
With ever-increasing volumes of data on hand and entity’s dependency on that data, certain relevant questions need to be answered.
Assessing the Risks
Existence of possibility of things going wrong justifies usage by internal audit function of the data analytics in the organization. Majority of chief audit executives (CAEs) opines that their organization’s net residual data analytics risks are ‘moderate’ to ‘extensive’.What are these risks?
- If the data under consideration is erroneous, incomplete, disorganized, old, or inconsistent, the conclusions based on itwouldn’t serve the desired purpose for the entity. As such most worrisome is the completeness and accuracy of the data which is presented to management based on which strategic decisions are taken. Management generally takes the information presented on face value and without doubting the accuracy and completeness of data.
- Thereis concern about data quality which is addressed through audit. The auditor evaluates the risks around the completeness, accuracy, integrity, and security of data. For instance, since data warehouse is part of the data analytics process, look at risks and controls are seen in entirety around the path the data takes viz. sources of raw data, the methods and technology used of transferring the data to the WH, the controls over the warehouse, and the transfer to the end user. If there are inaccuracies or issues with the data at any point along this path, then the end result may be misleading and any decisions / inferences based on this data may also be flawed.
- Further, the data may be sound, but the algorithms used to summarize it may be faulty. They may require some additional activities like edit check, doing something unwarranted, without the business unit being aware. Though this may not necessarily influence the result; then again, it might.
- Additionally, concerns are raised about the integrity of data collection process itself – ethical; used for the purpose for which it was collected; anddata collected in a way to provide objective results or to prove a point.
- Auditor needs to be careful ofbiasedhandling the data andshouldn’t be carried away by the initial impressions while performing subsequent analysis and actions. If done, auditor may go down wrong path, getting a result that appears accurate while not realizing we are unintentionally overlooking other data.
- Higher the volume of data the organization has, the greater is enticement it providefor hacker to prey into it and eventually compromising security and privacy.
Methodologies to be Used
While dealing with a diverse range of complex risks, certain tried and tested audit approaches shall provide best results.
Generally, the timing of analytical reviews depends on the nature of the data.If the data pertains to operational, technical, or regulatory risks, the frequency of reviews is taken into consideration during audit planning process.
The techniques used to conduct the audit can also be standardized. Based on the source of the data and it’s usage, the auditors need to look at it, since management may be making critical decisions based on it. Auditors rely on a structured approach to audit the data analytics process and reuses approaches that have worked well in different departments.
A traditional approach applies to the controls recommended to address various findings like:
- Input controls – completeness, accuracy, and reliability of the data;
- Processing controls – reconciliation of changes made to normalize/filter the data; and
- Output controls – accuracy, based on inputs and processes.
For instance, the Data Warehouse (DW) has teams of personnel dedicated to operating and maintaining data.DW is installed with defined routes from the sources of data to the warehouse and from the warehouse to the end users. It is advised to verify:
- Employees / users possess required expertise to ensure the completeness, accuracy, integrity, and security of the data.
- Evidence of documentation and communication of processes and controls pertaining to the use and security of data.
- Evidence of testing the operating and design effectiveness of controls pertaining to appropriate and relevant access and change management.
- Changes to the control environment and supporting databases are tracked and monitored.
- The analyses are supported by built-in quality and effectiveness checks to ensure they (and the data) mirror the changes and evolution of the business.
In relation to data analytics, certain personnel oriented controls are critical – especially management oversight and user education. If users are intended to be given flexibility for creating their own reports/analysis, they must understand how to use the tools correctly to evaluate the inputs and outputs. They must be able to validate the date to verify its completeness and accuracy using data and tools.
Certain Finer Points
It is appropriate to use proven methodologies during the process of auditing the business units’ data analytics use.However, as each auditdo present their own unique challenges – their subtleties must be recognized, understood, and resolved.
Business units perform data analyses in different shapes and forms, using different algorithms and basing their analyses on different assumptions. Riskis there when the internal auditor or the business unit itself incompletely or incorrectly understands or agrees on such foundational issues. Here certain concerns needs to be addressed:
- Validity of the assumptions
- Perform integrity checks
- Age of review of algorithm.
- Influence of one data event on subsequent activity.
(a). Internal auditors may commit a mistake if they do not validate key assumptions with facts (i.e., confirmation of key data points and the underlying assumptions) before continuing with testing. The auditor shall reach completely inaccurate conclusions by taking the wrong path early in testing. The root cause for the error in reporting mostly is – not sufficiently validating assumptions and initial results. The issue is a huge hit to the integrity of the testing and audit process.
(b). Another challenge is in knowing exactly what to audit. On a micro level, the auditor may look at a specific department to understand the objectives of the deliverables/reports, the sources of the data, and the distribution of the data. It is important to review the process undertaken to produce reports: how the data changes through the cycle and how the changes are accounted for.
(c). Prioritization is important on a macro level. Every department
- Generate, use and analyze data to produce a result,
- Has goals and objectives;
- Report on how it performs against those goals.
Auditors have to work with the departments to identify reports used in management’s decision-making process. That will help in understanding which activities to review and why.
(d). Internal audit is rendered ineffective, if its findings cannot be explained in a way that resonates with the business unit that has been audited. Internal auditors must consider the learning modalities of their audit clients when discussing the findings; people hear, see, and experience things differently.
The auditor may find it convenient to submit a simple written, text-heavy report, it may be more effective to use visually appealing, concise images in support of the text. A verbal presentation — in support of the written report — that includes concrete examples of the findings or the risks that may accompany the findings is also likely to make a more lasting impression. This gives clients multiple ways to absorb and understand the recommendations, based on the way they process information.