Professional Judgment When Using Data Analytics

March 3, 2020 – Just released results of research conducted by the American Institute of Certified Public Accountants (AICPA) and Rutgers Business School show that auditors need to use professional judgment in developing appropriate filters or to identify additional audit data analytics to help with the filtering processes in their audits. 

The Rutgers/AICPA Data Analytics Research Initiative (RADAR) Multidimensional Audit Data Selection (MADS) Research Project was aimed at evaluating the potential for a systematic approach to identifying and categorizing items that have one or more characteristics that could indicate a risk of material misstatement in data sets identified through the use of audit data analytics. This research explored the use of filtering techniques to analyze whole populations of data and ways of “categorizing” the items identified. 

The research team and RADAR board members conducted several experiments over various audit areas, including revenue, expenditures, payroll and journal entries, to help them understand how different data sets may affect the development and application of filters. 

According to the researchers, the following key observations “may be used to inform future research, develop audit data analytics, or considered when assessing potential changes to existing auditing standards and related guidance.”

  • Identifying and categorizing items for further analysis or testing requires flexibility and significant judgment.The development and application of filters depend on the data set being used, the financial statement account, the management assertion being tested, risks identified, the relevant business or industry, as well as other considerations, and requires significant professional judgment. “There is not one standard set of filters that can be used on all data sets or even on a single financial statement account.” While the board and research team initially sought a systematic approach or a framework, after considering the dependencies highlighted above, the board believes that it is more appropriate to allow for various filtering techniques to analyze audit data rather than establishing a single systematic approach.
  • Determining how and when to apply filters and the level of evidence obtained are areas of significant judgment. Throughout the research project, filtering techniques were used to identify populations of items with varying degrees and types of risk. “A significant amount of professional judgment is needed to consider the relevant filters to use, the nature of those filters and the persuasiveness of the evidence produced by the filtering process,” the researchers concluded. These considerations will depend on the data set, the financial statement account, the relevant assertions, the risks identified, the relevant business or industry, as well as other factors. 
  • Filtering assists in determining the nature, timing and extent of procedures.Auditors using filtering techniques apply professional judgment when determining how to appropriately categorize the results. As the items identified are often categorized by risk, the nature, timing and extent of procedures performed may vary for each group, including any populations that did not meet any filter criteria. For example, if filtering resulted in two different groups of identified items, such as those that meet the filter criteria and those that do not, the auditor may choose to perform representative sampling for one group of identified items and substantive analytical procedures for the second group.  
  • When using filtering techniques, professional judgment will be needed to determine how to appropriately assess potential errors identified.Auditors apply judgment when assessing errors in the population of items identified through filtering, including determining over which population to extrapolate an error or whether the initial filters were not appropriate. If they were not found to be inappropriate, auditors may revise the filters and apply the modified filters to the population. 
  • The development and application of filters will typically require relevant and disaggregated data.For filtering techniques to be effective, the data set used must have the appropriate level of granularity and include relevant attributes for each item. The determination of whether the data set contains the appropriate level of detail about a transaction is based on auditor judgment.  
  • Appropriate procedures will need to be performed to assess the reliability of the data set being used.While not a new learning, it is important that auditors appropriately assess the reliability of the data set being used.

The takeaway from this research is that applying filtering techniques to populations to segregate transactions based on risk requires significant professional judgment about the appropriateness of the data set, the determination of the filters, the nature and extent of the procedures to apply to the filtered populations and the impact of potential errors. When appropriate auditor judgment is applied, the use of filtering techniques assists auditors when assessing the risk of material misstatement in the data set. The RADAR board therefore “encourages the profession to continue to explore the use of filtering techniques and other audit data analytics within the financial statement audit, so that we can continue to improve audit quality through a data-driven audit.”