Do Audit Data Analytics Influence Juror Perceptions of Audit Quality and Auditor Negligence?

This paper summarizes a recent study by Barr-Pulliam, Brown-Liburd, and Sanderson (2022a) (BBS, hereafter) on the joint impact of audit data analytics (ADAs) and the auditor’s report on internal control over financial reporting (ICFR) as potential signals of audit quality that may impact juror perceptions of auditor liability. Although auditors use ADAs in various contexts, BBS is one of the first studies to examine how their use affects perceptions of auditor negligence. BBS also looks at the joint use of ADAs and reports on ICFR. BBS hypothesizes that ADAs reduce perceived auditor negligence compared to traditional audit sampling methods, highlighting an important reason for auditors to expand ADA use elsewhere on audit engagements.

The SEC and Public Company Accounting Oversight Board (PCAOB) have expressed concerns that the number of material misstatements in and restatements of financial statements exceeds the number of adverse ICFR opinions, suggesting that auditors must make careful decisions when issuing ICFR opinions (Securities and Exchange Commission (SEC) 2009; Public Company Accounting Oversight Board (PCAOB) 2015). Auditors (and accounting firms generally) leverage technology that enables them to respond to the evolving risk of material misstatement while also developing deep insights into their clients’ businesses and internal control environments (e.g., Cao, Chychyla, and Stewart 2015; Barr-Pulliam, Brown-Liburd, and Munoko 2022b). Consequently, one implication of the proliferation of audit data analytics is improved audit quality. Audit practitioners and standard setters assert that leveraging technology to enable ADAs, such as integrating advanced analytics to test a population of transactions, enhances auditors’ judgments and decision-making by improving the identification of errors that may warrant further investigation (Ernst & Young 2014; KPMG 2014; PricewaterhouseCoopers (PwC) 2015; Deloitte 2016; Public Company Accounting Oversight Board (PCAOB) 2016b). Auditors use ADAs across all phases of an engagement, and ADAs can manifest in various forms. This methodological approach has implications for the nature, timing, and extent of audit testing (e.g., Brown-Liburd, Issa, and Lombardi 2015). Although auditors tout the potential benefits of ADAs for audit quality, they also caution that these benefits should not increase the actual or perceived level of assurance that an engagement provides (Ernst & Young 2014; KPMG 2014; Deloitte 2016; PwC 2016). Importantly, auditors assert that current auditing standards address the collection and evaluation of audit evidence using the traditional sampling approach (Public Company Accounting Oversight Board (PCAOB) 2016a), but provide no specific guidance on using ADAs.

BBS uses the audit of revenue and a juror setting to examine these perceptions vis-à-vis auditor liability. A jury trial is one of the few situations where auditors are compelled to reveal their methodological approach to evaluating audit evidence underlying the internal control and financial statement opinions. BBS also considers whether the auditor signaling the potential risk of material misstatement in the financial statements vis-à-vis an adverse ICFR opinion attenuates any concerns about the efficacy of ADAs.

ADAs that involve the full population can be designed to enable auditors to conduct dual-purpose tests—simultaneously evaluating internal controls and conducting substantive tests. The findings of BBS offer insights into the consequences associated with the public and private signals that auditors send to stakeholders about the reliability of a company’s financial statements. The results suggest that the ICFR opinion contextualizes jurors’ evaluation of auditors’ testing methodology. These results could impact the work of audit practitioners, regulators, standard setters, and other stakeholders related to the use of ADAs and the importance of the ICFR opinion.

The digital transformation journey differs across accounting firm characteristics (e.g., size), engagement client characteristics, and individual auditor preferences and characteristics (e.g., algorithm aversion and firm tenure). The complexity of the available technological tools and their associated implementation also varies. The general categories of data analytic tools include descriptive, diagnostic, predictive, and prescriptive analytics. Descriptive analytic tools are less complex, whereas prescriptive analytic tools are the most complex.

A recent survey of the external auditing literature suggests that ADAs used in auditing currently fall into relatively less complex descriptive and diagnostic analytics categories, such as continuous auditing and population testing (Barr-Pulliam, Brown-Liburd, and Munoko 2022b). BBS focuses on audit data analytics as defined by the American Institute of Certified Public Accountants (AICPA): “the science and art of discovering and analyzing patterns, identifying anomalies, and extracting other useful information in data underlying or related to the subject matter of an audit through analysis, modeling, and visualization for planning or performing the audit” (American Institute of Certified Public Accountants (AICPA) 2015). The specific ADA underlying the study is population testing, a combined descriptive and diagnostic analytic that uses technology to inspect the entire population of transactions or records (e.g., journal entries) underlying a company’s financial statements for predetermined rules (e.g., only accounts that are on the chart of accounts are used in the journal entry; the correct individual approves the entry), rather than the traditional sampling methods (e.g., PwC 2015; Public Company Accounting Oversight Board (PCAOB) 2018). This type of ADA is commonly used in auditing practice. It could mitigate the efficiency-effectiveness tradeoff associated with traditional sampling methods by increasing the sufficiency, or quantity, of audit evidence at a similar cost (e.g., Barr-Pulliam 2019). One example where full population testing can be applied is the examination of journal entries. Population testing minimizes sampling risk and could effectively reduce detection risk.¹ When appropriately designed and calibrated, ADAs, such as full population testing, permit auditors to potentially identify all exceptions to any predefined expectations among the population.

BBS further examines how auditors’ ICFR assessments frame jurors’ evaluations of auditors’ testing methodology in light of an audit failure. In concert with seminal research suggesting that jurors evaluate auditors’ exercise of due professional care in performing the audit (D. Causey and S. Causey 1991), the study predicts that jurors will differentially assess auditor liability based on the testing method used and the ICFR opinion issued. Auditors assert that ADAs will not increase the level of assurance they provide, but audit stakeholders could benefit from a better understanding of when methodological decisions influence audit quality perceptions.

The ICFR opinion is a public audit quality signal about a company’s internal control system. It is a prominent signal to users about financial statement reliability (e.g., Hammersley, Myers, and Shakespeare 2008; Asare et al. 2013). Learning about auditors’ testing methodology is only possible in specific contexts, such as litigation, and it is, thus, a private signal of audit quality. BBS examines the interactive effect of ADA usage and the ICFR opinion on jurors’ auditor negligence verdicts. Specifically, BBS predicts that, absent a public signal of an ineffective system of controls (i.e., an adverse ICFR opinion), jurors will be less likely to find the auditors negligent after an audit failure when auditors employ ADA techniques compared to traditional sampling. Using a testing methodology perceived to be of higher quality could mitigate auditors’ litigation exposure when they issue an unqualified ICFR opinion and there is a subsequent audit failure. When auditors warn about management’s ineffective system of internal controls and financial reporting reliability, BBS predicts that jurors will perceive management and investors (the plaintiff) as contributorily negligent and more to blame for the ensuing loss. Therefore, jurors will not differentially evaluate the effectiveness of auditors’ testing methodology, and jurors’ resulting negligence judgments will not differ by testing approach.

A juror setting best examines the interactive effects of auditor testing methodology and ICFR assessments. Absent an audit failure or additional information, such as a critical audit matter paragraph (Public Company Accounting Oversight Board (PCAOB) 2017), the auditor’s report is silent about the substantive testing methodology auditors employ. In a litigation setting, part of an auditor’s defense includes a description of the testing methodology and other steps they employed to perform a high-quality audit in accordance with auditing standards (PCAOB 2016a). Eight hundred (800) jury-eligible participants were recruited using Amazon Mechanical Turk (MTurk).² MTurk participants have been used in other accounting research focused on auditor negligence. As shown in Table 1, most participants had no prior jury experience (88.4 percent). However, as part of the participant-selection process, participant screening used typical jury eligibility criteria (e.g., 18 years of age, U.S. citizens with no accounting/finance background). Participants meeting the eligibility criteria then reviewed information about the audit process and key concepts (e.g., ICFR opinions, reasonable assurance).

The specific context is an error related to revenue recognition where auditors use journal entry testing to gather evidence about whether the sales account was materially misstated.³ The audit client, Rapid Shipping, provides courier services both domestically and internationally. Because they deliver shipments via next-day air service within the U.S. and time-definite shipping internationally, Rapid Shipping uses advanced technology to track shipments from pickup to delivery. The company records a revenue journal entry when the label for the shipment is generated. The revenue entry should be reversed if the shipment does not achieve a specific number of scans as it passes through Rapid Shipping’s processing facilities. This process is critical because customers often use Rapid Shipping-provided devices or the internet to initiate a shipment that they may not immediately or ever physically submit to a Rapid Shipping courier.

The study includes two experiments. In Experiment 1, the 800 participants were randomly assigned to one of four experimental groups created by manipulating the auditor’s testing methodology (ADAs versus traditional sampling) and the ICFR opinion issued (unqualified versus adverse). Participants reported their assessments of auditor negligence and perceptions of auditor intent to perform a high-quality audit. They separately reported how they allocated blame for the audit failure among auditors (the defendant), management, and investors (the plaintiff). In Experiment 2, participants (108 additional jury-eligible participants) received a summary of the revenue recognition process described in the Rapid Shipping case used in the first experiment.⁴ However, the 108 participants only received one of the two testing methodologies (ADAs versus traditional sampling) with no other contextual factors. The purpose of this experiment is to attempt to disentangle perceptions of auditors’ testing methods from those related to an audit failure. Participants provided ratings for four measures capturing perceptions of (1) the extent of reliance on technology, (2) the sufficiency of the audit evidence obtained, (3) the quality of the audit testing methodology, and (4) the likelihood that the testing methodology would identify any existing material misstatement. Participants also shared their perceptions of the extent to which auditors tested revenue journal entries.

Consistent with BBS’s predictions, Experiment 1 results show that the ICFR opinion influences jurors’ negligence verdicts. When the ICFR opinion is adverse, jurors are less likely to find the auditor negligent. The results show a direct effect of the audit testing methodology on perceptions of auditor negligence, but the interactive effect provides an interesting context. The results suggest that when the auditor issues an unqualified ICFR opinion, jurors are more likely to assess auditors as negligent when they use traditional sampling techniques versus ADAs. Nevertheless, the audit testing methodology has a marginally insignificant effect on negligence assessments when the auditor issues an adverse ICFR opinion (see Figure 1).

Results of Experiment 2 provide evidence that jurors perceive auditors’ use of ADAs, compared with traditional sampling, as utilizing sophisticated technology to conduct audit tests beyond the mere examination of a more significant number of transactions (i.e., an entire transaction population versus a sample). Notwithstanding, the expectation that auditors will identify a material misstatement is no different whether they use ADAs or traditional sampling. This latter finding supports auditors’ assertion that an increase in perceived audit quality associated with using ADAs should not be associated with a perception that auditors provide more than reasonable assurance (Public Company Accounting Oversight Board (PCAOB) 2007).

In a series of supplemental analyses, the study first shows that when auditors issue an unqualified ICFR opinion, jurors evaluate auditors’ use of ADAs as a signal of auditors’ intent to conduct a higher-quality audit. This finding is encouraging for audit practitioners, and the perception mediates the effect of the auditor’s testing methodology on jurors’ negligence assessments (see Figure 2). However, when auditors issue an adverse ICFR opinion, jurors do not differentially evaluate auditors’ intention to conduct a quality audit vis-à-vis the testing approach (not tabulated). Second, jurors perceive that auditors exercise a higher standard of due professional care when using ADAs versus traditional sampling. This perception mediates the effect of the auditor’s testing methodology on jurors’ negligence assessments, particularly when auditors issue an unqualified ICFR opinion (not tabulated). In an extension of prior research (e.g., Wu and Tuttle 2014), results show that the public signal provided by the ICFR opinion and knowledge of the private signal provided by the auditor’s testing methodology not only reduces the likelihood that auditors are deemed negligent, but also increases the likelihood that jurors ascribe blame to management and investors after an audit failure (see Figure 3, Panels A and B).

The findings of BBS have implications for U.S. (e.g., the PCAOB) and international (e.g., the IAASB and Financial Reporting Council) regulators and audit standards setters. These organizations have current initiatives that focus on understanding the real effects of digital transformation on the audit, identifying audit quality indicators, and enhancing auditing standards to promote innovation in accounting firms (e.g., Barr-Pulliam, Brown-Liburd, and Munoko 2022b; Barr-Pulliam and Carlson 2022). BBS emphasizes the negative implications associated with auditors using traditional audit methodologies. The results also suggest that the ICFR opinion is a critical factor in juror perceptions regarding legal liability, placing further onus on auditors to determine the most appropriate opinion. This result is consistent with prior research (e.g., Wu and Tuttle 2014). Further, the study uses revenue recognition as the context. Revenue recognition is where standards setters continue to evaluate the post-implementation of the financial reporting guidance in ASC 606. The findings suggest that auditors could leverage technology to improve the precision and accuracy of revenue reported in the financial statements.

This study lends credence to the notion that utilizing ADAs enhances audit quality perceptions and can mitigate litigation exposure in specific circumstances (e.g., when auditors issue an unqualified ICFR opinion). The results of this study align with audit firm statements that the use of ADAs should not increase the level of perceived assurance that auditors can provide in a particular engagement; therefore, practitioners may want to consider whether there are other opportunities to employ ADAs for dual-purpose tests. BBS focuses on revenue (e.g., how shipping transactions are applied to the sales account) using a relatively less complex type of ADA—full population testing. One concern auditors typically raise with such testing is the potential for an excessive number of exceptions requiring follow-up (e.g., Brown-Liburd et al. 2015). Auditors could utilize the output of descriptive analytics to enable diagnostic analytics that help answer the question: “Why did it happen?” Auditors could further examine all or a sample of these exceptions based on available audit resources. These combined analytics would permit auditors to focus on higher-risk transactions that are more likely to increase the risk of material misstatement. In light of regulators’ focus on riskier transactions (e.g., the risk-based approach used by PCAOB inspectors), auditors should also carefully consider how they might use these analytics to calibrate the public signal to financial statement users that the ICFR opinion provides.