Comments of the Auditing Standards Committee of the Auditing Section of the American Accounting Association on Proposed Statement on Auditing Standards (SAS), Audit Evidence Open Access

We commend the Auditing Standards Board (the Board) on its actions to modify the Audit Evidence standard to address the evolving nature of business and audit services and issues that have arisen during its standard-setting activities including use of emerging technologies by both preparers and auditors, audit data analytics (ADA), the application of professional skepticism, and the expanding use of external information sources as audit evidence, and more broadly, the accuracy, completeness, and reliability of audit evidence which have resulted in the publication of the Exposure Draft, Proposed Statement on Auditing Standards (SAS), Audit Evidence (the proposed SAS).

Overall, we believe the proposed enhancements are necessary and substantially achieve the Board's intent; however, we have concerns regarding some definitions (or lack thereof) and the sufficiency of guidance and examples in the application of professional skepticism and emerging technologies. Our concerns are detailed below in our responses to Questions 2 through 5, and 7 through 12 of the proposed SAS.

• Question 2:

  If implemented, would the new requirements and application material assist the auditor in more effectively evaluating whether sufficient appropriate audit evidence has been obtained? If not, please explain why.

While some elements of relevance and reliability are improved by the proposed SAS (e.g., the definition of internal versus external information), others are less clear. For example, paragraphs A66 through A68 provide a good discussion of how ADA might be used as part of audit evidence. However, the examples may lead auditors to believe that visualization can only be used in risk assessment and data analytics can only be used when testing 100 percent of the population. Alternative examples where data visualization could be used for substantive tests or where data analytics can be used for testing less than 100 percent of the population would be useful to clarify that data analytics can be applied more broadly than what is presented in the examples.

The language in paragraph 4a is confusing and should be clarified. Paragraph 4a: “Whether the assessment of the risks of material misstatement at the relevant assertion level remains appropriate.” There are two possible interpretations of this sentence: (1) whether the original assessment of the risks of material misstatement at the relevant assertion level remains appropriate, or (2) whether the auditor even needs to conduct an assessment of the risks of material misstatement at the relevant assertion level.

Paragraph 10a cites the importance of evaluating accuracy and completeness of the information. Paragraph 10b discusses whether the information is sufficiently precise and detailed. In the cube diagram, accuracy and completeness are included indicating their importance. However, “precise” and “detailed” are not included in the cube. The latter terms get the same prominence as accuracy and completeness in paragraph 10 but not in the cube. This could lead to confusion about the importance of these attributes of relevance and reliability. Also, paragraph 10b refers the reader to paragraphs A12 through A33; however, in the proposed SAS the terms are only briefly mentioned in paragraph A25. There is more discussion in paragraphs A67 through A69 about precise and detailed evidence, but this is not referenced in paragraph 10b.

Using authenticity as a component of the cube is confusing given the discussion in paragraph A26. The cube implies that the auditor has a duty to ensure evidence is authentic; however, paragraph A26 specifically states, “AU-C Section 240 further explains that an audit performed in accordance with GAAS rarely involves the authentication of documents, nor is the auditor trained as, or expected to be, an expert in such authentication.” The conflicting messages in the cube and paragraph A26 could create ambiguity regarding the auditor's responsibility for determining the authenticity of audit evidence.

In the cube, the risk of bias is discussed. In paragraph A29, the standard recognizes that the bias may arise from the “development of information itself or may exist due to interpretation of the information by the entity or the auditor.” In the development of information, the bias can come from either management or the auditor. For relevance and reliability, bias that needs to be discussed relates to biases related to the development of the information such as availability bias. The importance of distinguishing judgment biases is that research has identified debiasing techniques that the auditor may employ. For example, Hall, Herron, Pierce, and Witt (2001) find selection bias in haphazard sampling is reduced by increasing the sample size. Also, avoiding starting sample selection on the first page of control listings and proceeding through control listings in a serial fashion could mitigate sample selection bias caused by visual attributes of listed information (i.e., page location, line location, listing characteristics) (Hall, Higson, Pierce, Price, and Skousen 2012).

• Question 3:

  Would the proposed attributes and factors expand the types and sources of information considered by the auditor as audit evidence by lessening the emphasis on how audit evidence is obtained (that is, “audit procedures performed”)? If not, please explain why.

We believe that the proposed SAS successfully shifts from emphasizing “how” to gather audit evidence (procedures) to “the assessment” of the audit evidence. We suggest examples be integrated to illustrate sources more commonly encountered with certain audit evidence gathering procedures. For instance, information generated from management's internal financial reporting system (e.g., purchase orders, sales invoices, trial balances) may be used in audit procedures such as inspection, recalculation, and analytical procedures. Additionally, information obtained by the auditor from external sources (e.g., debt agreements, vendor statements) may be used in the confirmation process. Illustrations such as these can provide the auditor with a clear linkage of the information sources to the audit procedures performed.

• Question 4:

  Are there relevant attributes and factors of audit evidence missing from the proposed SAS that should be considered by the auditor when evaluating the appropriateness of audit evidence? If so, please describe them.

We believe that the relevant attributes are included in the proposed SAS, but as stated in our response to Question 2 above, sometimes they are not discussed explicitly or in sufficient detail. For example, for relevance and reliability in paragraph 10, there is mention of accuracy and completeness and whether the information is precise and detailed. However, no mention is made of authenticity or bias as the cube diagram implies. These items are not discussed until paragraphs A26 and A28, respectively.

Timeliness is included in the Financial Accounting Standards Board's (FASB) definition of relevance but is missing from the definition in the proposed SAS. In addition, the auditing standards discuss when evidence is collected (e.g., interim versus year-end) and the Public Company Accounting Oversight Board's (PCAOB) definition of relevance (AS 1105, paragraph 7a) mentions timeliness as a consideration of relevance. The Board should consider whether the timeliness concept needs to be included in the discussion and definition of relevance in the proposed SAS.

The PCAOB definition of reliability states: “The reliability of information generated internally by the company is increased when the company's controls over that information are effective” (AS 1105, paragraph 8). Ergo, sufficiency is defined by the PCAOB, in part, by the reliability of the information. In the proposed SAS, sufficiency is related to persuasiveness, which is influenced by risk of material misstatement (RMM). As stated in the audit risk model, RMM is a function of control and inherent risk. The Board's definitions do not clearly make the connection between reliability, sufficiency, and the quality of internal controls. If the Board is trying to say that sufficiency is related to the quality of controls, it needs to be stated more explicitly. In the proposed definition of sufficiency, it is not clear how RMM, or its elements, tie to sufficiency.

• Question 5:

  Does the diagram in the proposed SAS appropriately depict the attributes and factors that the auditor considers in evaluating whether sufficient appropriate audit evidence has been obtained?

We find the cube diagram to be confusing and the related definitions raise several questions, including:

• How precisely are the four components (accuracy, completeness, authenticity, and risk of bias) related to relevance and reliability? For example, do all four components relate to both relevance and reliability, or do some of the four components map into only relevance, only reliability, or both? Should authenticity and risk of bias be sub-categories of reliability only? Importantly, in paragraph A18, it appears that all four components relate only to reliability, which appears inconsistent with the cube diagram showing these as relating to relevance and reliability.

• What are the definitions of the four components—accuracy, completeness, authenticity, and risk of bias? Should all terms in the diagram be defined in one place and clearly linked to each other? Specifically, the cube is introduced in paragraph 5, but the definitions provided in paragraph 8 and paragraphs A5 through A8 do not seem to define most of the terms in the cube (and they include definitions of other terms that do not appear in the cube). Should readers be looking elsewhere for definitions of all terms in the cube?

• Is there potential for confusion when using accuracy and completeness in this context given that these also are management assertions? This is particularly concerning when the PCAOB in its definition of relevance states the following: “The relevance of audit evidence refers to its relationship to the assertion or to the objective of the control being tested” (AS 1105, paragraph 7). So in one respect, there is a circular definition to relevance since the two components of the assertions are used in the definitions.

• What are the elements of persuasiveness? Are they the sources, corroborative, and contradictory elements in the diagram? We believe that “persuasiveness” needs to be defined using the elements in the diagram.

• Should “persuasiveness” be added to the diagram since it appears to be a fundamental construct?

• For consistency with other sections of the proposed SAS, should “sources external to the entity” simply be called “external information sources?”

• Should “appropriateness” and “sufficiency” be added to the diagram to help auditors fully assess whether evidence is appropriate and sufficient? This likely would require adding the assessment of the risk of material misstatement (RMM) and, it appears, the nature of audit work done above as well.

• As discussed in our response to Question 4, the concept of “sufficiency” seems significantly underdeveloped in the proposed SAS, and the definition of sufficiency given in the proposed SAS does not appear to align with the PCAOB standard (AS 1105, paragraph 5). This could result in inconsistent application across public company and private company audits.

Overall, there are many terms presented in the cube diagram and surrounding discussion, but most of the terms are not explicitly defined in paragraphs 8 and A5 through A8. Also, in many cases, it is unclear how the various terms in the cube and outside the cube are related to each other. The cube diagram and related text could be significantly enhanced to (1) clearly define all terms in one place and (2) clearly establish the linkages among terms, ultimately flowing up to “appropriate sufficient evidence.”

• Question 7:

  Do you agree with the approach taken by the ASB in addressing the topic of professional skepticism? If not, please explain why.

We agree with the approach taken by the Board, particularly with regards to interweaving the concepts related to professional skepticism throughout the proposed SAS (rather than merely using the words “professional skepticism”) and providing examples of activities auditors can undertake to demonstrate professional skepticism. However, we believe there is still a lack of meaningful guidance on the application of professional skepticism. We recommend that the proposed SAS include more specific guidance on what is to be measured and how professional skepticism can be applied. Although paragraph A3 references AU-C 200, Overall Objectives of the Independent Auditor and the Conduct of an Audit in Accordance with Generally Accepted Auditing Standards, which includes more details on professional skepticism, we suggest including more guidance in the proposed SAS. We also suggest modifying the wording in paragraph 4 to explicitly mention the scale or benchmark on which the auditor's evaluation must be based. Consistent with the findings of Nolder and Kadous (2018), this should enhance understandability and increase the application of professional skepticism. Specifically, we suggest modifying paragraph A3 to end as follows:

• whether the assessment of the risks of material misstatement at the relevant assertion level remains appropriate

• the nature, timing, and extent of the audit procedures performed as appropriate in response to risks at both the account level and disclosure level

• the audit evidence obtained from those procedures in accordance with this proposed SAS (Ref: A3) by considering the sources, relevance, and reliability of the information used as well as whether the information corroborates or contradicts the assertions in the financial statements.

• Question 8:

  If the guidance in the proposed SAS is implemented, would the application of professional skepticism be enhanced and more clearly understood in evaluating whether the sufficient appropriate evidence has been obtained? If not, please explain why.

The guidance instructs auditors to consider the risk of bias in both management's and the auditor's professional judgments. Research has demonstrated that generally prompts and similar attempts to influence auditor mindset positively impact professional skepticism (see for example, Nelson 2009; Hurtt, Brown-Liburd, Earley, and Krishnamoorthy 2013; Brazel and Schaefer 2017). Paragraph A33 of the proposed SAS describes four common decision biases. To increase auditors' understanding of these biases, we suggest including familiar examples in addition to the descriptions of what the bias entails. For example, confirmation bias may result in auditors accepting weak explanations from clients or inappropriately excluding errors or outliers from the results of a sample selected for testing. Overconfidence bias may result in auditors including too few items to sample—particularly in small populations (e.g., testing monthly reconciliations). Anchoring may result in auditors assuming there is no error in current year account balances when those balances approximate prior year account balances, but should not (e.g., the loss of a major customer should have resulted in a decrease in revenues and receivables).

We suggest A33 include language that requires the auditor to affirmatively consider the extent to which their specific judgments might be biased. For example, the guidance could direct auditors to consider and document the extent to which evidence collection was driven by available evidence rather than the most persuasive evidence. Another example of additional guidance includes directing auditors to consider the extent to which they sought confirming evidence as compared to disconfirming evidence.

Given the importance of applying the concept of professional skepticism throughout the audit and consistent with our recommendation to provide more guidance on applying professional skepticism, we believe incorporating references to other professional standards, where applicable, may serve to enhance the application of professional skepticism. For example, AU-C 230, Audit Documentation, directs auditors who identify evidence that is inconsistent with their final conclusions regarding a significant finding or issue to document how the inconsistency was addressed (AU-C 230.12 and A17-A18).

• Question 9:

  Are the changes to the definitions in extant AU-C Section 500 appropriate? If not, please explain why.

Referring to the background materials on page 6 of the proposed SAS, auditors have raised questions about (1) how they can clearly demonstrate professional skepticism, (2) how they can better describe the basis of their judgments, and (3) how their mindset played a role in performance.

Auditors' evaluative beliefs about the appropriateness and sufficiency of audit evidence serve as proxies for auditors' attitude of professional skepticism (Nolder and Kadous 2018). However, as discussed in Questions 4 and 5, the definitions of appropriate and sufficient audit evidence may better respond to auditors' questions if they included more explicit guidance. Below are the definitions with our suggestions italicized.

• Appropriateness (of audit evidence). The measure of the relevance and reliability of audit evidence in relation to supporting or refuting an assertion.

• Sufficiency (of audit evidence). The measure of the persuasiveness of audit evidence in responding to an assertion. The persuasiveness of audit evidence necessary is affected by the auditor's assessment of the risks of material misstatements.

• Question 10:

  Are there any other definitions that should be included in the proposed SAS? If so, describe them.

As noted in our response to Question 5 above, it appears that many terms in the cube diagram are not explicitly defined when the cube diagram is introduced (see paragraphs 5 and 8). For example, there is discussion of different types of bias, but no explicit definition of bias. Consider adding a definition of bias in the proposed SAS. Further, many biases are unconscious, and thus especially problematic for auditors to address. Our response to Question 8 may help to potentially address this issue.

Has the Board considered the extent to which the various definitions of terms are consistent or inconsistent with definitions provided by the PCAOB, FASB, and the International Auditing and Assurance Standards Board (IAASB), etc.? If there are inconsistencies between ASB definitions and those of other standard setters, are these differences intentional?

For example, in the definition of “sufficiency,” the proposed SAS states, “The persuasiveness of audit evidence necessary is affected by the auditor's assessment of the risks of material misstatement” (Ref: paragraph A5). In addition to the proposed changes outlined in our response to Question 9, we believe that the definition in the proposed SAS needs to be reframed to state that as RMM increases, the persuasiveness of audit evidence should increase. Both the IAASB and PCAOB mention the idea of RMM and the quality and quantity of information, but they do not mention persuasiveness per se.

• Question 11:

  Please provide your views on whether (a) the guidance added to the application material of the proposed SAS to explain the implications and role of automated tools and techniques in the current audit environment is beneficial and (b) the proposed SAS is enhanced by using illustrations of automated tools and techniques; that is, whether the proposed SAS is more relevant to audits conducted in today's environment.

We support the inclusion of guidance to the application material to explain the implications and role of automated tools and techniques. Consistent with research in psychology and management science (e.g., Eastwood, Snook, and Luther 2012; Dietvorst, Simmons, and Massey 2015; Yeomans, Shah, Mullainathan, and Kleinberg 2017), emerging studies in auditing and accounting find auditors rely more on their own firm's evidence when it comes from a human specialist rather than an artificial intelligence system (Commerford, Dennis, Joe, and Wang 2019). Additionally, audit firms are investing heavily in advanced cognitive technologies to assist with evidence gathering activities traditionally performed by human specialists (e.g., Alliott Group 2017).

We agree that the proposed SAS is enhanced by the illustrations included, but it could benefit from even more concrete examples. For example, in paragraph A27 it may be helpful to describe that data analytics aggregating/analyzing/visualizing is a type of information transfer or form of information. Paragraph A51 says, “In some instances, the auditor may achieve the objective of more than one type of audit procedure (for example, both a risk assessment and further audit procedure) simultaneously by using automated tools and techniques.” It is unclear what is intended. As such, clarity would be enhanced by including specific examples or additional guidance.

Clarity and understanding would also be enhanced by deleting “such as audit data analytics” from paragraph A56, modifying paragraph A58 to, “An example of an inspection procedure using an automated technique is the use of text-recognition programs to examine large populations of documents such as contracts to identify items for further audit consideration,” and revising paragraph A60 to, “Automated tools and techniques may aid the auditor in performing an observation procedure. For example, a camera mounted on a drone may aid the auditor in performing an observation procedure on management's physical inventory count.”

In addition, although automated tools and techniques may enhance audit efficiency, recent evidence suggests that overreliance on technology-enabled audit processes and tests may negatively impact professional skepticism (Boland, Daugherty, and Dickins 2019). Given concerns about the potential for auditors' overreliance on technology and consistent with recently expressed regulatory concerns in the United States (e.g., Harris 2017), we suggest also including a note of caution, in the proposed SAS, for auditors when using technology-enabled audit processes.

• Question 12:

  Do you agree that AU-C Section 330 combined with the attributes and factors in the proposed SAS would assist the auditor in concluding whether an oral confirmation should be supplemented by a written confirmation of the information?

Today, written communications are facilitated electronically by email, texting, and digital signatures, making the process of collecting written evidence less onerous than in the past. Electronic communications also make it easier to validate that confirmations were sent to and returned by the appropriate individual—see criteria specified in AU-C Section 505 for oral confirmations. Further, research finds electronic confirmations help address some of the failures of written confirmations collected as evidence of the existence assertion for cash and accounts receivable (Aldhizer and Cashell 2006).

Although we believe the proposed SAS will assist auditors in determining whether an oral confirmation should be supplemented by written confirmation, we are concerned that positioning oral confirmations as generally permissible, as opposed to only appropriate in certain circumstances, could be a slippery slope. The proposed SAS requires that “the auditor obtain more persuasive audit evidence the higher the auditor's assessment of risk.” In the extreme, this could be interpreted as no written confirmations are required, even for material uncertainties such as confirmations from an auditee's legal counsel, when the auditor judges an auditee to be low risk. To reduce ambiguity, we recommend that specific examples be included describing specific circumstances when oral confirmations would not be acceptable. Several instances come to mind including management representations, representations of legal counsel, compliance with laws, regulations, or the terms of material agreements, and, because AU-C Section 240.26 requires audits be planned “based on a presumption that risks of fraud exist in revenue recognition,” the existence and terms of revenue transactions.

We are also concerned that the proposed SAS omits details about how an oral confirmation should be documented. For example, former paragraph A27 described: “In appropriately documenting the oral response, the auditor may include specific details, such as the identity of the person from whom the response was received, his or her position, and the date and time of the conversation.” We appreciate that by removing this language the proposed SAS more-closely aligns with similar standards of the IAASB and the PCAOB, however, this may not be in the best interest of auditors. The failure to adequately authenticate confirmation responses and identify collusion between management and customers has been the subject of Accounting and Auditing Enforcement Releases (Caster, Elder, and Janvrin 2008). One role of the standards is to promote audit quality, which could be accomplished by educating practitioners on the proper way to document an oral confirmation.

• Question 1:

  Please provide your views on whether the revised scope section of the proposed SAS clearly explains the relationship between the proposed SAS and other AU-C Sections, including AU-C Sections 315, 330, and 700. If the scope section does not clearly explain the relationship, please indicate why.

• Question 6:

  Please provide your views on whether the examples in the proposed SAS are useful to auditors. If the examples are not useful, please explain why.

• Question 13:

  Is relocation of the content dealing with management's specialist from AU-C Section 500 to AU-C Section 501 or to a separate new standard appropriate? If not, please explain why.

• Question 14:

  If you agree that relocation is appropriate, what are your views about whether the management's specialist content should be addressed in AU-C Section 501 or in a separate new standard?

• Question 15:

  Do you believe that the application of this proposed SAS would result in audit documentation requirements beyond those in AU-C Section 230 and other AU-C Sections? If so, describe how the proposed SAS is perceived to expand the audit documentation requirements existing in AU-C Section 230 and other AU-C Sections.