Systems-Thinking in Complex Audit Situations Free

Public accounting practitioners and academic researchers have long recognized the importance of an accountant's ability to “think critically” regarding the complex, but often unstructured, judgments necessary in the auditing profession. In fact, accounting firms have devoted entire publications describing the need for both experienced auditors and young professionals to understand the complexities of how their clients interact with their industry and overall economy (Bell, Marrs, Solomon, and Thomas 1997; Bell, Peecher, and Solomon 2005; Managing Partners of the Big Eight Accounting Firms 1989; PwC 2003). Despite this need for critical thinking skills, there has been little guidance regarding the type of skills needed and how one should obtain such skills.

Over 20 years ago, Baril, Cunningham, Fordham, Gardner, and Wolcott (1998) interviewed public accounting professionals to understand how they defined critical thinking. Not surprisingly, practitioners had widely varying views on how to define the concept. Practitioners generally seem to use the term critical thinking to describe undefined characteristics that lead to quality judgments and decision making. There were several characteristics, however, that arose repeatedly, including ability to see the “big picture,” knowing when more information is needed, and the ability to apply knowledge gained in one situation to a separate, unrelated situation. Most of these definitions of critical thinking have, however, been at the conceptual level—they describe the results of critical thinking or what critical thinking might “look like,” but have not generally identified a methodology to implement critical thinking.

We summarize a series of three recent accounting studies examining an implementable critical thinking method that incorporates a defined set of skills. Systems-thinking, taking a holistic view of an environment while still being able to focus on how the components of the environment work together, has been developed and successfully practiced in other disciplines. The three auditing studies summarized here are: “How a Systems Perspective Improves Knowledge Acquisition and Performance in Analytical Procedures” (Brewster 2011), “The Effect of Client Lies on Auditor Memory Resistance and False Memory Acceptance” (Brewster 2016), and “Enhancing Auditors' Critical Thinking in Audits of Complex Estimates” (Bucaro 2019). Together, they examine how systems-thinking skills enhance the way individuals acquire and build knowledge that ultimately improves their information processing and audit decisions.

Management scientists suggest that systems-thinking is one critical thinking method that could improve decision making in complex business settings. Meadows (2008, 2) describes a system as “a set of things—people, cells, molecules, or whatever—interconnected in such a way that they produce their own pattern of behavior over time.” So systems-thinking is a method that enhances people's use of their own cognitive abilities to adopt a “big picture” perspective that focuses on understanding how the complex processes comprising the multiple parts of a system (business, natural, or otherwise) interact and function as a whole.¹ A critical part of this perspective, therefore, is understanding how the often multi-directional processes and various parts interact and work together to produce an output (Bucaro 2019, 38; Meadows 2008).

For a biologist, a natural system's output could be the changing population pattern of a particular species over time. For an accountant or auditor, the output of a business system could be the fluctuation of inventory over time. The auditor must obtain an understanding of the immediate factors influencing inventory inflows (purchases, freight costs, available supply) and outflows (sales, shrinkage). More advanced analysis, such as inventory impairment, would also need to incorporate qualitative inflows (competitor products, potential regulation) from industry and economic events. The auditor must also understand the volume of each inflow/outflow and how this volume changes over time. An inability to accurately predict the time-series movement of inventory means that the auditor's expectations could be incorrect, potentially reducing the effectiveness of inventory-related audit procedures. In sum, the auditor views the client as one component of a larger “economic system,” and accurately understanding the complex processes that lead to inventory changes requires understanding the forces both inside and outside the client's environment that affect the account (Bell et al. 1997).

However, research suggests that even experienced decision makers struggle when faced with such dynamically complex tasks (Cronin, Gonzalez, and Sterman 2009; Diehl and Sterman 1995). This is because the complexity of modern business environments overwhelms individuals' abilities to process information accurately. Complex systems include circular feedback loops and time delays that are often difficult to understand. Understanding such feedback loops requires the individual to recognize multiple causal relationships and how the strength of these relationships changes over time (e.g., time delays). The result is that decision makers fail to recognize nonlinear movements of key system components (account balances or cost drivers), and how the complexity of such systems affects their decisions.

Individuals taking a systems-thinking view of a phenomena (e.g., a client industry) will have knowledge that is organized in a way that more closely resembles the actual structure of the “real world” (Clark, Nguyen, and Sweller 2006).² The improved knowledge, information processing ability, and overall judgments resulting from systems-thinking can be applied to a variety of tasks. Hence, systems-thinking is not specific to any particular audit methodology, and it is not limited to any single audit step. Instead, it is a way to enhance a decision maker's critical thinking skills across the entire audit process. Developing systems-thinking skills provides multiple long-term advantages, including the ability to critique the deficiencies in one's own knowledge and understanding of a problem (Richmond 2000). Additionally, since basic system structures are often repeated throughout nature, economics, and biology, systems-thinkers can better transfer their understanding of one context—like a business process—to other contexts—like the impact of information on investors' likely decisions (Meadows 2008; Richmond 2004; Ruth and Hannon 2012).

Traditionally, management scientists thought systems-thinking skills could only develop over time with extensive training, and researchers did little to examine them in an experimental laboratory setting. The three studies we summarize here, however, successfully prompted auditors to use systems-thinking skills in experimental laboratory tasks. Participants were upper-level accounting students (Brewster 2011) and professional auditors (Brewster 2016; Bucaro 2019). In this research, certain participants (hereafter, systems-thinkers) received a brief training tutorial that emphasized system dynamics concepts using general nonbusiness topics (e.g., blowing up a balloon or filling a bathtub) and/or business topics unrelated to the audit issue to which they would soon be exposed. The tutorials emphasized taking a holistic approach to a problem and emphasized how multiple causal relationships can interact to affect the rates and levels of important components. The remaining participants (hereafter, reductionist-thinkers) were required to learn about the client and industry using only their own intuition or guided only by professional standards. All participants then drew diagrams of certain processes in order to instill the respective thinking perspectives in their minds; they then responded to case materials based on fictitious companies. The tutorials should allow the systems-thinking participants to develop knowledge about the companies and industries, including the complex aspects (i.e., feedback loops, time delays, multiple causality) of a system that are often difficult to learn. All participants in each study were exposed to identical study-specific client and industry information. They made audit-related judgments including whether and how inherent and control risks affect processes and financial statements in a complex environment (Brewster 2011; Bucaro 2019), whether management is credible and their information is reliable (Brewster 2011, 2016), and whether large potential financial misstatements are likely to impact investors' decisions when reason suggests that the misstatements should have no impact (Bucaro 2019).

The three studies summarized above provide evidence that adopting a systems-thinking perspective can improve auditors' decisions across the various stages of the financial statement audit, from planning to concluding. If the audit is viewed as an iterative process designed to reduce the risk of material misstatement and detection risk to acceptable levels (e.g., Bell et al. 2005; PCAOB 2010a, 2010b), then systems-thinking improves risk assessment by allowing auditors to better (1) understand how different risk factors affect business processes and financial reporting, (2) incorporate these factors into the audit risk model and substantive procedures, and (3) recognize whether actual audit differences are likely to be material to financial statement users.

One of the greatest challenges that an auditor faces is understanding how macro-economic or industry-based events can flow down to affect a client's risk of material misstatement through its inherent risk component (Bell et al. 1997, 23). Brewster (2011) examined methods for improving how auditors analyze a complex client industry and then build knowledge that accurately reflects that industry. In this study, upper-level accounting students analyzed the operating environment for a fictional fast food restaurant. Results show that systems-thinking participants are better able to understand how business risks in the client industry ultimately affect the client's financial statements. Systems-thinkers identified more causal linkages between the industry forces and the financial statements, and they did it with less effort than reductionist-thinking participants. This result—better task-performance with less effort—indicates improved information processing resulting from more organized knowledge that is more easily accessible from memory (Clark et al. 2006).

Bucaro (2019) builds on the previous study by showing that professional auditors using systems-thinking skills can better incorporate qualitative risk factors into risk-based judgments about a complex estimate, warranty reserve. The emphasis on an unstructured, complex accounting estimate is important since many important risk factors cannot be easily quantified—auditors need to also evaluate qualitative factors when reaching audit conclusions (e.g., Knechel, Krishnan, Pevzner, Shefchik, and Velury 2013; PCAOB 2001, 2010d). The study evaluates the effect of systems-thinking on how well auditors incorporate one such qualitative factor, the complexity of the company's estimation process, into their inherent risk assessments. This is an important result as it provides direct evidence that systems-thinking skills improve auditors' likelihood of incorporating complex risk factors into their assessments of the risk of material misstatement. And while this study evaluated auditors' risk assessments for a specific account, warranty reserve, it is not hard to see how this same thinking perspective might lead to improved risk identification in other stages of the audit process, like client acceptance and client continuance, when inherent risk can be particularly meaningful.

Systems-thinking skills can also improve how auditors implement substantive procedures, particularly analytical procedures. Brewster had both professional auditors (Brewster 2016) and student participants (Brewster 2011) evaluate a company's industry and then develop expectations for how a certain raw material cost (beef prices) changed over time. The key facts of the case suggested that beef prices should rise and then fall midway through the audit period. Theory and prior research suggest that it should have been difficult for the participants to recognize this nonlinear movement. Systems-thinkers, though, provided expectations for beef prices that recognized the fluctuating nonlinear movement. Reductionist-thinkers provided beef price estimates that incorrectly suggested that prices increased continually during the period. These results suggest that better organized knowledge, resulting from systems-thinking, enhanced auditors' likelihood of developing more accurate expectations of actual financial statement balances tested during substantive procedures.

Brewster (2011) next examined how the novice auditor participants used these improved expectations to evaluate management's explanations when performing analytical procedures. In particular, how the participants evaluated management's credibility when the explanations are inconsistent with industry evidence. This is an important audit judgment during the substantive testing process, as perceptions of management credibility are likely to influence auditors' skepticism (e.g., Goodwin 1999; Knechel and Messier 1990). Recall that participants were asked to determine the raw material costs (beef prices) based on a set of complex facts and circumstances, and that the estimates of systems-thinkers were more accurate than those of reductionist-thinkers. Results show that systems-thinkers were better able to identify the incorrect management explanations than were reductionist-thinkers, and lowered their evaluations of management credibility accordingly.

Brewster (2016) replicated these results using professional auditors. Additionally, auditors were told a persuasive lie intended to deceive the auditor. Systems-thinkers were able to resist management's persuasion attempt by recalling the correct industry facts. In contrast, reductionist-thinkers, although they received exactly the same case information as the systems-thinkers, appeared to struggle with resisting management's persuasion attempt, because they were more likely to recall management's lie instead of the known industry facts.

Finally, professional standards state that auditors must assess the risk of material misstatement during planning, but must also update these initial assessments in light of new information throughout the audit (PCAOB 2010c). Auditors must evaluate whether any potential audit differences are actually material to financial statement users, requiring a risk assessment modification. After linking systems-thinking to a better understanding of the role of process complexity in assessing inherent risk, Bucaro (2019) assessed the effects of systems-thinking on auditors' ultimate determination of whether a potential misstatement is considered material. The study found that when systems-thinking auditors are prompted to exercise professional judgment, they are more likely to appropriately consider the qualitative characteristic (i.e., the complexity of the process itself) when deciding whether a misstatement is material to the financial statement users. In other words, systems-thinking auditors thought critically about the judgment as opposed to resorting to a more mechanistic decision (i.e., simply comparing the audit difference to tolerable misstatement). In contrast, reductionist-thinkers exhibited no indication of applying critical thinking when determining the effect of the misstatement.

Critical thinking, as a concept, has been a part of the audit professional's dialog for decades. The three studies described above show recent scientific evidence of a specific, implementable method to achieve results that practitioners seek. Critical thinking has been defined as the ability to understand how local actions impact global outcomes, to view problems as components of an overall system rather than surface outcomes or events, and to see and understand complex relationships and how they fit into an overall ecosystem (Valerdi and Rouse 2010). But these are outcomes, not skills that can be taught and implemented. The primary takeaway of the three summarized studies is that teaching specific systems-thinking skills can have measurable effects on actual decisions across the continuum of the audit.

With minimal training via short tutorials, auditors were able to make more thoughtful and complete decisions in several audit contexts. Systems-thinking participants included a more comprehensive set of information in their risk assessment decisions, which can impact client acceptance and continuance as well as ongoing and iterative risk assessments throughout the audit. Although not specifically tested in the three studies, better risk assessments can lead to better planned and therefore more effective substantive testing. The summarized studies did evaluate substantive analytical procedures and found that systems-thinking participants were better able to identify mistakes and even lies told by management, suggesting systems-thinkers may be more appropriately skeptical during the substantive testing process. Finally, systems-thinking auditors were able to incorporate their more advanced understanding of industry complexity into what is arguably the most critical audit decision, materiality judgments.

While the studies described here suggest evidence of improved decisions in complex accounting scenarios, it is unlikely that systems-thinking will improve every decision. For example, less complex financial statement accounts, or those that require less estimation or speculation, may be better evaluated in a linear, reductionist-thinking manner. An example of this might be physical inventory counts (although, arguably, deeper thought into the business might still lead to better valuations of, for example, obsolete inventory). Therefore, as academic researchers and professionals continue to study this critical thinking method, they should identify situations and contexts in which systems-thinking improves results and those in which it does not.

These studies have also shown, in contrast to most previous research, that the skills can be instilled through the use of short explanatory tutorial statements and graphics that can be introduced into firm documentation templates. Practitioners interested in achieving critical thinking outcomes might consider introducing some combination of formal training of systems-thinking methods and small reminders within audit documentation systems. With today's considerable advances in technology and the growing implementation of those technologies in the audit profession, practitioners should carefully consider how systems-thinking might be employed in the development of tools and the evaluation of their output, as well as how systems-thinking might be instilled in professionals through the technology itself.

Finally, although the research described here focuses on audit processes, there is no reason to believe that systems-thinking techniques will not offer similar improvements across other accounting contexts, like financial reporting, management accounting, and even advisory services, making the results and future research potentially valuable to controllers, consultants, and other financial executives as well.