Preparing Auditors for the Blockchain Oracle Problem Free

Blockchain is an emerging technology of particular interest to the auditing profession. Most notably, blockchain offers an innovative way to maintain a shared ledger such that transactions and contracts are only executed and recorded once validated and agreed-upon by the community of users. As such, transactions and contracts that exist solely within the blockchain ecosystem execute based on validated data, which allows the shared ledger to be highly trusted (Sheldon 2018; Williams 2019). With blockchain, transactions are typically used to move value between addresses (Lantz and Cawrey 2021), but can also be used to submit information to contracts. These contracts, known as smart contracts, are short software programs that allow users to set up agreements on the blockchain typically in the form of IF a happens, THEN b happens. Smart contracts can therefore be used to codify business agreements on the blockchain, and automatically process an exchange of consideration(s) once a pre-defined condition does/does not occur. For example, IF goods are delivered by a certain date, THEN payment is released to the vendor. The advantage of having such agreements codified on a blockchain is that smart contracts are difficult to modify once deployed, contract terms are transparent to the community of users (on a public blockchain), and the blockchain offers resiliency such that the contract will remain intact even if a portion of the blockchain nodes are attacked or compromised (Warburg, Wagner, and Serres 2019).

Given that smart contracts exist on the blockchain, they are limited to information available on the blockchain for how to execute. To make smart contracts more useful in settling real-world agreements, information about off-blockchain events must be obtained, and this information is provided by oracles (Diedrich 2016; AICPA and CPA Canada 2017; BlockchainHub 2021). In this role, oracles are parties that must be trusted to submit reliable information to an otherwise verified blockchain ledger, a point of vulnerability that has become known as the oracle problem (Delphi 2017; Curran 2018; Orcutt 2018; Fecke 2018). As blockchain implementations become more mainstream, auditors will have to evaluate the risks that emerge in these scenarios when exogenous data is introduced (by oracles) to the ecosystem (e.g., an oracle providing data to a smart contract about an event in the physical world, such as a specific firm's financial results or the delivery of goods).

A recently published study, “Auditing the Blockchain Oracle Problem” (Sheldon 2021), examines the blockchain oracle problem and its implications for the auditing profession. Specifically, Sheldon (2021) proposes a working definition of oracles, which highlights their role in providing exogenous data to blockchains that is then used to initiate smart contracts and process associated transactions. Given this function, the author argues that oracles can be viewed as service organizations under the auditing standards of both the American Institute of Certified Public Accountants (AICPA) and the Public Company Accounting Oversight Board (PCAOB). In support of this argument, the author references AICPA and PCAOB auditing standards that describe service organizations as parties that provide services that are part of a user entity's information system, including services that impact how the user's transactions are initiated or processed. Sheldon (2021) then goes on to propose control objectives to mitigate the risks posed by the oracle problem. The current article summarizes the previously published study's background on oracles, its argument that oracles function as service organizations, and its illustrative control objectives that auditors can consider when evaluating the reliability of information provided by an oracle.¹ Furthermore, the current article extends the previous study by also discussing complementary user entity controls that user entities might be held responsible for when using the services of an oracle.

Sheldon (2021) considers recent discussions and writings in the blockchain community (e.g., Bertani 2016; ConsenSys 2016; Curran 2018; MyCryptopedia 2018; Slobodnik 2018; Voshmgir 2019) to develop a working definition of oracles. Oracles are then defined as:

Given this definition, oracles should be unbiased agents that operate outside the boundaries of a blockchain's network. To remain unbiased, oracles must perform requested tasks in an objective manner without regard for how the provided data will impact any individual smart contract party. For example, in transforming data, the original data point(s) might exist in an unstructured format (e.g., as part of an image or video), or might need to be run through a calculation performed by the oracle (e.g., to determine an average or consensus across several observations). In all instances, the oracle must remain unbiased to the smart contract parties as it transforms the observation(s) into text or code that can be submitted to the blockchain as a transaction. Sheldon's (2021) definition also highlights that oracles gather information about events that do not already exist on the blockchain, and that oracles are responsible for some level of temporary storage of data. Finally, the oracle must release the observed data at a specified time or interval directly to the target blockchain.²

As described above, oracles collect information rather than create it on their own. Therefore, oracles rely on different data sources from which to extract the target information. The two most common sources are hardware and software data sources (Curran 2018; MyCryptopedia 2018; Voshmgir 2019). Information available online is said to come from a software data source, and could include (for example) weather data from The Weather Channel (www.weather.com) or National Weather Service (www.weather.gov) as well as financial data from Bloomberg or Thomson Reuters (Bertani 2016; ConsenSys 2016; Fecke 2018; Werbach 2018; Sheldon 2019; BlockchainHub 2021). See Figure 1 for an example of an oracle using a software data source. Information about the physical world is provided by hardware data sources. Here, the oracle collects data from smart devices such as heat sensors, geolocators, radio frequency identification (RFID) sensors, or nearly any other Internet-of-Things (IoT) device that generates information about the physical world (Brown-Liburd and Vasarhelyi 2015; ConsenSys 2016; Dai and Vasarhelyi 2017; Fecke 2018; Sheldon 2019; BlockchainHub 2021). See Figure 2 for an example of an oracle using a hardware data source.

In the simplest form, oracles can act in a deterministic manner, meaning a single oracle is using a single software/hardware data source to inform the smart contract. By using one oracle and/or one data source, there is a heightened risk that the captured data could be manipulated or inappropriately managed. As such, there are also methods of using oracles and/or data sources in a consensus manner. For example, the smart contract might require a specific level of consensus on an observation from several oracles that are independent from one another (or some pre-defined M-of-N proportion of oracles) (Bertani 2016; Curran 2018; MyCryptopedia 2018; Crowder 2019). If the concern is that the oracle would manipulate the data once observed to help one of the smart contract parties (e.g., as part of the transformation of unstructured data), requiring multiple oracles to obtain data from the same source might mitigate this risk. Similarly, one oracle might obtain data from several sources and report the consensus/average observation to the smart contract (e.g., applying calculations to transform data). These approaches to mitigate the risk of incorrect information can be expanded to include any number of oracles and any number of data sources. Auditors should be aware of the underlying tradeoffs that smart contract users make in these scenarios, such that users might accept slower speeds and higher costs of obtaining exogenous data by using more oracles and/or data sources to increase their confidence that the obtained information is reliable (Crowder 2019).

Regardless of the consensus reached among oracles, it is unlikely that an auditor will be able to use this approach to establish the reliability of data provided by an oracle. In some instances, auditors might be able to independently obtain evidence to validate the information provided by an oracle (e.g., weather data or financial reports). While this might be a cost-effective way to validate data provided by oracles, smart contract users take on significant risk by not requiring the oracle to have robust controls in place to ensure data reliability. At the core of this risk, smart contracts are largely irreversible once executed, meaning any assets incorrectly distributed will not be recovered. Auditors should be prepared to discuss this risk, and encourage clients to only use oracles with established/evaluated processes in place to provide reliable information to smart contracts. In doing so, auditors will need to be prepared to evaluate the oracle and the processes it has in place to collect, store, transform, and transmit data. The next section offers one way that auditors can approach this analysis, by treating oracles as service organizations.

The AICPA's AU-C Section 500, Audit Evidence, and the PCAOB's AS 1105, Audit Evidence, both require auditors to obtain evidence that is sufficient (i.e., amount) and appropriate (i.e., relevant and reliable) (PCAOB 2010; AICPA 2012). As such, under both the private and public company audit standards in the U.S., auditors must be prepared to obtain sufficient and appropriate audit evidence that oracles provide reliable information to smart contracts. Further, the oracle appears to be providing a service to smart contract users as it collects, stores, transforms, and transmits the target data. According to the AICPA's AU-C Section 402, Audit Considerations Relating to an Entity Using a Service Organization, a service organization is “An organization or segment of an organization that provides services to user entities that are relevant to those user entities' internal control over financial reporting” (AICPA 2018, 381). The PCAOB's AS No. 2601, Consideration of an Entity's Use of a Service Organization, similarly defines a service organization as “The entity (or segment of an entity) that provides services to a user organization that are part of the user organization's information system” (PCAOB 2002, Par .02). As such, if a smart contract acts as an internal control to ensure that transactions are processed appropriately, then the information provided by the oracle is relevant to these users' internal control over financial reporting, thus making the oracle act as a service organization.

If an oracle is to be viewed as a service organization, it is also important to determine whether it functions as part of the users' information system. For private company audits, AICPA AU-C Section 402, Audit Considerations Relating to an Entity Using a Service Organization, specifies that a service organization is part of a user's information system if it affects “the procedures within both IT and manual systems by which the user entity's transactions are initiated, authorized, recorded, processed, corrected as necessary, transferred to the general ledger, and reported in the financial statements” (Par .03.e). Guidance from the PCAOB is similar. Under AS 2601, Consideration of an Entity's Use of a Service Organization (PCAOB 2002), a service organization is part of a user's information system if it affects the “procedures, both automated and manual, by which the entity's transactions are initiated, recorded, processed, and reported from their occurrence to their inclusion in the financial statements” (PCAOB 2002, Par .03). In considering these points, Sheldon (2021, 14) argues that as “oracles provide information to smart contracts that initiate the execution of transactions, and these transactions are recorded in the information systems of the smart contract parties, an oracle appears to operate as part of the user entity's information system.” Given this and the preceding point about the definition of a service organization, it appears the auditing standards from both the AICPA and PCAOB support the notion that an oracle acts as a service organization.

If the oracle operates as a service organization, the auditor will want to ensure the smart contract users either (1) secure a right to audit the oracle's internal control system, or (2) obtain a System and Organization Control (SOC) type of attestation report from the oracle on the implementation and effectiveness of its internal controls. In either case, it remains important to establish controls that mitigate risks to data reliability as the oracle collects, stores, transforms, and transmits data to a smart contract. Some of these risks can be mitigated by smart contract users implementing complementary user entity controls (i.e., internal controls implemented by users to ensure the service is appropriately delivered). Remaining risks to data reliability reside with the service organization or subservice organization (i.e., a service organization to another service organization), and control objectives must be established to mitigate those risks. These topics are addressed in the next section.

To evaluate the risk posed by an oracle, it is necessary to begin by understanding the unique smart contract it informs. Specifically, who are the parties involved, what assets are held by the smart contract, what is the trigger used to execute the smart contract, and how are the assets distributed based on whether that trigger occurs? It is also important to consider whether the trigger event can be objectively observed and reported. Next, the auditor will want to understand the oracle being used to provide this exogenous data to the blockchain, along with the sources it is being asked to use. For software data sources, an illustrative control objective is whether controls provide reasonable assurance that:

• CO #1 … the oracle obtains complete and accurate data from the requested software data source.³

When collecting information from hardware data sources, there are inherently more risks to address in gaining comfort that the data collected are reliable. For example, auditors will need to evaluate how the hardware device is commissioned, what evidence is available that the device remains affixed to the intended asset (e.g., an RFID tag remaining attached to a vehicle), how the device observes and captures the target event, how the device temporarily stores data before transmission, how the device transmits data to the oracle so that the data cannot be intercepted or manipulated along the way, and the process of maintaining and/or replacing the device. Illustrative control objectives to consider for hardware data sources include whether controls provide reasonable assurance that:

• CO #2 … hardware data sources are affixed to, and remain affixed to, the intended asset (or remain in a specified geographic location).

• CO #3 … hardware data sources are configured to completely and accurately capture the intended occurrence.

• CO #4 … hardware data sources are physically maintained and/or replaced to prevent unreliable data capture and transmission.

• CO #5 … the oracle only accepts data transmissions from active hardware data sources (i.e., signals from decommissioned sources are not processed).

• CO #6 … data stored in hardware data sources cannot be modified inappropriately.

• CO #7 … the oracle obtains complete and accurate data from the specified hardware data source(s).³

With the data now collected by the oracle, it is necessary to evaluate how the oracle temporarily stores this data in a way that it cannot be inappropriately modified. Data residing with the oracle will also likely need to be transformed into a format readable by the target blockchain/smart contract, and the auditor should ensure that the integrity of the original observation is maintained during this transformation and that the transformed data remains a complete and accurate representation of the data received from the hardware source. When the data is ready to be transmitted to the smart contract, auditors will also want to evaluate how the oracle knows when to transmit this data (i.e., at a specific date/time or interval), and then evaluate the security of the transmission. Illustrative control objectives to consider for data stored, transformed, and transmitted by the oracle include whether controls provide reasonable assurance that:

• CO #8 … unauthorized changes to data stored by oracles is prevented or detected and corrected.

• CO #9 … the oracle performs a complete and accurate transformation of the original data to being data that is readable by smart contracts.

• CO #10 … information is transmitted to smart contracts in accordance with any agreed-upon timing specifications.

• CO #11 … the oracle establishes a secure link to the smart contract for data transmission, such that the transfer of data is complete, accurate, and direct between the oracle and smart contract.

Table 1 provides a summary of the risks and control objectives just discussed, and Figure 3 provides an illustration of the oracle process and where these control objectives appear as the oracle collects, stores, transforms, and transmits data to smart contracts.

It is important to note that some of the processes described above might be performed by a subservice organization, in which case the auditor would want comfort over the achievement of the related control objective(s) at the subservice organization. For example, a subservice organization might be hired to provide/administer the devices used as hardware data sources, in which case Control Objectives 2 through 7 would be addressed at the subservice organization. Furthermore, as part of the interface between a subservice organization and a service organization, or between a service organization and a user entity, the party receiving services will need to implement complementary user entity controls to ensure the service is delivered appropriately. These controls will typically be specified by the party providing the service. For example, when a subservice organization provides services to a service organization, the service organization might be required to provide complete and accurate data so that the subservice organization can perform its service correctly, and might also be required to have controls in place to ensure the service output is received correctly (i.e., complete and accurate receipt of a data transmission to a storage facility with restricted access). Similar complementary user entity controls should also be expected to be in place when the user entity interacts with a service organization. In the event that the subservice organization is carved-out of the service organization's report, the auditor will want to obtain the subservice organization's attestation report to identify its complementary user entity controls and to determine whether it achieved its related control objective(s).

Given the topics discussed in this section on how to address risks posed by oracles, it is also prudent to consider the impact of control objectives that are not achieved along with possible limitations in these types of audits.

As previously discussed, smart contracts are difficult to modify once deployed on either a public or private blockchain, and are largely irreversible once executed.⁴ It is therefore critical that smart contracts be informed by reliable oracles. This paper proposes a series of control objectives that, if achieved, should help users gain comfort that an oracle is providing reliable data to a smart contract. However, if any of these control objectives are not achieved, there is a heightened risk that unreliable data might be provided to a smart contract, which could cause the smart contract to execute in an unintended manner. Given the nature of blockchain, correcting this mistake and/or recouping any funds distributed upon the execution of a misinformed smart contract might be impossible. Smart contract users should be aware of this risk, and only use oracles when necessary and only when they have comfort that the oracle has robust and effective internal control processes in place. If this comfort cannot be achieved, users should reconsider whether it is too early in the evolution of smart contracts and oracles to use these tools for material transactions.

The control objectives proposed in this study make it clear that the supporting internal controls should work to provide reasonable assurance that the objectives are achieved. Given the novel technologies and processes introduced by blockchain, auditors will likely need to evaluate a wide range of control activities to determine whether the control objectives have been achieved, and might be advised to initially include some level of compensating and/or overlapping controls. As part of testing control activities, auditors will need to obtain sufficient and appropriate evidence as to the effective operation of the controls (PCAOB 2010; AICPA 2012), and thus will need to determine whether such evidence is likely to exist (or be accessible) before accepting an engagement. For example, smart devices attached to physical assets (i.e., hardware data sources) might not be operated by the oracle, and thus the auditor would need to identify the party that maintains those devices (i.e., a subservice organization). Here, if the auditor determines that the subservice organization should be included in the scope of the engagement, the auditor will need access to the subservice organization to evaluate the design and operation of controls used to maintain the smart devices. If any part of this process cannot be evaluated, it becomes difficult to defend that the oracle provides reliable exogenous data to the blockchain. Furthermore, even if this data is deemed reliable, the auditor is still not providing assurance that the smart contract itself is designed or coded correctly.

Sheldon (2021) is the first study to outline an approach for auditors to use when confronted with an oracle that is trusted to provide reliable data to an otherwise secure blockchain (i.e., the blockchain oracle problem). Given the rapid pace of blockchain development and the importance of smart contract functionality, it should be expected that auditors will soon face the challenge of evaluating oracles that provide smart contracts with much needed off-chain data. As such, auditors will need to understand how oracles operate, how to classify oracles from an audit standpoint, and how to evaluate the risks presented by oracles when used to inform the execution of business agreements codified as smart contracts. The importance of oracle data reliability is further amplified by the fact that this data is used to inform irreversible smart contracts, and the outcomes of these smart contracts might initiate action by other smart contracts, thus triggering a cascading effect of the unreliable data. It is therefore critical that information provided by oracles is reliable so that smart contracts execute as intended, which also contributes to the sustained reliability of the blockchain ledger.

This article summarized some of the most important take-aways from Sheldon (2021) on how auditors should evaluate oracles. First, auditors must understand the function that oracles play in the blockchain ecosystem. This is conveyed in the working definition of oracles, which is proposed as “third-party agents that exist outside a blockchain, collect data on external occurrences (i.e., exogenous data), temporarily store then transform this data into a format that can be read by a smart contract, then transmit the resulting information to a designated smart contract to inform its execution” (Sheldon 2021, 122). The article also highlights how the auditing standards from both the AICPA and PCAOB can be interpreted to view an oracle as part of a smart contract users' information system, in which case the oracle's function would align with that of a service organization. Finally, the article describes the process used by an oracle to collect, store, transform, and transmit data and highlights relevant risks and illustrative control objectives throughout this process.

While this article introduces the blockchain oracle problem and proposes ways for auditors to address it, there is much left for the profession to consider with the impact of oracles. Now is the time to preemptively face these challenges, so auditors are not forced to simply react once the challenges become unavoidable.