Accountants and auditors frequently make judgments that require complex evaluations of facts and rules. Researchers examine these judgments by conducting experiments in which participants make accounting decisions and then describe the information they used and how they used it in making their decisions. These elicited descriptions provide researchers insight into participants’ knowledge structures. This study uses a design science approach to develop a semantic similarity (a natural language processing analytic method) measure of how closely participants’ knowledge structures match an exemplar knowledge structure derived from authoritative accounting sources. The study then tests the semantic similarity metric in an experimental setting with two accounting tasks: assessing inventory obsolescence and production process business risk. Results show that the semantic similarity metric evaluates the knowledge structure match comparably to other methods requiring human judges. Importantly, the semantic similarity metric yields comparable results more efficiently.

This content is only available via PDF.