Is it All Hype - ChatGPT’s Performance and Disruptive Potential in the Accounting and Auditing Industries

Sep 4, 2026

ChatGPT frequently appears in the media, with many claims that it will significantly disrupt accounting and auditing. Prior work, however, reported weak results on student assessment questions. This study evaluates whether newer ChatGPT models and capabilities can pass major accounting certification exams, including the CPA, CMA, CIA, and EA. Early ChatGPT 3.5 fails all exams, but iterative enhancements allow ChatGPT to achieve an average score of 85.1%. The findings suggest sufficient capability for potential industry disruption and offer adaptation guidance for practitioners, investors, and stakeholders.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages53

Reading Time2 h 14 min

AccessFree to download

Introduction

Prior hype and technology revolutions

Research purpose and exam coverage

Performance of ChatGPT on accounting exams

Baseline results with ChatGPT 3.5

Model improvements and enhanced capabilities

Is it All Hype? ChatGPT’s Performance and Disruptive Potential in the Accounting and Auditing Industries Marc Eulerich University of Duisburg-Essen [marc.eulerich@uni-due.de](marc.eulerich@uni-due.de) [ORCID: 0000-0002-9965-7584](ORCID: 0000-0002-9965-7584) Aida Sanatizadeh University of Illinois Chicago [asanat2@uic.edu](asanat2@uic.edu) ORCID: 0000-0003-4225-4415 Hamid Vakilzadeh University of Wisconsin – Whitewater [vakilzas@uww.edu](vakilzas@uww.edu) ORCID: 0000-0001-7398-3472 David A. Wood Brigham Young University [davidwood@byu.edu](davidwood@byu.edu) ORCID: 0000-0002-9748-8561 Abstract ChatGPT frequently appears in the media, with many predicting significant disruptions, especially in the fields of accounting and auditing. Yet, prior research has demonstrated relatively poor performance of ChatGPT on student assessment questions. We extend this research to examine whether more recent ChatGPT models and capabilities can pass major accounting certification exams including the CPA, CMA, CIA, and EA (enrolled agent) certification exams. We find that the early released ChatGPT 3.5 model is unable to pass any exam (average score across all assessments of 53.1 percent) . However, with additional enhancements, ChatGPT can pass all sections of each tested exam: moving to the ChatGPT 4 model improved scores by an average of 16.5 percent, providing 10-shot training improved scores an additional 6.6 percent, and allowing the model to use reasoning and acting (e.g., allow ChatGPT to use a calculator and other resources) improved scores an additional 8.9 percent. After all these improvements, ChatGPT passed all exams with an average score of 85.1 percent. This high performance indicates that ChatGPT has sufficient capabilities to potentially disrupt the accounting and auditing industries, which we discuss in detail. Lastly, this research provides practical insights for accounting professionals, investors, and stakeholders on how to adapt and mitigate the potential negative consequences of this cutting-edge technology in accounting and auditing firms. Key Words: Artificial Intelligence, ChatGPT, Large Language Model, GPT 3.5, GTP 4, Accounting Certification 1 Electronic copy available at: [https://ssrn.com/abstract=4636807](https://ssrn.com/abstract=4636807) 1. Introduction A study by McKinsey estimates that by 2030, artificial intelligence (AI) could displace 15 percent of the global workforce, or 400 million workers, and hit the accounting profession particularly hard (Manyika and Sneader, 2018) . Indeed, the release of ChatGPT, a large language model developed by OpenAI and one of the fastest-growing technologies in history (e.g., Reuters, 2023), has revived a discussion of how AI and automation will change the accounting profession. When it was released in November of 2022, news stories immediately discussed how technology would massively disrupt the business world. For example, Eloundou et al. (2023) report “that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of [large language models], while approximately 19% of workers may see at least 50% of their tasks impacted.” They list, among others, accountants, auditors, and tax preparers as having a 100 percent exposure to significant automation. Concurrently, large accounting firms have announced the use of large language model-based AI systems: PwC and KPMG will spend $1 and $2 billion, respectively, while EY has already spent $1.5 billion (PwC, 2023; WSJ 2023a, 2023b) . This is not the first time predictions have been made of how technology will drastically revolutionize the accounting profession. These other “technology revolutions” include, among other things the introduction of the computer; software improvements like spreadsheets, databases, and ERP systems; continuous auditing; distributed ledger technology (blockchain); automation tools like robotic process automation (

FAQs

What exams does the study test ChatGPT on?

The study evaluates performance on the CPA, CMA, CIA, and EA (enrolled agent) certification exams.

How did ChatGPT 3.5 perform compared with later models?

ChatGPT 3.5 could not pass any exam, with an average score of 53.1%. After enhancements and moving to ChatGPT 4, the average improved to 85.1%.

Which enhancements contributed to higher exam scores?

Scores improved after switching to ChatGPT 4, adding 10-shot training, and allowing reasoning and acting, such as using a calculator and other resources.

Sep 4, 2026

Uploaded by Oliver Hayes