Connect with us

Accounting

Which generative AI model did best on the CPA exam? Depends on the section

Published

on

ChatGPT is no longer the only large language model to pass the CPA exam.

After ChatGPT 3.5 initially bombed the CPA exam and then version 4.0 passed, it does remain the top performer overall. However, like any human accountant, it has its strengths and weaknesses.

These were part of the findings of a recent paper from Case Western Reserve University and accounting automation solutions provider AIgency. The researchers systematically evaluated the performance of Google Gemini, ChatGPT-4, Claude, Mixtral and Llama-2b on multiple-choice questions from CPA test preparation tools.

Overall, they found that ChatGPT-4 scored the best, with Claude 3-opus coming in a close second, followed by Google Gemini Advanced, then Mixtral-8x7b-32768. Llama 2B-70b-4096 did the worst.

Source: William Zacher Jr. & Sanmukh Kuppannagari

However, as the results show, not every model did uniformly well on all sections. ChatGPT, while a strong performer overall, was especially good on the BAR section for business analysis and reporting. Meanwhile, although its weakest point is REG, the regulatory area that is mostly devoted to tax regulations, it did better on this section of the exam than any other model. Claude was the best performer in the AUD section on auditing and attestation. While its weakest point was FAR, the section on financial accounting and reporting, even there its performance was second only to ChatGPT. Gemini was the second strongest performer on the BAR section, but did not do so well on REG. Mixtral, overall, had decent enough scores compared to a human but would only pass BAR, making it a mediocre player compared to its peers. Llama was the only one that would not pass any section, and it did especially poorly on REG. It was also the only one that did worse than a human. The average score for human test takers on REG was 59.19%, according to the paper.

“The study revealed that while some LLMs have made significant advances in mimicking the complex decision-making skills required for CPA exams, there remains variability in performance across different sections of the test,” said the paper. “This variability underlines the importance of tailored training and specialization in developing LLMs for professional applications such as the CPA exams.”

To perform the test, the researchers drew their multiple choice questions from the Becker CPA test preparation suite. Google Gemini, Claude and ChatGPT-4 were accessed via their online platforms. Mixtral and Llama-2b models were accessed through the Groq platform, an advanced computational infrastructure for high-speed AI processing. The questions were directly copied and pasted into the AI platforms from Becker’s test preparation material without any additional prompting or modification to ensure each AI model received the questions in their original form as they would appear in a CPA exam context.

Becker’s platform randomized the questions in batches of 15 questions, which the research said further mitigated potential selection bias. The tester, responsible for inputting the questions into the AI models, deliberately refrained from reading or evaluating the questions beforehand to prevent any unconscious bias in the prompting process. For each question, the tester selected the AI model’s first response marked as “correct,” irrespective of any variations in the explanations or outputs provided by different models.

Each AI model was subjected to each multiple choice section of the CPA test three times, allowing for a comprehensive assessment of its performance across multiple attempts. The criterion for determining an AI model’s success in this study was achieving a passing score, defined as an average score of 75 or higher, on any given section.

The researchers said the data indicates there is no one universal model for all tasks, so it is important to use the right model for the right applications. For example, the paper concluded that ChatGPT is “the only real option for zero-shot BAR automation,” as “no other model came close to its performance, and it had a relatively narrow variance,” meaning that ChatGPT-4 could be used to help with automated financial statement preparation or additional forecasting. On the other hand, the researchers said Claude was probably better on auditing-related tasks, which the paper said “is a solid indication that it can be used for fraud detection and internal control validation.”

“It is apparent from the results that there is no clear-cut winner,” the researchers concluded. “Most companies utilizing AI to perform financial administration functions should use a software infrastructure that allows them to use multiple task-dependent AI models.”

However, the researchers did recommend that “model selection for AI in an applied accounting setting should avoid Llama-2B, which performed worse than any other model in every section.”

Continue Reading

Accounting

Global ESG Reporting Standards and Double Materiality Compliance

Published

on

Corporate accounting departments face expanding reporting expectations as international sustainability disclosure standards achieve regulatory enforcement across major global jurisdictions. Chief Accounting Officers (CAOs) and corporate controllers are establishing rigorous internal accounting controls to treat Environmental, Social, and Governance (ESG) metrics with the same data precision, auditability, and governance as traditional financial statements.

Regulatory Harmonization Under Global Sustainability Frameworks
The implementation of standardized sustainability reporting frameworks—notably rules established by international sustainability accounting boards—has created unified expectations for public and large private enterprises. Corporations must report standardized metrics covering greenhouse gas emissions (Scope 1, 2, and material Scope 3), energy utilization, workforce demographics, and supply chain governance.

In Europe and other participating international jurisdictions, double materiality principles are mandatory. Under double materiality, organizations must report both how external sustainability risks impact corporate financial performance, and how internal corporate operations affect surrounding environmental and social structures.

Integrating Sustainability Metrics into Core ERP Systems
To provide auditable non-financial data, enterprise organizations are integrating specialized carbon accounting and ESG management platforms directly into core ERP systems. Automated data collectors capture energy utility invoices, logistics fuel consumption metrics, and vendor compliance records in real time.

Establishing automated, traceable data pipelines ensures that non-financial reporting is supported by clear audit trails. This structured approach allows external financial auditors to provide reasonable assurance on sustainability disclosures during annual corporate reporting cycles.

Financial Impacts and Capital Market Disclosure
Accurate ESG reporting directly influences corporate cost of capital and institutional credit ratings. Commercial lenders and institutional asset managers systematically incorporate sustainability metrics into risk pricing models. Companies that demonstrate transparent, verifiable progress in operational energy efficiency and climate risk mitigation benefit from expanded access to green bond markets and lower debt pricing.

Action Steps for Accounting Leadership
1. Implement Double Materiality Frameworks: Conduct comprehensive assessments to identify material financial and operational sustainability metrics.
2. Build Auditable Non-Financial Data Pipelines: Automate ESG data collection within core accounting software to ensure data integrity.
3. Align Sustainability with Annual Financial Filings: Prepare non-financial disclosures concurrently with financial statements to satisfy regulatory audit expectations.

Continue Reading

Accounting

Modernizing Internal Controls: Machine Learning and Continuous Monitoring in Auditing

Published

on

Internal audit departments and corporate risk managers are modernizing internal control frameworks by shifting from periodic sampling techniques to continuous monitoring and machine learning analytics. As operational data volumes increase across enterprise organizations, automated control testing ensures financial integrity, prevents corporate fraud, and streamlines annual audit engagements.

The Limitation of Periodic Audit Sampling
Historically, internal and external auditors evaluated internal controls by reviewing random samples of financial transactions—often analyzing less than five percent of total ledger entries. In complex enterprise environments, periodic sampling methods carry inherent risks of overlooking localized financial misstatements, unauthorized disbursements, or operational control breakdowns.

In 2026, progressive internal audit functions are utilizing automated continuous monitoring platforms that evaluate one hundred percent of financial transactions in real time. Continuous control auditing systems continuously monitor general ledger entries, procurement approvals, and expense reimbursements across all operating subsidiaries.

AI-Powered Fraud Detection and Anomaly Identification
Machine learning models trained on historical corporate financial data excel at identifying subtle transactional anomalies that indicate potential fraud or operational error. Automated systems instantly flag duplicate invoice payments, unapproved vendor creation, unusual journal entry timing, and unauthorized override of authority thresholds.

When an anomaly is detected, the automated auditing platform generates an instant risk alert, allowing internal audit teams to investigate root causes immediately. Early detection prevents minor operational errors from escalating into material weaknesses in financial reporting.

Streamlining External Audit Preparation
Continuous internal control monitoring delivers significant benefits during annual external financial audits. External audit firms can review continuous audit logs and automated control testing documentation, reducing the time required for manual field testing.

This integrated approach lowers overall audit compliance fees, reduces administrative burdens on corporate accounting staff, and provides senior management and audit committees with real-time visibility into the organization’s overall risk profile.

Core Implementation Guidelines
1. Transition to 100% Data Testing: Replace legacy sampling methods with automated continuous audit monitoring systems.
2. Deploy Anomaly Detection Algorithms: Implement machine learning models to identify unauthorized transactions and operational control overrides.
3. Align Internal and External Audit Workflows: Coordinate continuous control testing protocols with external auditors to optimize annual compliance cycles.

Continue Reading

Accounting

Automated Tax Compliance and Global Regulatory Harmonization in 2026

Published

on

Corporate tax accounting departments are navigating an era of unprecedented regulatory complexity as global tax harmonization frameworks take full effect alongside real-time digital tax reporting mandates. Tax directors and accounting teams are adopting cloud-based tax compliance automation tools to manage multi-jurisdictional tax liabilities and satisfy stringent reporting rules across international jurisdictions.

Implementation of Global Minimum Tax Provisions
The implementation of international tax reform agreements—notably the Pillar Two global minimum tax framework—has reshaped multinational corporate tax planning. Multinational enterprises with consolidated revenues exceeding established thresholds must ensure an effective tax rate of at least 15% across every jurisdiction in which they operate.

Accounting teams are implementing specialized tax calculation modules integrated directly into enterprise resource planning (ERP) platforms. These automated tools calculate effective tax rates per country, identify top-up tax liabilities, and generate standardized compliance documentation required by national tax authorities.

Real-Time Digital Invoicing and E-Reporting Mandates
Tax authorities across Europe, Latin America, and Asia-Pacific have enacted mandatory electronic invoicing (e-invoicing) and continuous transaction controls (CTC). Under these systems, corporate transaction data must be submitted electronically to government portals in real time at the point of sale or invoice issuance.

This shift toward continuous digital tax reporting eliminates traditional annual tax audits in favor of ongoing automated compliance monitoring. Accounting departments are upgrading invoicing software to ensure seamless XML data formatting, digital signature authentication, and real-time validation against tax authority databases.

Automation and Data Analytics in Corporate Tax Strategy
To keep pace with dynamic tax legislation, tax departments are transitioning from reactive compliance teams to proactive strategic advisors. Machine learning algorithms analyze corporate transactional data to identify tax credits, research and development (R&D) incentives, and cross-border transfer pricing adjustments.

By automating routine tax return filings and calculations, corporate tax directors can focus on long-term capital structuring, evaluating the tax implications of corporate mergers, and optimizing international supply chain networks.

Strategic Priorities for Tax Executives
1. ERP System Upgrades: Ensure enterprise software is capable of generating real-time, granular tax data required for global minimum tax compliance.
2. E-Invoicing Integration: Implement scalable e-invoicing platforms to satisfy regional continuous transaction control regulations.
3. Strategic Tax Analytics: Utilize predictive tax modeling tools to evaluate structural changes in corporate operations and cross-border trade.

Continue Reading

Trending