Connect with us

Accounting

Which generative AI model did best on the CPA exam? Depends on the section

Published

on

ChatGPT is no longer the only large language model to pass the CPA exam.

After ChatGPT 3.5 initially bombed the CPA exam and then version 4.0 passed, it does remain the top performer overall. However, like any human accountant, it has its strengths and weaknesses.

These were part of the findings of a recent paper from Case Western Reserve University and accounting automation solutions provider AIgency. The researchers systematically evaluated the performance of Google Gemini, ChatGPT-4, Claude, Mixtral and Llama-2b on multiple-choice questions from CPA test preparation tools.

Overall, they found that ChatGPT-4 scored the best, with Claude 3-opus coming in a close second, followed by Google Gemini Advanced, then Mixtral-8x7b-32768. Llama 2B-70b-4096 did the worst.

Source: William Zacher Jr. & Sanmukh Kuppannagari

However, as the results show, not every model did uniformly well on all sections. ChatGPT, while a strong performer overall, was especially good on the BAR section for business analysis and reporting. Meanwhile, although its weakest point is REG, the regulatory area that is mostly devoted to tax regulations, it did better on this section of the exam than any other model. Claude was the best performer in the AUD section on auditing and attestation. While its weakest point was FAR, the section on financial accounting and reporting, even there its performance was second only to ChatGPT. Gemini was the second strongest performer on the BAR section, but did not do so well on REG. Mixtral, overall, had decent enough scores compared to a human but would only pass BAR, making it a mediocre player compared to its peers. Llama was the only one that would not pass any section, and it did especially poorly on REG. It was also the only one that did worse than a human. The average score for human test takers on REG was 59.19%, according to the paper.

“The study revealed that while some LLMs have made significant advances in mimicking the complex decision-making skills required for CPA exams, there remains variability in performance across different sections of the test,” said the paper. “This variability underlines the importance of tailored training and specialization in developing LLMs for professional applications such as the CPA exams.”

To perform the test, the researchers drew their multiple choice questions from the Becker CPA test preparation suite. Google Gemini, Claude and ChatGPT-4 were accessed via their online platforms. Mixtral and Llama-2b models were accessed through the Groq platform, an advanced computational infrastructure for high-speed AI processing. The questions were directly copied and pasted into the AI platforms from Becker’s test preparation material without any additional prompting or modification to ensure each AI model received the questions in their original form as they would appear in a CPA exam context.

Becker’s platform randomized the questions in batches of 15 questions, which the research said further mitigated potential selection bias. The tester, responsible for inputting the questions into the AI models, deliberately refrained from reading or evaluating the questions beforehand to prevent any unconscious bias in the prompting process. For each question, the tester selected the AI model’s first response marked as “correct,” irrespective of any variations in the explanations or outputs provided by different models.

Each AI model was subjected to each multiple choice section of the CPA test three times, allowing for a comprehensive assessment of its performance across multiple attempts. The criterion for determining an AI model’s success in this study was achieving a passing score, defined as an average score of 75 or higher, on any given section.

The researchers said the data indicates there is no one universal model for all tasks, so it is important to use the right model for the right applications. For example, the paper concluded that ChatGPT is “the only real option for zero-shot BAR automation,” as “no other model came close to its performance, and it had a relatively narrow variance,” meaning that ChatGPT-4 could be used to help with automated financial statement preparation or additional forecasting. On the other hand, the researchers said Claude was probably better on auditing-related tasks, which the paper said “is a solid indication that it can be used for fraud detection and internal control validation.”

“It is apparent from the results that there is no clear-cut winner,” the researchers concluded. “Most companies utilizing AI to perform financial administration functions should use a software infrastructure that allows them to use multiple task-dependent AI models.”

However, the researchers did recommend that “model selection for AI in an applied accounting setting should avoid Llama-2B, which performed worse than any other model in every section.”

Continue Reading

Accounting

SEC’s Semiannual Reporting Proposal Faces Investor Pushback: What CFOs Need to Know

Published

on

U.S. Securities and Exchange Commission (SEC)

A proposal from the U.S. Securities and Exchange Commission to potentially shift some public companies away from quarterly financial reporting toward a semiannual model is drawing significant pushback from investors, even as it continues moving through the regulatory process. The debate has direct implications for corporate finance teams, auditors, and the broader transparency of U.S. capital markets.

What the SEC Proposed

According to a summary published by accounting advisory firm Cohen & Co., the SEC issued a proposed rule on May 19, 2026, aimed at simplifying financial reporting requirements for many U.S. public companies. The proposal would potentially reduce the frequency of certain mandatory disclosures from quarterly to semiannual, a structural change that has not been made to core U.S. reporting requirements in decades.

The proposal follows an extended debate within U.S. policy circles, with proponents arguing that reduced reporting frequency could lower compliance costs and free up management time for longer-term strategic planning rather than quarter-to-quarter results management.

Why Investors Are Pushing Back

Comment letters submitted in response to the proposal have been extensive, and according to Cohen & Co.’s review of the public record, investors “appear to be largely opposed” to the shift, viewing frequent interim reporting as a core benefit of U.S. capital markets relative to other jurisdictions.

Accounting and law firms have taken a more measured position, generally urging any changes to remain aligned with the Financial Accounting Standards Board (FASB), whose existing disclosure requirements and guidance are built around a quarterly reporting cadence. A shift to semiannual reporting without corresponding changes to FASB guidance could create friction between SEC filing requirements and GAAP-based disclosure expectations.

Lessons From the U.K. Experience

The debate is not without precedent. The United Kingdom moved away from mandatory quarterly reporting for listed companies in 2014, returning to a semiannual disclosure requirement. According to Cohen & Co.’s analysis, that experience offers a cautionary data point: there was no measurable increase in capital expenditure or R&D investment following the change, while analyst coverage of affected companies declined as reliable interim information became less available — a particular risk for smaller and newly public companies that rely on analyst coverage to maintain investor visibility.

Practical Implications for Finance Teams

Beyond the debate over disclosure philosophy, the proposal carries practical complications. Many companies have debt covenants and credit agreements structured around quarterly financial delivery; a shift to semiannual reporting could require renegotiating those terms. Reduced reporting frequency would also extend the “window of market silence” between disclosures, a factor that governance and investor-relations teams would need to manage carefully to avoid information asymmetry.

Separately, and unrelated to the reporting-frequency debate, the SEC and FASB have continued finalizing more routine updates this year. New Accounting Standards Updates are taking effect for December 31, 2026, fiscal year-ends covering income tax disclosures, credit loss measurement, induced debt conversions, and stock compensation, according to Eide Bailly’s review of 2026 ASU activity. Additional guidance on paid-in-kind dividends and environmental credits is also on the near-term horizon.

What to Watch Next

The semiannual reporting proposal remains in the comment and review phase, and no final rule has been adopted as of this writing. Finance leaders should monitor the SEC’s regulatory agenda for further movement, while treating the current quarterly reporting requirement as the operative standard until any final rule is issued and an effective date is set.

Given the extent of investor opposition documented in the comment file, a full shift to mandatory semiannual reporting appears more likely to result in either a scaled-back compromise or continued study rather than swift adoption — though the SEC’s ultimate direction remains uncertain.

 

Continue Reading

Accounting

AI-Driven Automation and Continuous Accounting Frameworks

Published

on

The accounting profession is undergoing a fundamental structural transition as enterprise finance departments shift from periodic month-end closes toward automated continuous accounting models. By integrating specialized machine learning algorithms directly into enterprise resource planning (ERP) platforms, chief accounting officers are transforming financial reporting from a retrospective exercise into a real-time operational asset.

The Shift from Periodic Close to Continuous Financial Reporting
Traditional accounting workflows heavily relied on manual data reconciliation, spreadsheet calculations, and multi-week closing cycles at the end of each fiscal period. In contrast, continuous accounting frameworks utilize automated software agents to process, validate, and post transactional data in real time as business activities occur.

Automated bank reconciliation tools cross-reference incoming bank feeds, invoice records, and purchase orders automatically. By resolving transactional variances instantly throughout the month, corporate accounting teams eliminate the traditional workload spikes associated with quarterly and annual closes.

Machine Learning in Audit Trails and Anomaly Detection
Advanced natural language processing (NLP) and machine learning tools are redefining internal audit and financial control environments. Automated systems analyze 100% of general ledger entries, identifying anomalous transactions, duplicate payments, and unauthorized journal entries in real time.

Rather than relying on random statistical sampling, corporate internal auditors can focus their attention on high-risk flags automatically surfaced by algorithmic monitoring platforms. This continuous risk assessment strengthens internal controls over financial reporting (ICFR) and significantly reduces fraud risk.

Evolving Roles for Accounting Professionals
As routine data entry and manual reconciliation tasks become fully automated, the skill set required for accounting professionals is shifting toward data analysis, system design, and strategic business advisory.
– Systems Governance: Accountants are increasingly responsible for monitoring algorithmic accuracy and managing data integration pipelines.
– Business Partnership: Finance professionals leverage real-time financial dashboards to advise operational leaders on margin management and working capital allocation.
– Regulatory Compliance Management: Accounting teams utilize automated platforms to ensure compliance with dynamic tax codes and international accounting standards.

Core Implementation Recommendations
1. Deploy Automated Reconciliation Tools: Integrate continuous transaction processing modules into existing enterprise ERP architectures.
2. Establish Algorithmic Governance Controls: Implement strict internal testing protocols to ensure automated accounting rules comply with GAAP/IFRS standards.
3. Reskill Accounting Teams: Invest in training finance staff on data analytics, workflow automation, and predictive financial modeling.

Continue Reading

Accounting

Global ESG Reporting Standards and Double Materiality Compliance

Published

on

Corporate accounting departments face expanding reporting expectations as international sustainability disclosure standards achieve regulatory enforcement across major global jurisdictions. Chief Accounting Officers (CAOs) and corporate controllers are establishing rigorous internal accounting controls to treat Environmental, Social, and Governance (ESG) metrics with the same data precision, auditability, and governance as traditional financial statements.

Regulatory Harmonization Under Global Sustainability Frameworks
The implementation of standardized sustainability reporting frameworks—notably rules established by international sustainability accounting boards—has created unified expectations for public and large private enterprises. Corporations must report standardized metrics covering greenhouse gas emissions (Scope 1, 2, and material Scope 3), energy utilization, workforce demographics, and supply chain governance.

In Europe and other participating international jurisdictions, double materiality principles are mandatory. Under double materiality, organizations must report both how external sustainability risks impact corporate financial performance, and how internal corporate operations affect surrounding environmental and social structures.

Integrating Sustainability Metrics into Core ERP Systems
To provide auditable non-financial data, enterprise organizations are integrating specialized carbon accounting and ESG management platforms directly into core ERP systems. Automated data collectors capture energy utility invoices, logistics fuel consumption metrics, and vendor compliance records in real time.

Establishing automated, traceable data pipelines ensures that non-financial reporting is supported by clear audit trails. This structured approach allows external financial auditors to provide reasonable assurance on sustainability disclosures during annual corporate reporting cycles.

Financial Impacts and Capital Market Disclosure
Accurate ESG reporting directly influences corporate cost of capital and institutional credit ratings. Commercial lenders and institutional asset managers systematically incorporate sustainability metrics into risk pricing models. Companies that demonstrate transparent, verifiable progress in operational energy efficiency and climate risk mitigation benefit from expanded access to green bond markets and lower debt pricing.

Action Steps for Accounting Leadership
1. Implement Double Materiality Frameworks: Conduct comprehensive assessments to identify material financial and operational sustainability metrics.
2. Build Auditable Non-Financial Data Pipelines: Automate ESG data collection within core accounting software to ensure data integrity.
3. Align Sustainability with Annual Financial Filings: Prepare non-financial disclosures concurrently with financial statements to satisfy regulatory audit expectations.

Continue Reading

Trending