Connect with us

Accounting

Which generative AI model did best on the CPA exam? Depends on the section

Published

on

ChatGPT is no longer the only large language model to pass the CPA exam.

After ChatGPT 3.5 initially bombed the CPA exam and then version 4.0 passed, it does remain the top performer overall. However, like any human accountant, it has its strengths and weaknesses.

These were part of the findings of a recent paper from Case Western Reserve University and accounting automation solutions provider AIgency. The researchers systematically evaluated the performance of Google Gemini, ChatGPT-4, Claude, Mixtral and Llama-2b on multiple-choice questions from CPA test preparation tools.

Overall, they found that ChatGPT-4 scored the best, with Claude 3-opus coming in a close second, followed by Google Gemini Advanced, then Mixtral-8x7b-32768. Llama 2B-70b-4096 did the worst.

Source: William Zacher Jr. & Sanmukh Kuppannagari

However, as the results show, not every model did uniformly well on all sections. ChatGPT, while a strong performer overall, was especially good on the BAR section for business analysis and reporting. Meanwhile, although its weakest point is REG, the regulatory area that is mostly devoted to tax regulations, it did better on this section of the exam than any other model. Claude was the best performer in the AUD section on auditing and attestation. While its weakest point was FAR, the section on financial accounting and reporting, even there its performance was second only to ChatGPT. Gemini was the second strongest performer on the BAR section, but did not do so well on REG. Mixtral, overall, had decent enough scores compared to a human but would only pass BAR, making it a mediocre player compared to its peers. Llama was the only one that would not pass any section, and it did especially poorly on REG. It was also the only one that did worse than a human. The average score for human test takers on REG was 59.19%, according to the paper.

“The study revealed that while some LLMs have made significant advances in mimicking the complex decision-making skills required for CPA exams, there remains variability in performance across different sections of the test,” said the paper. “This variability underlines the importance of tailored training and specialization in developing LLMs for professional applications such as the CPA exams.”

To perform the test, the researchers drew their multiple choice questions from the Becker CPA test preparation suite. Google Gemini, Claude and ChatGPT-4 were accessed via their online platforms. Mixtral and Llama-2b models were accessed through the Groq platform, an advanced computational infrastructure for high-speed AI processing. The questions were directly copied and pasted into the AI platforms from Becker’s test preparation material without any additional prompting or modification to ensure each AI model received the questions in their original form as they would appear in a CPA exam context.

Becker’s platform randomized the questions in batches of 15 questions, which the research said further mitigated potential selection bias. The tester, responsible for inputting the questions into the AI models, deliberately refrained from reading or evaluating the questions beforehand to prevent any unconscious bias in the prompting process. For each question, the tester selected the AI model’s first response marked as “correct,” irrespective of any variations in the explanations or outputs provided by different models.

Each AI model was subjected to each multiple choice section of the CPA test three times, allowing for a comprehensive assessment of its performance across multiple attempts. The criterion for determining an AI model’s success in this study was achieving a passing score, defined as an average score of 75 or higher, on any given section.

The researchers said the data indicates there is no one universal model for all tasks, so it is important to use the right model for the right applications. For example, the paper concluded that ChatGPT is “the only real option for zero-shot BAR automation,” as “no other model came close to its performance, and it had a relatively narrow variance,” meaning that ChatGPT-4 could be used to help with automated financial statement preparation or additional forecasting. On the other hand, the researchers said Claude was probably better on auditing-related tasks, which the paper said “is a solid indication that it can be used for fraud detection and internal control validation.”

“It is apparent from the results that there is no clear-cut winner,” the researchers concluded. “Most companies utilizing AI to perform financial administration functions should use a software infrastructure that allows them to use multiple task-dependent AI models.”

However, the researchers did recommend that “model selection for AI in an applied accounting setting should avoid Llama-2B, which performed worse than any other model in every section.”

Continue Reading

Accounting

How to Create an Effective Invoice Process for Small Businesses

Published

on

How to Create an Effective Invoice Process for Small Businesses

A well-designed invoice is crucial to ensuring timely payments, maintaining consistent cash flow, and building strong client relationships. Invoicing is more than just paperwork—it plays a key role in the financial health and professional image of a business. When invoices are clear and professional, they encourage prompt payments and minimize disputes. Poorly constructed invoices, however, can result in delays, misunderstandings, and even missed payments.

The Basics of Professional Invoicing

Crafting a professional invoice begins with the basics. Essential elements should include the business name, logo, and contact information. Each invoice should be assigned a unique invoice number—using a format like “2024-01-001” (year-month-number) helps in keeping them easily organized. Additionally, clearly stating the issue date and due date is vital for clarity.

Creating Clear Service Descriptions

A detailed service or product description is the core of an effective invoice. Specificity is key—list the quantities, rates, and applicable taxes for each item. Assuming that clients recall the details of a service can lead to confusion; clarity prevents disputes. Invoices should include subtotals for each category and a bold final amount due, ensuring that the payment amount is easily identifiable. Additionally, it’s crucial to outline accepted payment methods and provide clear instructions for how payments should be made.

Avoiding Common Invoicing Mistakes

Sending invoices to the wrong contact is a common error that can lead to unnecessary payment delays. Maintaining an up-to-date database of client billing contacts and payment preferences can prevent these issues. Confirming who is responsible for accounts payable before sending invoices is a prudent practice.

————————————————————————————————-

Stanley Quencher H2.0 FlowState Stainless Steel Vacuum Insulated Tumbler with Lid and Straw for Water, Iced Tea or Coffee, Smoothie and More, Peony, 30oz

>>> Order Your Stanley Quencher Now<<<

Paid ads at no cost to buyer

————————————————————————————————–

Importance of Timing and Payment Options

The timing of invoice issuance can impact payment speed and client relations. Invoices should be sent promptly upon project completion to ensure timely payments. Establishing and adhering to a regular invoicing schedule fosters consistency and reduces delays.

Offering multiple payment options can further expedite payments. Clients often expect flexible and convenient payment methods. While digital payments like ACH transfers and credit cards may incur small fees, the benefits of faster payments usually outweigh the costs. Many businesses have seen significant reductions in average payment times by offering online payment solutions.

Leveraging Technology for Invoicing

Technology can greatly enhance the invoicing process. Reliable invoicing software can automate routine tasks such as issuing recurring invoices, sending payment reminders, and tracking outstanding payments. However, it is important to remember that technology is not infallible. Regular human oversight is necessary to identify potential errors that automated systems might overlook.

Essential Checklist for Invoice Accuracy

Consistency in the invoicing process is critical. Creating a checklist for invoice preparation can help maintain accuracy. Key items to verify include:

  • Confirming correct client details.
  • Checking all calculations for accuracy.
  • Ensuring the stated payment terms align with agreements.
  • Reviewing client preferences for invoice delivery.
  • Double-checking the applicable tax rates.

This checklist serves as a final review before sending any invoice to ensure it meets professional standards.

Implementing Effective Follow-up Procedures

Prompt follow-up on overdue payments is a necessary component of an effective invoicing system. Sending a gentle reminder around 15 days after the due date, followed by a firmer notice at 30 days, can often encourage payment without damaging client relationships. Maintaining a record of all communications related to payments is essential for clarity and documentation.

Conclusion

An efficient invoicing process not only facilitates timely payments but also reinforces professionalism, showing respect for both the business’s work and the client’s time. A clear, consistent, and well-maintained invoicing system directly impacts financial stability and client satisfaction. By focusing on accuracy, timing, and communication, businesses can significantly improve their cash flow and strengthen professional relationships with clients.

A successful invoicing strategy lies in keeping the process simple, ensuring consistency, and always maintaining a professional standard. This disciplined approach to invoicing contributes to better financial outcomes and more enduring client partnerships.

Continue Reading

Accounting

PCAOB calls off NOCLAR standard for this year

Published

on

Facing a backlash from audit firms over its proposal to toughen the standards for failing to detect noncompliance with laws and regulations, the Public Company Accounting Oversight Board has decided to delay action on the standard this year.

The PCAOB proposed the so-called NOCLAR standard in June, with the goal of strengthening its requirements for auditors to identify, evaluate and communicate possible or actual noncompliance with laws and regulations, including fraud. However, the proposed standard provoked resistance from a number of auditing firms and state CPA societies like the Pennsylvania Institute of CPAs and spurred a comment letter-writing campaign organized by the Center for Audit Quality and the U.S. Chamber of Commerce that was supported by prominent business trade groups like the American Bankers Association, the Business Roundtable, the Retail Industry Leaders Association and more. 

Earlier this week, the PCAOB issued staff guidance outlining the existing responsibilities of auditors to detect, evaluate and communicate about illegal acts. The PCAOB was slated to finalize the NOCLAR standard by the end of this year, but after the election it has put the standard on hold for now, anticipating the upcoming change in the administration in Washington, D.C.

“Following the recent issuance of staff guidance, the PCAOB will not take additional action on NOCLAR this year,” said a PCAOB spokesperson. “We will continue engaging with stakeholders, including the SEC, as we determine potential next steps. As our process has demonstrated, the PCAOB is committed to listening to all stakeholders and getting it right.”

PCAOB logo - office - NEW 2022

One reason for the change of plans is that the PCAOB anticipates changes in the regulatory environment under the Trump administration, especially in the Securities and Exchange Commission, which would have to approve the final standard before it could be adopted. The Trump administration is likely to replace SEC chairman Gary Gensler, who has spearheaded many of the increased regulatory efforts at the Commission and encouraged the PCAOB to update its older standards and take a tougher stance on enforcement and inspections. President-elect Trump, in contrast, has promised to eliminate regulations, and Gensler’s push for increased regulation has attracted the ire of many in the financial industry.

According to a person familiar with the PCAOB process, no further action is expected until further consultation with the SEC under the incoming administration can take place. 

Questions have arisen over whether the PCAOB might decide to repropose the standard with modifications given the amount of opposition it has attracted. That is to be determined pending review of the comment letters that have been received, as well as a roundtable from earlier this year, along with responses from targeted inquiries from firms in their approach relating to NOCLAR. 

PCAOB board members Christina Ho and George Botic were asked about the NOCLAR proposal on Wednesday at Financial Executives International’s Current Financial Reporting Insights Conference, and Ho acknowledged the pushback. 

“We’ve heard strong opposition from the auditing profession, public companies, audit committees, investors, academics and others,” said Ho. “The PCAOB has received 189 individualized comments to date on that proposal. This proposal now has the third highest number of comment letters in the history of PCAOB. That did get a lot of attention. Commenters overwhelmingly called for a reproposal or withdrawal of the proposed standard so that that is definitely something that I am looking at a lot, and I also voted against the proposal. I have spoken to various stakeholders, including investors, audit committee chairs and members, and some preparers as well. The question I got asked repeatedly was, what problem is PCAOB trying to solve? And the people I spoke to believe that there have been improvements in financial reporting quality over the past 20 years, and that obviously is consistent with the CAQ study noting a consistent decline in restatements. While there’s always room for improvement, they noted that a balance is necessary between increased investor protection and increased auditor implementation costs that are ultimately passed on to issuers, and that the NOCLAR proposal lacks such a balance. That is what I have heard from the comment letters, so that pretty much summarizes what I have seen, and I’m still obviously thinking about it.”

Botic noted that the proposal came before he joined the board, but he referred to the staff guidance that had been issued earlier in the week by the PCAOB on the existing requirements.

Last week, the PCAOB updated its standard-setting and rulemaking agendas before the outcome of the election was known. Now with the uncertainty over the regulatory environment, the PCAOB is mindful of the difficulty of having the SEC decide on whether to approve it, especially if the five-member commission becomes evenly split among two Republican members and the two Democrats if Gensler departs or is ousted. The PCAOB feels the SEC needs adequate time to review and educate itself on the proposed standard, rather than having to jam it through a two-two commission, especially with the amount of engagement that will need to take place given such an important standard, according to a person familiar with the matter.

The PCAOB expects it to remain on the docket for 2025 but doesn’t want to try to jam it through this year. However, the PCAOB announced Friday that it has scheduled an open board meeting next Thursday, Nov. 21, on another proposed standard on firm and engagement metrics, which has also provoked pushback from many commenters, but is still slated to be finalized this year.

Continue Reading

Accounting

Accountants eye sustainable business management

Published

on

Accountants are increasingly being asked to deal with sustainability issues as more businesses are called upon by investors to report on how they are dealing with issues like climate change and carbon emissions.

This week, amid the United Nations COP29 climate change conference in Azerbaijan, business leaders have been playing a larger role, including fossil fuel companies, prompting an open letter on Friday from environmental groups calling for reforms in the COP process. 

ESG standard-setters have also been playing a role at COP, with groups like the Global Reporting Initiative and the Carbon Disclosure Project signing a memorandum of understanding to deepen their collaboration on making their standards interoperable as the International Sustainability Standards Board reported progress on growing acceptance of its standards by 30 jurisdictions around the world.

Last month, the Institute of Management Accountants released a report on why business sustainability depends on the competencies of management accountants. The report discusses the critical areas in which management accountants are crucial to ensuring sustainability within their organizations, along with how existing accounting capabilities support sustainable business.

Institute of Management Accountants headquarters in Montvale, N.J.

“The main focus and the main attention right now in the ESG field is going to compliance, to the reporting parts,” said Brigitte de Graaff, who chaired the IMA committee that authored the report. “There are a lot of rules and regulations out there.” 

For right now, those rules and regulations are mostly voluntary in the U.S., especially with the Securities and Exchange Commission’s climate disclosure rule on hold. But in the European Union, where de Graaff is based in Amsterdam, companies have to comply with the Corporate Sustainability Reporting Directive. 

“In Europe, of course, there is not a lot of voluntary reporting for the larger companies anymore, but it’s all mandatory with a huge amount of data points and aspects that they need to report, so there’s a lot of focus right now on how to comply with these rules and regulations,” said de Graaff. “However, there’s also a lot of discussion going on about whether it should be about compliance. What’s the reason for reporting all these aspects? For us what was really important was that there is a lot of opportunity for management accountants to work with this kind of information.”

She sees value beyond purely disclosing ESG information. “If you use this information, and you integrate this in your organization, there’s much more value that you can get out of it, and it’s also much more part of what kind of value you are creating as an organization, and it’s much more aligned with what you were doing,” said de Graaff. 

The report discusses the benefits of the information, and how management accountants can play an important role. “You can use and integrate this in your FP&A and your planning processes,” said de Graaff. “You can integrate this kind of information in your strategy, something that management accountants are very well equipped for, but also to track performance and see how you’re actually achieving your goals, not only on financial aspects, but also on these nonfinancial aspects that are much broader than the E, S and G factors.”

The report discusses how to go beyond the generic environmental, social and governance parts of ESG to understand how they relate to a business’s core operations and make it more sustainable.

Management accountants can even get involved in areas such as biodiversity. “Even though, as a management accountant, you might not be an expert on marine biology and what the impact of your organization is underwater, you are able to tell what are the checks that have been performed on this,” said de Graaf. “Is this a common standard? Is this information that is consistently being monitored throughout the organization? Or is it different and what are the benchmarks? What are the other standards? These kinds of processes are something that management accountants are well aware of, and how they can check the quality of this information without being a subject matter expert on every broad aspect that may entail in this ESG journey that an organization is on.”

ESG can become part of the other work that management accountants are already involved in performing for their organizations.

“Ultimately there are a lot of competencies that management accountants were already doing in their organization, and ESG might sometimes seem unrelated, but it basically ties in into the competencies that we already know,” said de Graaff. “I hope that with this report, we can also show that the competencies that we are so familiar with, that we’ve been dealing with other strands of financial information, that you can basically also use these competencies in the ESG arena. Even though there’s a lot that seems very new, if you are aware of how you can tie that in, you can use the skills that you already have, the skill set that you have as a management accountant, to really improve your risk management processes, your business acumen, your operational decision making, etc. I hope that with this publication, we can also take away a little bit of the big fear that might be around a huge topic, as ESG is now. This is actually just a very interesting and exciting way to look at this kind of information, and we are very well equipped to help organizations navigating through this changing ESG regulation world.”

Continue Reading

Trending