ChatGPT Can Give Tax Advice, But You Really Get What You Pay For

ChatGPT is among the latest AI technology, but is it qualified to give tax advice? Though it’s usually on the right track, it misses nuances of the tax code 100% of the time, says Lee Reams II of TaxBuzz.

“ChatGPT can write essays, poetry, letters and more, so we posed the question – can it do our taxes? Our company, TaxBuzz, decided to take the technology for a test drive, asking: What status I should use when filing my taxes?”

It offered typical definitions of the IRS’ various filing options. Then came the best advice of the day: “You can consult a tax professional to help determine the best option for you.” Says Reams II, “while ChatGPT might be able to give you a CliffsNotes-style answer to your query, expecting a how-to tip or complete strategy isn’t in the cards.”

He adds that this brings up quite a few questions. “When it comes to the US tax code, there are numerous gray areas and concepts that are set up for interpretation. There might be several ways to solve a problem, but will the AI be able to parse that out? Right now, it seems the answer is no.”

According to ChatGPT, “If you find an answer is incorrect, please provide that feedback by using the Thumbs Down button.” In addition, ChatGPT “will occasionally make up facts or ‘hallucinate’ outputs.”

When it comes to analyzing the tax code, expertly trained humans have access to trusted and verified libraries of information that they can use to verify tax strategy claims. According to Reams II, “ChatGPT simply doesn’t have the same level of brainpower or education as a human tax professional.”

He continues, “we must realize that the tax code is complicated – so complicated that the AICPA recently sent a 163-page letter to Congress with suggestions for simplifying it. More importantly, everyone has their own preferences regarding tax preparation and tax planning.”

ChatGPT’s current version was only trained on data sets available through 2021. So, if you are asking for tax advice, there’s a good chance something has changed in the tax code. ChatGPT’s service agreement is pretty clear: “The services are provided as is. We do not warrant that the services will be uninterrupted, accurate or error free, or that any content will be secure or not lost or altered.”

TaxBuzz put ChatGPT to the test for tax advice on real-world scenarios. Reams II notes that “we used actual questions posted on our tax practitioner technical support forum. ChatGPT provided incorrect answers 100% of the time – though usually on the right track, it missed nuances of the tax code.”

Reams II concludes that “unlike previous technology, ChatGPT and AI tools aren’t going away. We plan to integrate the search tool within our TaxBuzz vertical search engine and tax and accounting professionals websites, where the AI pulls its information from our thousands of trusted experts, guides, and published advice versus the vastness of the web. The goal here is a better user experience – whether it’s faster answers for tax professionals doing research or consumers looking for quick advice.

For more, see ChatGPT Can Give Tax Advice, But You Really Get What You Pay For (bloombergtax.com).