Tax loopholes abound, but AI could shut them down

To eliminate tax loopholes that cost the federal government billions of dollars every year, tech and law experts are working together to create artificial intelligence that can find loopholes better than a legion of blue-chip tax accountants.

Though the task is daunting—a recent test with the tax code “completely baffled” ChatGPT—leaders of the Johns Hopkins University-based effort not only believe it’s doable, they worry a corporate-funded effort will beat them to the punch and use their own AI to find even more tax loopholes.

“The rules of the game are if you’re operating within the law, how you bend the law to save on taxes is OK,” said Benjamin Van Durme, a Johns Hopkins computer scientist specializing in AI who’s leading the effort. “It’s very difficult to predict all possible ways a law will work in the real world. The point of using AI is it could guess ahead before these laws get locked down.”

What’s understood as “tax law” is a massive, unwieldly array of laws enacted by Congress, Treasury regulations, Internal Revenue Service rulings, and court decisions. Even if all of it had been enacted with the best intentions, workarounds exist for taxpayers clever enough to find them.

“The problem is when a clever taxpayer takes one ruling and combines it with another to produce an idea that saves a lot of money,” said Andrew Blair-Stanek, a tax attorney turned law professor who is part of the AI team. “It’s the pairing of all these tax laws by clever lawyers.”

The IRS estimates an annual tax gap of about $500 billion, and some part of that results from these clever pairings. Hoping to narrow the gap with better technology, Blair-Stanek enrolled at Johns Hopkins to get a doctoral degree in computer science in Van Durme’s lab. They recruited another student, Nils Holzenberger, to help create an AI system they’re calling Shelter Check.

The idea is to create software that Congress, the IRS, or the courts, could use to easily scan proposed tax legislation or rulings for loopholes they might unintentionally create.

“That’s why we call it Shelter Check—it’s like a spell checker, but for tax shelters,” Van Durme said. “We want to build a system that could read proposed changes in the law and inform Congress and the IRS about ramifications for the tax code, or warn people writing new policies about unintended side effects.”

Such sophisticated analysis will require a machine that can not only read, parse and understand complex tax law, but blend all of it like an accountant or a tax attorney.

The project is in the early stages, with the team building on the lab’s extensive work in machine learning to explore language processing options and test them. As yet, they haven’t found a way to conquer the legalese let alone replicate the deductive reasoning of a real human.

Despite recent advances in language-based artificial intelligence, the latest in tech is no match for tax code. First, while the material is text-based, the legal language it’s written in is complex if not downright arcane. Then, the thousands of pages of documents are peppered with tables that are vital to interpreting tax outcomes.

The team is experimenting now with language prediction technology like ChatGPT and GPT-3. Tax code stumped both.

“GPT-3 was completely baffled by the tax code,” said Blair-Stanek, a law professor at the University of Maryland. “Literally flipping a coin on these would get 50% of the questions we were asking right and GPT-3 only got about 70%. And these were just basic questions about the tax code like so-and-so is a dependent, makes $100,000 a year, does this tax section apply? It couldn’t handle it.”

Still, the team feels that a solution is possible. For example, initial experiments with the recently announced GPT-4 suggest that progress is being made but that there is still work to be done.

“We have yet to find a way to easily produce the judgements of a human tax professional with high accuracy, but we have been making progress,” Van Durme said. “But, if we’re making progress, certainly a much larger, better funded corporate effort could be making it faster.”

The team worries that corporations, which have the most to gain from tax shelters, are also working on this type of artificial intelligence—and might already have it. Team members believe their AI could also be adapted for broader law uses in medicine and business.

“I’m planning to spend the rest of my career trying to make it work,” Blair-Stanek said.