Each few weeks, it looks like there’s a brand new headline a few lawyer getting in hassle for submitting filings containing, within the phrases of 1 choose, “bogus AI-generated analysis.” The main points fluctuate, however the throughline is similar: an legal professional turns to a big language mannequin (LLM) like ChatGPT to assist them with authorized analysis (or worse, writing), the LLM hallucinates circumstances that don’t exist, and the lawyer is none the wiser till the choose or opposing counsel factors out their mistake. In some circumstances, together with an aviation lawsuit from 2023, attorneys have needed to pay fines for submitting filings with AI-generated hallucinations. So why haven’t they stopped?
The reply principally comes all the way down to time crunches, and the way in which AI has crept into practically each occupation. Authorized analysis databases like LexisNexis and Westlaw have AI integrations now. For legal professionals juggling large caseloads, AI can appear to be an extremely environment friendly assistant. Most legal professionals aren’t essentially utilizing ChatGPT to jot down their filings, however they’re more and more utilizing it and different LLMs for analysis. But many of those legal professionals, like a lot of the general public, don’t perceive precisely what LLMs are or how they work. One legal professional who was sanctioned in 2023 mentioned he thought ChatGPT was a “tremendous search engine.” It took submitting a submitting with pretend citations to disclose that it’s extra like a random-phrase generator — one that would provide you with both appropriate data or convincingly phrased nonsense.
Andrew Perlman, the dean of Suffolk College Legislation College, argues many legal professionals are utilizing AI instruments with out incident, and those who get caught with pretend citations are outliers. “I believe that what we’re seeing now — though these issues of hallucination are actual, and legal professionals should take it very severely and watch out about it — doesn’t imply that these instruments don’t have huge doable advantages and use circumstances for the supply of authorized companies,” Perlman mentioned. Authorized databases and analysis programs like Westlaw are incorporating AI companies.
In truth, 63 p.c of legal professionals surveyed by Thomson Reuters in 2024 mentioned they’ve used AI prior to now, and 12 p.c mentioned they use it recurrently. Respondents mentioned they use AI to jot down summaries of case legislation and to analysis “case legislation, statutes, varieties or pattern language for orders.” The attorneys surveyed by Thomson Reuters see it as a time-saving software, and half of these surveyed mentioned “exploring the potential for implementing AI” at work is their highest precedence. “The position of a very good lawyer is as a ‘trusted advisor’ not as a producer of paperwork,” one respondent mentioned.
However as loads of latest examples have proven, the paperwork produced by AI aren’t all the time correct, and in some circumstances aren’t actual in any respect.
In a single latest high-profile case, legal professionals for journalist Tim Burke, who was arrested for publishing unaired Fox Information footage in 2024, submitted a movement to dismiss the case in opposition to him on First Modification grounds. After discovering that the submitting included “vital misrepresentations and misquotations of supposedly pertinent case legislation and historical past,” Choose Kathryn Kimball Mizelle, of Florida’s center district, ordered the movement to be stricken from the case report. Mizelle discovered 9 hallucinations within the doc, based on the Tampa Bay Instances.
Mizelle finally let Burke’s legal professionals, Mark Rasch and Michael Maddux, submit a brand new movement. In a separate submitting explaining the errors, Rasch wrote that he “assumes sole and unique duty for these errors.” Rasch mentioned he used the “deep analysis” characteristic on ChatGPT professional, which The Verge has beforehand examined with combined outcomes, in addition to Westlaw’s AI characteristic.
Rasch isn’t alone. Legal professionals representing Anthropic lately admitted to utilizing the corporate’s Claude AI to assist write an professional witness declaration submitted as a part of the copyright infringement lawsuit introduced in opposition to Anthropic by music publishers. That submitting included a quotation with an “inaccurate title and inaccurate authors.” Final December, misinformation professional Jeff Hancock admitted he used ChatGPT to assist arrange citations in a declaration he submitted in help of a Minnesota legislation regulating deepfake use. Hancock’s submitting included “two quotation errors, popularly known as ‘hallucinations,’” and incorrectly listed authors for an additional quotation.
These paperwork do, in actual fact, matter — at the least within the eyes of judges. In a latest case, a California choose presiding over a case in opposition to State Farm was initially swayed by arguments in a quick, solely to seek out that the case legislation cited was utterly made up. “I learn their transient, was persuaded (or at the least intrigued) by the authorities that they cited, and regarded up the selections to be taught extra about them – solely to seek out that they didn’t exist,” Choose Michael Wilner wrote.
Perlman mentioned there are a number of much less dangerous methods legal professionals use generative AI of their work, together with discovering data in massive tranches of discovery paperwork, reviewing briefs or filings, and brainstorming doable arguments or doable opposing views. “I believe in nearly each activity, there are methods wherein generative AI might be helpful — not an alternative choice to legal professionals’ judgment, not an alternative choice to the experience that legal professionals convey to the desk, however to be able to complement what legal professionals do and allow them to do their work higher, sooner, and cheaper,” Perlman mentioned.
However like anybody utilizing AI instruments, legal professionals who depend on them to assist with authorized analysis and writing have to be cautious to test the work they produce, Perlman mentioned. A part of the issue is that attorneys usually discover themselves quick on time — a problem he says existed earlier than LLMs got here into the image. “Even earlier than the emergence of generative AI, legal professionals would file paperwork with citations that didn’t actually tackle the difficulty that they claimed to be addressing,” Perlman mentioned. “It was only a completely different sort of drawback. Generally when legal professionals are rushed, they insert citations, they don’t correctly test them; they don’t actually see if the case has been overturned or overruled.” (That mentioned, the circumstances do at the least sometimes exist.)
One other, extra insidious drawback is the truth that attorneys — like others who use LLMs to assist with analysis and writing — are too trusting of what AI produces. “I believe many individuals are lulled into a way of consolation with the output, as a result of it seems at first look to be so nicely crafted,” Perlman mentioned.
Alexander Kolodin, an election lawyer and Republican state consultant in Arizona, mentioned he treats ChatGPT as a junior-level affiliate. He’s additionally used ChatGPT to assist write laws. In 2024, he included AI textual content in a part of a invoice on deepfakes, having the LLM present the “baseline definition” of what deepfakes are after which “I, the human, added within the protections for human rights, issues like that it excludes comedy, satire, criticism, creative expression, that sort of stuff,” Kolodin informed The Guardian on the time. Kolodin mentioned he “might have” mentioned his use of ChatGPT with the invoice’s foremost Democratic cosponsor however in any other case needed it to be “an Easter egg” within the invoice. The invoice handed into legislation.
Kolodin — who was sanctioned by the Arizona State Bar in 2020 for his involvement in lawsuits difficult the results of the 2020 election — has additionally used ChatGPT to jot down first drafts of amendments, and informed The Verge he makes use of it for authorized analysis as nicely. To keep away from the hallucination drawback, he mentioned, he simply checks the citations to ensure they’re actual.
“You don’t simply sometimes ship out a junior affiliate’s work product with out checking the citations,” mentioned Kolodin. “It’s not simply machines that hallucinate; a junior affiliate may learn the case mistaken, it doesn’t actually stand for the proposition cited anyway, no matter. You continue to should cite-check it, however you need to try this with an affiliate anyway, except they have been fairly skilled.”
Kolodin mentioned he makes use of each ChatGPT’s professional “deep analysis” software and the LexisNexis AI software. Like Westlaw, LexisNexis is a authorized analysis software primarily utilized by attorneys. Kolodin mentioned that in his expertise, it has the next hallucination charge than ChatGPT, which he says has “gone down considerably over the previous 12 months.”
AI use amongst legal professionals has turn out to be so prevalent that in 2024, the American Bar Affiliation issued its first steerage on attorneys’ use of LLMs and different AI instruments.
Legal professionals who use AI instruments “have an obligation of competence, together with sustaining related technological competence, which requires an understanding of the evolving nature” of generative AI, the opinion reads. The steerage advises legal professionals to “purchase a basic understanding of the advantages and dangers of the GAI instruments” they use — or, in different phrases, to not assume that an LLM is a “tremendous search engine.” Attorneys also needs to weigh the confidentiality dangers of inputting data referring to their circumstances into LLMs and contemplate whether or not to inform their shoppers about their use of LLMs and different AI instruments, it states.
Perlman is bullish on legal professionals’ use of AI. “I do suppose that generative AI goes to be probably the most impactful know-how the authorized occupation has ever seen and that legal professionals might be anticipated to make use of these instruments sooner or later,” he mentioned. “I believe that in some unspecified time in the future, we are going to cease worrying concerning the competence of legal professionals who use these instruments and begin worrying concerning the competence of legal professionals who don’t.”
Others, together with one of many judges who sanctioned legal professionals for submitting a submitting filled with AI-generated hallucinations, are extra skeptical. “Even with latest advances,” Wilner wrote, “no moderately competent legal professional ought to out-source analysis and writing to this know-how — notably with none try to confirm the accuracy of that materials.”
