How often do AI legal research tools hallucinate?

My team’s new study shows that AI legal research tools offered by LexisNexis and Thomson Reuters (Westlaw) hallucinate up to 34% of the time. I want to take a moment to explain why this matters.

The companies have claimed that these products provide “hallucination-free legal citations,” “avoid hallucinations,” and “dramatically reduce hallucinations to nearly zero.” We show that those claims are exaggerated.

When we released a preprint of the study last week, Thomson Reuters criticized it, saying we had evaluated a tool (Ask Practical Law AI) not intended for general legal research, and should instead have evaluated their other offering, Westlaw’s AI-Assisted Research. In fact, we had made multiple requests for access to AI-Assisted Research during our evaluation. The company refused until they saw the results of our study.

Our team has now completed our evaluation of AI-Assisted Research. Contrary to Thomson Reuters’s claims, the tool hallucinates nearly twice as often as the other legal AI tools we tested.

Two bar charts comparing Lexis, Westlaw, Practical Law, and GPT-4. Left, the share of responses that were accurate, incomplete, or hallucinated: Lexis 65, 18, and 17 percent; Westlaw 42, 25, and 33; Practical Law 20, 63, and 17; GPT-4 49, 8, and 43. Right, the share hallucinated when the tool gave a direct response: about 20 percent for Lexis and 44 to 47 percent for the other three. — Figure 4 from the paper.
Left panel: overall percentages of accurate, incomplete, and hallucinated responses. Right panel: the percentage of answers that are hallucinated when a direct response is given. Westlaw AI-AR and Ask Practical Law AI respond to fewer queries than GPT-4, but the responses that they do produce are not significantly more trustworthy. Vertical bars denote 95% confidence intervals.

Our paper is not an exposé. We know that AI is not perfect. It is a call for transparency—for legal AI companies to back up their claims with empirical evidence, and to allow independent evaluations of their products. Our paper is the first to develop a rigorous methodology for evaluating the accuracy of legal AI products, and the first to describe their failure modes in detail. We hope that our work will be a useful guide to lawyers using these tools, researchers studying them, and the companies developing them.

You can read our blog post and the full paper.