How do we prevent “hallucinations”?
No AI system can eliminate hallucinations. Including ours.
For the AI models in use today, no AI system can completely remove the risk that a model invents a case, a quote, or a fact (what’s commonly called a “hallucination”).
Bells Up AI features many recently released flagship models, also called “frontier” or “foundation” models, from leading AI developers. Those developers work hard to reduce how often their newest models hallucinate. Bells Up AI also offers older and faster models. No model, new or old, is immune.
Some companies claim they’ve solved the hallucination problem by using one AI to “verify” another. This verification step doesn’t eliminate the risk. It just moves it from one model to the next. And even a system that works perfectly can’t completely stand in for your own review, because verifying what you submit to courts or clients is part of a lawyer’s work, just as it’s always been.
The bottom line
Verifying AI output is your responsibility as a lawyer, and it can’t be handed off to a machine.
ABA Formal Opinion 512 (opens in new tab), the AI guidance issued by a growing list of state bars (California, New York, Pennsylvania, Virginia, DC, and more), and a growing number of court rules (Florida Rule 2.515(d)(2) among them) all treat that verification as a duty you can’t delegate. ABA Model Rule 1.1, Comment 8 makes technological competence part of your duty of competence. Model Rule 5.3 requires you to supervise nonlawyer assistance, and commentary to this rule is emphatic that this duty is intended to reach the technology you rely on as well as staff and other lawyers.
What Bells Up AI can do is give you tools to check
Bells Up won’t verify your work for you. What it will do is give you tools to check output before a document leaves your desk:
- Compare Models. Send the same prompt to several models at once and see where they agree and where they don’t. When one model returns a citation, a legal point, or a fact that none of the others do, treat that as a signal to look closer.
- Panel of Experts. Choose a set of models to cross-check one another. As with Compare Models, an answer that surfaces in only one model might be a red flag, meaning the outliers show you exactly where to look first, while you keep in mind that models agreeing with each other still isn’t verification.