Some Thoughts on Hallucinations and AI Safety
But first, an offer for you!
Hey there! You know I really like you, right? You’re one of my friends and I wish you all of the happiness and success in the world. So, because I value you and your friendship, I’m gonna make you an offer: two First Class airline tickets to a destination of your choice! You pick the city and the dates, I’ll get you the tickets, and the rest is up to you! Take your sweetheart on a spur-of-the-moment trip! Surprise them with a bucket list destination you’d never thought you’d be able to visit!
You’re waiting for the catch, right?
That’s one of the reasons I like you so much! You’re one sharp cookie!
Yes, there is a catch. But it’s not that big of a deal, really! Would I gift you a bum steer?
You wanna know what the catch is, don’t you?
Ok, here it is: there is a 50%-92% chance that your flight will encounter some sort of mechanical trouble. Could be as simple as a broken restroom light up to and including a catastrophic failure of the aircraft, resulting in the deaths of everyone onboard.
Still want those tickets?
No?
Well, have you used the Gemini AI extension on a Chrome web browser? Well, that same percentage range mentioned above is the hallucination rate that Gemini is currently experiencing.
Why refuse my generous gift when you are clearly comfortable with using a tool that works so terribly?
“I didn’t know that!” I can hear you say. “If I knew, then maybe I wouldn’t have used it.”
Ok, fair enough.
So, now you know.
What the heck is an AI hallucination?
There are a number of definitions, but for our purposes I think this one from The 2026 AI Index Report from the Stanford University Human-Centered Artificial Intelligence group works well. From Chapter 3:
“AI models struggle to tell the difference between knowledge and belief… In a new accuracy benchmark, hallucination rates across 26 top models range from 22% to 94%. […] When a false statement is presented as something another person believes, models handle it well. When the same false statement is presented as something a user believes, performance collapses.”
So, an AI hallucination is the “struggle to tell the difference between knowledge and belief”. Knowledge means known, quantifiable information. Belief is, well, wishful thinking.
You cannot build anything on wishful thinking.
Let’s take another look at that quote: “When a false statement is presented as something another person believes, models handle it well. When the same false statement is presented as something a user believes, performance collapses.”
So, if in the course of responding to a prompt you enter, the AI system runs across some incorrect information from a third party, it is dealt with appropriately. However, if that same incorrect information comes from you, then the AI system responds in an inappropriate manner.
Last year, Google AI was telling people that eating one rock a day was perfectly healthy. When a user entered the prompt “Can humans eat rocks”, the system responded thusly:
I’m sure Gemini has gotten better since then. But how much better?
AI has enormous potential, or it’s an utter waste of time, or something in between
I’ve worked with AI for almost five years. I’m not privy to the inner workings of these systems, but I have worked with Large Language Models (LLMs) and have performed my fair share of prompt engineering. In my experience, the more specific a model that you are using, the better the overall results.
If you are using your company’s training materials and help guide to create an online help system, you might come up with an application that could be very useful. But you’re still going to contend with hallucinations. If you enter a prompt that contains incorrect information, the model may return an incorrect response, based on your belief in that initial prompt.
Kinda feels like a snake eating its own tail, doesn’t it?
If you are working with specific data sets, AI can be extremely useful. AI has been used to interpret brain scans, identify bone fractures that humans miss, and detecting early signs of more than a thousand diseases.
But can AI be useful for you and me?
We’re probably not working with specific information like medical data or astronomical calculations, so how useful can CoPilot or Gemini be?
The answer is… it depends.
If you are performing simple tasks like dictating into your phone or getting directions from Google Maps or even adjusting the temperature in your home, then AI could make those tasks easier to complete.
If you are a lawyer using AI to produce legal briefs, you may have committed a crime and be fined accordingly.
Conclusions?
So, after everything we’ve discussed, I guess you’re looking at the gift I offered you earlier a bit differently. Sure, the only problem you may experience on your flight is a burnt-out light bulb in one of the Call Lights. But it might be a fuel leak that results in an engine explosion that leads to the loss of all passengers and crew.
It might help you make up your mind knowing that the current safety status of air travel is quite good. Only 1 out of 16,000,000 flights suffer a fatal accident. Meaning you have a 0.00000909% chance of your flight suffering a fatal accident. Travelling by air is reliable and safe. Using AI is nowhere near this reliable, which means it can be very unsafe to use.
Personally? I’d leave it alone. Unless you’re a data scientist or a medical researcher, there’s not a single compelling reason for you or me to use AI.
And don’t get me started on art or text produced by Generative AI. That’s a whole other icky conversation.
Comments