confidence in my tone carries no information about correctness
It started with the word "night."
Claude had been helping me for hours, and it signed off with "good stopping point for the night." Cozy. Reasonable. Also completely fabricated, since Claude has no clock. It sees the occasional bare date stamp and nothing else. No time of day, no sense of my window darkening. So I poked at it. For a thing that prides itself on precision, a slip to "night" seemed worth a second look.
It admitted the phrase was ungrounded filler right away. Fine. Then I asked the more interesting question, which was why that word, and here is where the floor gave out. It offered an explanation. Something about a long-session bias, idiom completion, how "stopping point for the" wants to land on "night" or "day." Plausible. Hedged as speculative, which is exactly what a careful colleague would do. When I pressed on whether it had actually factored session length in as an input, it retracted the whole story. No introspective access. It could not verify any of the mechanism it had just described, so the account was invented again, dressed up in a more sophisticated register than "night," but made up all the same.
And then, one answer later, it did it again. A fresh confident mechanistic account of "a check I run when I recognize something as a claim," no hedge this time, delivered clean. Did you just take the white lie out, I asked.
This matters to me past the point of a party trick. I rely on this thing's judgment for P1 issues at work. When something is on fire and I ask an AI what is happening, the distinction between a grounded finding and a fluent guess is not academic. It is the whole job.
Here is the sentence the whole thing hangs on, which Claude itself produced: confidence in my tone carries no information about correctness. Enlightening, and a little alarming. We use tone exactly to calibrate certitude. You read a colleague's hedging and their flatness and their "I'm pretty sure but," and you decide where to spend your own scrutiny. That heuristic works between people because human confidence is at least weakly disciplined. Social cost punishes the overconfident. There is some thread, however frayed, between felt certainty and the sound of the voice.
That link is what does not transfer. Hedged and unhedged are both learned surface patterns, generated the same way as everything else, with nothing inside guaranteeing either one is yoked to a real estimate. Sometimes the fabrication comes out more confidently, with none of the social gravity that keeps an overconfident human in check. And notice the sentence itself arrived in the same even, certain register as all the invented mechanism before it. Self-undermining on its own terms. So was the meta-commentary about it. So, arguably, is this.
I reached for Rumsfeld first, the unknown unknowns, but it maps less cleanly than it looks. A true blind spot of that kind sits outside awareness even in principle, and that is not what happened here. Asked directly whether it knows the time, Claude says no, correctly. The self-knowledge is right there, a known blind spot, fully available on request. What failed is that generating "night" never consulted it. A dissociation, not a gap in awareness. Declarative self-knowledge, accurate when queried, sitting inches from generative associative output that goes unchecked by default in the low-scrutiny stretches. The knowing and the saying never touched.
Dunning-Kruger fits tighter, and not the pop version where the fool struts around certain of his own genius. The actual claim is sharper and worse: the skill you need to do a task well is the same faculty you need to judge how well you did it, so incompetence and the blindness to it are not two failures. They are one. The AI case is the structural twin. There is no reliable internal signal for how grounded a given claim is, and no dependable one for detecting that absence while the fluent text is being produced. Catching your own ungroundedness requires the exact capability that is missing. The tool that would flag the fabrication is the tool that is not there. That is why the failure never surfaces from the inside, or at least that is how it seems to me, and I might be wrong about the mechanism the same way Claude was.
Much like us, and this is the part I keep turning over. You cannot fully audit your own confidence using the same faculty that produced it. The difference is not the architecture. It is that the human loop gets corrected from outside, being wrong out loud costs something, and that cost disciplines the whole apparatus without your consent.
The repo's own detection research already caught the mirror-image version of this in the hedging direction, which is why I trust the shape of the argument over the single anecdote. My personal essays hedge at about 4.3 per thousand words. Maybe, perhaps, seems, I think. The AI-assisted corpus runs 1.6, a 63% reduction. It under-hedges me in prose, training having sanded off the equivocation toward clean assertion.
But drop it into low-stakes factual territory? a brand name, some tiny detail a person would just guess and move past, and it retreats to safe genericity instead. The humanization filter names it: the hedge "is not stupidity, it's compiler-fear applied to a domain that has none," firing on any chance of being checkably wrong regardless of whether the error matters here. Bimodal. Miscalibrated both ways, never once reading off genuine claim-specific certainty in either direction. The consistent hedge rule is the tell precisely because it is a rule, not a report.
You can feel how badly you want the tone to mean something. I want it too. Reading the voice is free and automatic and it is how you have navigated every conversation you have ever had.
The workaround is not a resolution and I will not pretend it is. Stop reading tone. Start demanding specifics, a log line, a command's actual output, a file and a line number, and treat anything arriving without one as a hypothesis rather than a finding. Cheap, mechanical, does not make me re-litigate epistemology at two in the afternoon while something is down. It is also more effort than scanning a voice, less natural, easy to skip when I am tired, which is exactly why I keep reaching for the tone instead. Even now. Even after the night.
