A Model for When Results Look More Certain Than They Should

In the last post, I wrote about a pattern I keep seeing in pulmonary function testing.

Results that look clean. Technically valid. Aligned with standards.

And still… something feels off.

Not obviously wrong. Not clearly flawed.

Just more certain than they should be.

I kept coming back to that.

It didn’t seem random. It showed up in a particular kind of situation—when a complex process gets compressed into a clean, final output.

By the time you see the result, most of what went into producing it is gone.

The coaching. The maneuver. The variability. The context.

What’s left is something that looks stable, bounded, and reliable. And because it looks that way, it’s treated that way.

At some point, I realized I needed a better way to think about it.

So I built a model.

Not to determine what’s right or wrong, but to map something simpler: how much confidence a result appears to carry relative to how much distortion may have been introduced along the way.

The idea is pretty simple.

Some results have low distortion and deserve high confidence. Those are the easy ones.

But others have high distortion and still appear to deserve high confidence. Those are the ones that tend to slip through, because nothing about them looks like a problem.

The model doesn’t tell you what the correct answer is. It doesn’t replace clinical judgment.

It just gives you a way to see when confidence might be coming from the structure of the system rather than the underlying signal.

I’ll walk through what this looks like in practice next.