LEARNED IT · 08/26
Three sharpness metrics, three different wrong answers — and Brendan's eye beat all of them
Every automatic sharpness measure found a different way to reward something other than sharpness. Judging by whether crater rims are actually resolved beat all three.
Ranking the night's 15 stacks, I tried three metrics in succession. Each one failed differently, and each failure looked like a result.
| attempt | measured | why it was wrong |
|---|---|---|
| whole-frame Laplacian variance | the entire image | the limb against black sky is one enormous edge and swamps everything. Run 000091 scored 2.36 on that single boundary while the next best managed 0.83 |
| surface-only (limb eroded out) | inside the disc | brighter images win regardless of sharpness — Laplacian energy scales with contrast, and 000091 peaked at 155 against 000092's 111 |
| contrast-normalised | scale-free sharpness | dim, noisy frames win by dividing near-zero signal by near-zero contrast. Run 000083, contrast 1.21, "won" outright |
Meanwhile Brendan looked at the images and said run 000092 was sharpest, because the craters on the left are well defined.
He was right. Compare only images of similar exposure (contrast 14-19) and the normalised measure agrees — 000092 at 0.00390 against 000091's 0.00260, 50% sharper. 000091 was never the sharpest; it was the brightest, with the strongest limb edge, and all three metrics found a different route to rewarding that.
The principle
A single-number proxy for image quality is a hypothesis, not a measurement. Before trusting any ranking, ask what else correlates with a high score — brightness, noise, how much black sky is in frame, one high-contrast edge — and check whether the winner is merely maximising that instead.
Judging whether real features are resolved is not a lesser method than computing a number. It is a better one, because the eye is testing the thing you actually care about while the metric tests a proxy for it.
Three failures in one session, on top of the same trap in [[sharpness-metrics-reward-noise]]. The rule now: compute the number, then look at the image, and when they disagree, the image wins.