Math & Algorithms Sep 2026 · 5 min

Why we suck at predicting weather... and how I didn't help.

Adam Faust · an independent test, not peer-reviewed

This is going to be a little bit of math. But you clicked into the personal blog of the guy who runs an AI company, on your own free time — so a little math doesn't scare you. Right?

Weather prediction has a hard ceiling built into it, and it's not really an engineering problem — it's math. Small errors in today's measurements grow, roughly, exponentially as you project forward; a discovery from the 1960s (the "butterfly effect" is the popular name for it) that's the actual reason forecasts get less reliable the further out they go, no matter how good our models or our computers get. That's real, it's been known for sixty years, and it's not what this post is about.

This is about something smaller and further upstream. Before chaos even gets a chance to compound your errors, you need the underlying fluid-flow physics in the simulation to be right in the first place — the actual mechanics of how air and heat move. A huge amount of engineering depends on getting that part right: aircraft wings, turbine blades, wind farms, ship hulls, and yes, the weather models too. Nobody can run the real experiment for every case; it's too expensive, too slow, or physically impossible before the thing is built. So engineers run a simulation instead, and trust the result.

But a turbulence simulation can look finished — numbers settle down, it stops changing between iterations — while still being quietly wrong. Actually knowing whether a simulation "converged" to something physically real, versus just stopped moving, is a genuinely hard problem on its own, separate from the chaos problem entirely.

Yes, I know it's a little strange how I spend my free time, but I decided to see if I could help with that piece of it. I built two checkers meant to catch that difference without needing the real answer already in hand, and pointed both at the best real experimental turbulence data available — the kind physicists have spent years measuring in real channels of real flowing fluid, not simulations.

The Dissipation Gate

a physical relationship that should hold almost exactly, in the part of a turbulent flow closest to a wall, if a simulation is behaving the way real turbulence does

a real, measured constant from decades of turbulence physics (≈ 0.384) — not something I chose, something borrowed from nature

how far from the wall you are, measured in the flow's own natural units

how fast turbulent energy is turning into heat at that exact point

how much I let it drift from "perfect" before I called it broken

Tested by me · disproven · not peer-reviewed

Here's what actually happened, and why — because the reason turned out to be more interesting than the failure itself.

That relationship, κy⁺ε⁺ landing close to 1, is only exactly true in an idealized version of the region right next to the wall — a mathematical limit that real turbulence only approaches, never fully reaches. Even a genuinely correct simulation drifts away from exactly 1, just from ordinary physical effects near the wall that have nothing to do with anything being broken. So "how far from 1" was quietly measuring two different things at once: real errors, and normal physics, with no way to tell which one moved the number. One version of my checker, tuned tight around the idealized case, never saw enough drift to distinguish the two and just went silent — a flat, constant nothing, on every case I gave it. The looser version caught real problems about as often as a coin flip would.

Then I found the part that actually bothered me. I deliberately broke six simulations on purpose — injected obvious, known errors — specifically to see if either checker would notice. Neither one caught a single one. Not because the errors were subtle. Because each checker was built to watch exactly one narrow relationship, in exactly one part of the flow, and none of my injected errors happened to disturb that one specific thing. Everything else about those six simulations could have been catastrophically wrong, and both checkers would have signed off without hesitation.

WHY THIS ACTUALLY MATTERS TO THE PEOPLE WHO USE THIS STUFF

An engineer deciding whether a wind turbine blade design is safe to build. A national weather service trusting a turbulence model as one input to tomorrow's forecast. An aerospace team deciding whether a wing shape is ready for a wind tunnel, or straight to production. None of these people re-derive the physics by hand every time — they trust that somewhere in the pipeline, a check like this one is doing its job. A narrow check isn't dishonest. It's just blind to everything outside the one thing it was built to watch — and that blindness is invisible from the outside, which is exactly what makes it dangerous.

In your life

You've almost certainly never run a CFD simulation, but you've flown on an aircraft shaped by one, driven past a wind farm designed with one, and checked a weather forecast partly built on one. You will never personally audit any of it. Most of the time, that trust is deserved. This is a story about the times it might not be, and about how a check that looks rigorous can still only ever see the one narrow thing it was built to see.

The bigger pattern

Every field that can't afford to run the real experiment every time — climate modeling, reactor safety, drug interactions, financial risk — ends up leaning on exactly this kind of self-check: a machine built to tell you whether to trust another machine, without a human re-deriving the answer from scratch. That's not a flaw in how we do engineering, it's the only way to do it at this scale. But a check that watches one relationship can only ever catch the failures that show up in that one relationship. Everything else passes through in silence.

So: the chaos problem in weather prediction is real, well-studied, and not something anyone is going to fix with a cleverer check. The much smaller, much more fixable problem — whether the physics underneath the chaos is even being simulated correctly — is exactly the kind of thing I thought I might be able to help with. I didn't. Not yet, anyway.

A note on how to read this: I'm an independent, amateur scientist. I don't work at a university, and nothing here has been peer-reviewed. I tested this myself, as carefully as I could, against real published data — but I can't promise I'm the first to notice this, or that I haven't made a mistake somewhere. If you know more about turbulence certification than I do, I'd genuinely like to hear where I got this wrong, or what I should test next.