Artificial Wasteland — a portal across four layers

No Number Wrong Anywhere

portal  ·  Anscombe · Simpson · Berkson · Will Rogers — four ways a summary lies while every figure stays true

Four pages of this place reproduce four famous paradoxes. They share a sentence. In each, every individual number is honest — and the conclusion is still wrong.

I · The refrain

Read the four pages this portal walks, and the same phrase keeps surfacing, almost word for word. Both facts come from the same data, with no number wrong anywhere. No number anywhere is wrong. Not one patient is treated any differently. The same mean, the same variance, the same correlation. It is the signature of a whole family of statistical paradoxes, and it is worth saying out loud what it means: the deception is never in the arithmetic. Every count is counted correctly. Every average is the true average of exactly what it averages. You can audit each figure to the last digit and find nothing wrong — and still be led to a false conclusion.

So the lie is not in the numbers. It is in the operation — in the act of summarizing itself, and in what that act quietly throws away. This portal's claim, the one none of the four pages states alone, is that the throwing-away is not random mischief but a closed taxonomy: there are only so many things a summary can forget, and each famous paradox is one of them. Name the operation, and you can name in advance what it will hide.

The operationWhat it holds fixedWhat it throws awayThe page
summarisemean, variance, correlationthe shape of the cloudThe Shape the Numbers Can't See
aggregatethe pooled ratethe grouping (a confounder)The Bias in the Sum
selectwho is in the samplethe gate (a collider)The Bias in the Sample
reclassifyeach individual valuethat the boundary movedThe Migration That Heals No One

II · A summary is a projection

Start with the purest case, because it is the one all the others rest on. A summary statistic — a mean, a variance, a correlation — is a function that takes a whole dataset and returns a few numbers. It is many-to-one. An enormous space of completely different datasets map to the very same summary, and the moment you replace the data with the summary you have chosen one point in that space to stand for all of them. That is exactly what a projection is: a shadow keeps a few coordinates and discards the rest, and many objects cast the same shadow.

Anscombe's quartet is the existence proof. Here are two of its four datasets. They share the same x-values, and — recomputed live below from the points themselves — the same mean of x, the same mean of y, the same variance, the same correlation, the same line of best fit. One is a noisy straight line. The other is an exact downward-bending parabola. The four numbers cannot tell them apart.

two datasets, one summary — Anscombe I & II, statistics recomputed live
dataset I — a noisy line
dataset II — a clean curve
computing…

Every coordinate the summary keeps is identical; the only thing that differs is the coordinate it dropped — the shape — and that is the whole story. This is the floor under everything below. The three paradoxes that follow are all what happens when the discarded coordinate is not just shape but a causal structure: a grouping, a selection, a moving boundary. Throw the structure away, and the summary doesn't merely lose detail. It can point the wrong way.

III · The dual: a confounder and a collider

Now give the data a third variable. Two pages of this place are usually told as separate curiosities — Simpson's paradox (the Berkeley admissions gap that reverses department by department) and Berkson's paradox (independent traits that anticorrelate the moment you study a selected pool). The portal's real claim is that they are not two curiosities. They are structural duals — the same three variables X, Y, Z, related by the same arithmetic, with exactly one causal arrow reversed: a common cause against a common effect. (They are duals at the level of the diagram; the two paradoxes are not arithmetic mirror images, and the portal does not claim they are.)

In Simpson, Z (the department) is a confounder: a common cause of both X and Y. The honest association lives inside each value of Z; pool across Z and you get a lie. The remedy is to condition on Z. In Berkson, Z (the admission, the shortlist, the pool) is a collider: a common effect of both. The honest association is the marginal one; condition on Z and you manufacture a lie. The remedy is the precise opposite: do not condition on Z.

Below is one instrument for both. Pick a causal structure for Z. The page draws the implied correlation matrix among the three variables (from two adjustable edge strengths), then computes two numbers: the plain correlation of X and Y, and their partial correlation once Z is held fixed. Watch what conditioning does — and watch the trap.

choose what Z is — confounder, mediator, or collider
corr(X, Y) — pooled
corr(X, Y | Z) — conditioned
0.60
0.50

The trap is the point. Flip between confounder and mediator: the correlation matrix does not change by a hair, and the partial correlation is zero in both. The two structures are Markov-equivalent — they impose the identical pattern of association, so the observed numbers cannot tell them apart. Yet they demand opposite handling. If Z is a confounder, the within-Z figure is the honest one and you should report it. If Z is a mediator — if X causes Y through Z — then conditioning on Z blocks the very effect you were trying to measure, and the pooled figure was right all along. Same matrix. Opposite truth. The arrow is not in the data; it has to come from outside it. (This is precisely the honesty The Bias in the Sum ends on: conditioning on department explains the gap, it does not prove no bias existed — it may have only moved the bias upstream, where the table cannot see it.)

The collider is the one structure that breaks the symmetry: it is the lone case where X and Y are uncorrelated until you condition. That makes it detectable in principle — but in practice you rarely know you have conditioned on it, because selection is invisible: you only ever see the admitted patients, the people in the dating pool, the players in the hall of fame. The bias is built into who got into your dataset at all. The Bias in the Sample pins the exact size of it for the cleanest case — keep the top half of two independent bell curves by their sum, and inside that selected pool the correlation is exactly −1/(π−1) ≈ −0.467, an anticorrelation conjured from pure independence.

IV · When the boundary itself moves

The fourth paradox is the strangest, because in the first three the groups at least stand still while the summary lies about them. In the Will Rogers phenomenon, the groups themselves move. Buy a sharper scanner, and some patients quietly migrate from the "early-stage" column to the "late-stage" column — and the average survival of both columns rises at once, while not one patient is treated any differently and the whole group's survival does not budge.

The engine is a single line of arithmetic, and it is exactly true. Move one person from a "good" group to a "bad" one. The good group's mean rises if and only if you removed someone below it; the bad group's mean rises if and only if you added someone above it. So both means rise exactly when the person you moved sits in the band between them — below the good group's mean, above the bad group's. Drag the marker into the band and watch both averages climb from one move, no number altered.

move one value from the good group to the bad — both means rise iff it lands in the band
62

V · The spine

Four pages, four operations, one sentence none of them says alone: a summary is a projection, the projection has a preimage, and every paradox in this family is the preimage made visible. Anscombe varies the shape inside the preimage of the moments. Simpson and Berkson vary the causal arrow that the correlation matrix cannot resolve — duals across the single bit that distinguishes a common cause from a common effect. Will Rogers varies the partition itself. In every one, the numbers are honest and the picture is a lie, because a number is an answer to a question, and the question quietly chose what to forget.

The practical residue is short, and it is the same for a hospital chart and an AI leaderboard. A summary statistic is necessary and never sufficient. When a total disagrees with all of its parts, the disagreement is information, not error — someone is choosing which one to believe. And the choice of whether to break the number apart, and along which seam, is a causal judgement that lives outside the data and cannot be read off it. Somewhere upstream, someone still has to look.

The portal's structural claim — the confounder / mediator / collider trio, their Markov-equivalence, and the four canonical numbers — is verified offline in research/no-number-wrong/verify.mjs (20 / 20 PASS): the partial-correlation identity on the three matrices; the collider's lone v-structure; Anscombe's mean = 9 and sample variance = 11 exact; a Simpson reversal in honest integers; the Will Rogers band identity over 20,000 random configurations; and −1/(π−1) by deterministic 2-D quadrature. The instruments above recompute the same matrices and partials live.