In 1930, Scotland ran the biggest nutrition experiment anybody had attempted. Twenty thousand schoolchildren across 67 Lanarkshire schools, four months, everybody weighed and measured at both ends. Ten thousand kids got free milk every day. Ten thousand got nothing and served as the control.
The teachers picked who got the milk.
Not on purpose, exactly. They were allowed to move children between the groups, and standing in a Lanarkshire classroom in 1930, looking at kids who were visibly underfed, they did the human thing. The skinny ones went into the milk group. The sturdy ones got left in the control.
Twenty thousand children, four months of measurements, and the study couldn't answer its own question.
The critique that buried it ran in Biometrika the following year, written by a man publishing under the name "Student." His verdict: a hundred identical twins, split into pairs, would have produced a more accurate answer than all twenty thousand kids. Scale was the problem. Everybody was so busy running the biggest experiment ever attempted that nobody stopped to ask whether it could be read.
We are burning budget the same way right now. Generative AI made ad concepts nearly free, so a team ships twenty a week, but live traffic still costs what it always did: roughly $500 to $2,000 per variant to read one with any confidence. An audit of 2,288 A/B tests found only 19% produced a statistically significant winner. Four out of five tests are tuition.
Today's idea is a creative triage layer for performance marketing agencies. Rank a batch of Meta ad concepts before any of them touch paid traffic, then keep score of every call against what actually happened in market. That ledger of predictions and outcomes is the part nobody can copy off you. It starts as a $1,500 to $3,000 concierge sprint with no software at all, and twenty agencies at $1,500 a month is $30,000 MRR.
Read the full playbook here:
Generative AI made ad concepts nearly free while live testing stayed expensive. Only 19% of tests produce a defensible winner, and nobody owns the layer deciding which concepts earn traffic.
From the Vault:
The SBA's first Critical Suppliers Prize exposed a gap: 239,000 US manufacturers make six-figure equipment decisions with no assembled capital evidence. One operator can productize the proof.
Four states now regulate surveillance pricing in groceries, two effective October 1, and most regional grocers cannot prove which customer data reached a displayed price.
