Research methods · Study designs

Bias, confounding & the healthy-user effect

Imagine two studies of the same supplement. The first follows 2,000 people; the second follows 200,000. Your instinct says the big one is more trustworthy, and for one kind of error that's exactly right. With more people, the random wobble — the luck of who happened to land in each group — averages out. But suppose both studies recruited the same way: they compared people who chose to take the supplement against people who didn't. Then both share the same hidden flaw, and the big study doesn't escape it. It just measures the flawed answer with more decimal places. A bigger biased study isn't closer to the truth. It's a more confident wrong answer.

Two different problems, one of which more data can't fix

Random error is noise. Flip a fair coin ten times and you might see seven heads; flip it ten thousand times and you'll land very close to half. Sample size is the cure. Bias is a tilt — a scale that reads three pounds heavy. Weighing yourself a thousand times doesn't find the error; it just gives you a thousand readings that are all three pounds wrong, and a tighter average around the wrong value. That's why the size of a study tells you how precise it is, not how correct it is.

The shape of confounding

The healthy-user effect is one face of a deeper problem called confounding. A confounder is a third thing that causes both the behaviour you're studying and the outcome you care about. When it's lurking, the behaviour and the outcome rise and fall together — not because one causes the other, but because the confounder is pulling both strings. The study sees the two move in step and reports a link. The link is real on the page and spurious in the world.

Three named tilts to suspect first

The healthy-user effect: people who adopt a healthy habit — a daily vitamin, a screening test, a "clean" diet — tend to do many other healthy things too. They're a self-selected group, healthier before the habit ever started. The sick-quitter effect is its mirror image: people drop a habit because they got sick, so the "non-users" quietly fill up with the unwell. Confounding by indication shows up with drugs: doctors prescribe a treatment to the patients who most need it, so the treated group is sicker by design, and the drug can look harmful when it's the illness doing the damage.

Work one, then finish one

Worked: the famous moderate-drinking "J-curve" — surveys long showed light drinkers outliving teetotallers, and headlines turned that into "a glass of wine is good for you." Walk it back through the sick-quitter trap. Who's in the "non-drinker" group? Not just lifelong abstainers — it's also people who used to drink and quit, often because a doctor told them to after a heart scare, a liver warning, or a cancer diagnosis. Those former drinkers are sick, and now they're counted as non-drinkers. That drags the non-drinkers' health down and makes moderate drinkers look protected by comparison. Separate lifelong abstainers from sick-quitters and much of the "protection" evaporates. The drink didn't guard their hearts; the comparison group was rigged by who left it.

Your turn: a study finds that people who eat mostly organic food get less cancer. Name the healthy-user effect hiding inside it before you switch your grocery list. (Organic eaters tend to be richer, exercise more, smoke less, drink less, and see doctors more often — all of which lower cancer risk on their own. The organic diet may be a marker of that whole healthier life, not a cause of the lower risk. The food gets the credit; the lifestyle did the work.)

Why this matters

This is the single most useful reflex you can carry into a supplement aisle. Walk down it and nearly every claim — fish oil for your heart, a multivitamin for longevity, a greens powder for "immunity" — traces back to observational data showing that people who take the thing are healthier than people who don't. The healthy-user effect explains most of those findings on its own, and it's the default explanation you should rule out before believing any observational health claim, no matter how big the study or how confident the podcaster. When researchers say a result is "adjusted for confounders," that's real work, but it's only partial credit: they can only adjust for the confounders they thought to measure and measured well. The careful-person-ness that drives the healthy-user effect is exactly the kind of thing that never fully fits in a column. The honest move, before you buy, is to ask whether a randomised trial — where a coin, not the person, decides who takes the pill — ever showed the benefit. For most supplements, it didn't.