It Ain't Necessarily So

There's an elegant article at Medscape by Christopher Labos called "It Ain't Necessarily So: Why Much of the Medical Literature Is Wrong." Key points:

Given a statistical association between X and Y, most people make the assumption that X caused Y. However, we can easily come up with 5 other scenarios to explain the same situation.

1. Reverse Causality

Given the association between X and Y, it is actually equally likely that Y caused X as it is that X caused Y.

2. The Play of Chance and the DICE Miracle

Whenever a study finds an association between 2 variables, X and Y, there is always the possibility that the association was simply the result of random chance.

The Frequency of False Positives

It is sometimes humbling and fairly disquieting to think that chance can play such a large role in the results of our analyses.

3. Bias: Coffee, Cellphones, and Chocolate

Bias occurs when there is no real association between X and Y, but one is manufactured because of the way we conducted our study.

4. Confounding

Confounding, unlike bias, occurs when there really is an association between X and Y, but the magnitude of that association is influenced by a third variable. 

Real-World Randomization

Confounding can be dealt with through randomization. When study subjects are randomly allocated to one group or another purely by chance, any confounders (even unknown confounders) should be equally present in both the study and control group. However, that assumes that randomization was handled correctly.

5. Exaggerated Risk

Finally, let us make the unlikely assumption that we have a trial where nothing went wrong, and we are free of all of the problems discussed above. The greatest danger lies in our misinterpretation of the findings.