(Ioannidis 2005) Why most published research findings are false


Ioannidis, John P. A. Why most published research findings are false. PLoS Medicine. 2005 Aug; 2(8):e124. Available from: http://dx.doi.org/10.1371/journal.pmed.0020124.

Ioannidis presents a statistical argument that most published results of scientific research are false. The positive predictive value (PPV) of a reported result is the probability that it is actually true, given that it has been reported as a positive result. Studies are often evaluated, even by scientists, as if PPV were equivalent to statistical significance, which is not the case. When a study reports a relationship between variables, its PPV depends not only on its statistical significance level, but also on three other characteristics: the study’s statistical power (the ability to detect a true relationship if one exists), its bias (the probability of reporting a positive result when the evidence is actually negative), and the ratio R of true to false hypotheses tested in studies in the relevant scientific field. For most fields of empirical science, it is very difficult to get a PPV better than 50%, and most studies have very low PPV, particularly in fields where bias is strong or R is very low. Ioannidis emphasizes that although financial conflicts of interest are rampant in biomedical research, bias from other sources is also common:

Scientists in a given field may be prejudiced purely because of their belief in a scientific theory or commitment to their own findings. Many otherwise seemingly independent, university-based studies may be conducted for no other reason than to give physicians and researchers qualifications for promotion or tenure. Such nonfinancial conflicts may also lead to distorted reported results and interpretations. Prestigious investigators may suppress via the peer review process the appearance and dissemination of findings that refute their findings, thus condemning the field to perpetuate false dogma.

Despite the somewhat technical nature of the argument, this article is essential reading for anyone who wishes to understand the limitations of the present-day scientific literature.

 

Shortlink to this page: http://is.gd/ZsMEB0
Last revision: April 5, 2013