09 May 2012

Sarewitz on Bias in Science

In the current issue of Nature, Dan Sarewitz has a column about the threat posed by bias to scientific research.  (The image above is a screenshot of a paper cited by Sarewitz, which is by J. Ioannidis, 2005, "Why Most Published Research Findings are False").

Sarewitz explains the systemic findings of bias in clinical trials as follows:
Like a magnetic field that pulls iron filings into alignment, a powerful cultural belief is aligning multiple sources of scientific bias in the same direction. The belief is that progress in science means the continual production of positive findings. All involved benefit from positive results, and from the appearance of progress. Scientists are rewarded both intellectually and professionally, science administrators are empowered and the public desire for a better world is answered. The lack of incentives to report negative results, replicate experiments or recognize inconsistencies, ambiguities and uncertainties is widely appreciated — but the necessary cultural change is incredibly difficult to achieve.
Why are such findings of bias turning up in clinical trials?
A biased scientific result is no different from a useless one. Neither can be turned into a real-world application. So it is not surprising that the cracks in the edifice are showing up first in the biomedical realm, because research results are constantly put to the practical test of improving human health. Nor is it surprising, even if it is painfully ironic, that some of the most troubling research to document these problems has come from industry, precisely because industry’s profits depend on the results of basic biomedical science to help guide drug-development choices.
Is the problem of bias limited to clinical studies?
It would therefore be naive to believe that systematic error is a problem for biomedicine alone. It is likely to be prevalent in any field that seeks to predict the behaviour of complex systems — economics, ecology, environmental science, epidemiology and so on. The cracks will be there, they are just harder to spot because it is harder to test research results through direct technological applications (such as drugs) and straightforward indicators of desired outcomes (such as reduced morbidity and mortality).
Read the whole thing here.