Nature News has a very interesting article by Richard Van Noorden about retractions of scientific papers. I hadn't thought about some of the points raised, and I think many of you will likewise find them and the associated graphic compelling. Some excerpts:
[R]etraction notices are increasing rapidly. In the early 2000s, only about 30 retraction notices appeared annually. This year, the Web of Science is on track to index more than 400 . . . even though the total number of papers published has risen by only 44% over the past decade.
When the UK-based Committee on Publication Ethics (COPE) surveyed editors' attitudes to retraction two years ago, it found huge inconsistencies in policies and practices between journals. . . . That survey led to retraction guidelines that COPE published in 2009. But it's still the case, says Wager, that "editors often have to be pushed to retract".
Other frustrations include opaque retraction notices that don't explain why a paper has been withdrawn, a tendency for authors to keep citing retracted papers long after they've been red-flagged . . .and the fact that many scientists hear 'retraction' and immediately think 'misconduct' — a stigma that may keep researchers from coming forward to admit honest errors.
[A]s more retractions hit the headlines, some researchers are calling for ways to improve their handling. Suggested reforms include better systems for linking papers to their retraction notices or revisions, more responsibility on the part of journal editors and, most of all, greater transparency and clarity about mistakes in research.
[R]etraction notices are increasing rapidly. In the early 2000s, only about 30 retraction notices appeared annually. This year, the Web of Science is on track to index more than 400 . . . even though the total number of papers published has risen by only 44% over the past decade.
When the UK-based Committee on Publication Ethics (COPE) surveyed editors' attitudes to retraction two years ago, it found huge inconsistencies in policies and practices between journals. . . . That survey led to retraction guidelines that COPE published in 2009. But it's still the case, says Wager, that "editors often have to be pushed to retract".
Other frustrations include opaque retraction notices that don't explain why a paper has been withdrawn, a tendency for authors to keep citing retracted papers long after they've been red-flagged . . .and the fact that many scientists hear 'retraction' and immediately think 'misconduct' — a stigma that may keep researchers from coming forward to admit honest errors.
[A]s more retractions hit the headlines, some researchers are calling for ways to improve their handling. Suggested reforms include better systems for linking papers to their retraction notices or revisions, more responsibility on the part of journal editors and, most of all, greater transparency and clarity about mistakes in research.
Yes, anyone at all interested in this subject should subscribe to the Retraction Watch blog.
ReplyDeleteSurging retractions aren't per se bad - I suspect it simply means that transparency is coming to the journal process. And as always, the goal of transparency is to make it more likely that process flaws will get spotted and improved.
Activated researching e-patients too often get insulted by clinicians who aren't yet aware of the flaws in their own system. We've written "training articles" on the e-patient blog over the years, on research issues and understanding statistics.
On a related note, Society for Participatory Medicine member Peter N. Schmidt, of the Parkinsons Foundation, is particularly outspoken about the mathematical skills of the editors and curators who are supposed to ensure quality; he goes into depth about editors and conference organizers who accept papers that use completely inappropriate statistical measures, e.g. using metrics that are valid in Gaussian distributions on data sets that aren't Gaussian.
Retractions are admissions of error (of ignorance, lack of due diligence, or duplicity) by author or editor. "Reversals" represent the related but far more interesting revelation that we've all been duped, and it is remarkable how frequent reversals occur in medical practice. See V. Prasad et.al., Arch Int Med v.171, Oct 10 2011, pp 1675-76. We take pride in practicing evidence based medicine, but most of us delegate to others the job of evaluating the strength of evidence, and as a result subject our patients to far more snake-oil than we know.
ReplyDeleteFile this under 'unintended consequences' of rewarding scientists for quantity rather than quality, resulting in high stakes turnover and occupationally strategic, rather than problem-oriented, risk-taking. Again, an infrastructure problem not addressed by moving players on the board. It will get worse with impending budget constraints, and winner-take-all funding.
ReplyDeleteMedical education is another problem altogether. Here, practitioners learn by the argument-by-authority method, time honored and error-prone. See-one, do-one, teach-one is a poor pedagogical infrastructure for evaluating scientific information. Shouldn't physicians be required to spend a year in public health epidemiology and clinical effectiveness research? They would be in a better position to improve the methods behind this current madness.