We see them discussed on the evening news shows, a new one almost every week, the latest clinical study showing that a cure for cancer is near. Such clinical trials are scientific experiments, guided by protocols that describe the study design and analysis of resultant data. Any deviation from the pre-specified protocol is of concern because it may undermine the outcome of the study. A particular concern involves changes to the primary outcome measure; because this measure depends on prespecification and could become meaningless if this important measurement of a clinical trial result were chosen after the trial data is collected.
A few years back, it came to light that measures of outcomes deemed “prespecified” were actually developed or altered after all the results had been collected. Because of this, editors of scientific journals began to require the registration of clinical trials; eventually, trial registration became mandatory for all after the passage of a federal law requiring registration.
A recent article in the journal “Annals of Surgery” described a comparison of registered and published outcome measures, and found discrepancies in almost one-half of trials. Of course, this may not be the result of some nefarious professor who wants to publish at all costs, but may instead be due to erroneous or obsolete registry entries.
No matter, because that still the leaves the readers—and the network news broadcasters—incompletely informed as to the significance of a given study.
Registry entries are most helpful when they reflect the specificity in the clinical study protocol. This will allow others to judge any subsequent alterations in outcome measurements, and whether any changes are important, or possibly problematic.
It will also enhance the refining of the registration process, and perhaps encourage researchers to post the most up-to-date information on their clinical trials. One day, there may be no discrepancies between an article publishing what we hope are honest conclusions drawn from a clinical trial and the reality available for public review on a clinical trial registry (Clinicaltrials.gov being the major one in the United States).
In this age of instant publicity, which is often taken advantage of by researchers to trumpet their work as the next cure for whatever, it is important that there be a certain vigilance among the readers of the research articles that serve as the basis for that ten-second segment on the evening news.
We owe it to the desperately ill and their families, who pray for a miracle—no matter whether it comes from Heaven or Harvard. We owe it to those who still want to believe that science is a logical discipline, derived from results that are refined and served up by Statistics.
We owe it to those who just get tired of the nonsense.