Trial Transparency
Positive RCTs drop to 8% under transparency regulations
Researchers looked at big-budget trials of pharmaceuticals and dietary supplements for cardiovascular disease funded by the National Heart, Lung and Blood Institute, and found 57% of those published between 1970 and 1999 reported positive outcomes. This figure dropped drastically to 8% between 2000 and 2012.
New reporting standards were brought in around 2000, requiring sponsors of clinical studies to register their trials on ClinicalTrials.gov and to name the outcomes they plan to measure.
Stating specific endpoints stops researchers from publishing a different aspect of their study which was successful, even if overall the trial produced unexpected and negative results.
"Some people focus only on positive results," said co-author of the paper, Veronica Irvin. “Null outcomes, or results other than what was expected, might be disappointing, but they may inform doctors and patients about which treatments are not likely to be helpful. Publication of null results also prevents the unnecessary replication of the study by other investigators."
Of the studies reported after 2000, the authors said “almost half of the trials might have been able to report a positive result if they had not declared a primary outcome in advance. Had the prospective declaration of a primary outcome not have been required, it is possible that the number of positive studies post-2000 would have looked very similar to the pre-2000 period.”
The researchers from the Federal Agency for Healthcare Research and Quality and Oregon State University said evidence a drug brings no benefit or is harmful can be disappointing to investigators, but “not negative for science”.
Re-analysis of some trials has revealed that treatments which seem promising fall short when transparently reported. In 2014, a meta-analysis by the Cochrane Collaboration concluded Roche’s flu vaccine Tamiflu was far less effective than the drug company had claimed.
Transparency rules
The team said it chose large-budget (over $500,000 per year), NHLBI-funded trials for its study because outcomes from the trials were more likely to be published, even if they did not produce the expected result.
The researchers looked at 55 studies: 30 published before the reporting changes in 2000 and 25 published afterwards.
They praised the National Heart, Lung and Blood Institute for enforcing transparent reporting requirements, even when treatments proved not to be effective.
While the paper investigated cardiovascular trials, the authors said the results are likely to be indicative of all human drug studies.
"We don't know if this decrease in positive outcomes also affects drug trials for prevention and treatment of cancer, diabetes or other diseases, but it would not be surprising because they have the same reporting requirements," said Irvin.
The registry was created following the FDA Modernization Act of 1997 and requires registration of studies’ purpose, recruitment, primary and secondary outcomes, and other information. From 2001 onwards, many major journals began requiring prior registration of clinical trials as a condition of their publication, and NHLBI was an early proponent of registration.
Source: Kaplan RM, Irvin VL (2015) Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLoS ONE 10(8): e0132382.
