Why medical clinical trials are so wrong so often (2024)

The finding comes from a survey of 37 clinical trials that had been analyzed more than once inthe past 64 years. In 13 cases, the re-dos came to entirely different conclusions than the original study.

Second looks at the results of clinical trials are hard to come by, often because scientists are reluctant to share raw data from their research. But the research team behind study said their findings prove the need for more data sharing and reanalysis of clinical trials.

“There is increasing lack of trust in whether the results of published, randomized trials are credible and can be taken at face value,” John Ioannidis, director of the Stanford Preventative Research Center and the senior author of the study, said in a press release.“Making the raw data of trials available for re-analyses is essential … for re-evaluating whether the original claims were correct.”

This study, “Reanalyses of Randomized Clinical Trial Data,” comes in the wake of a number of controversies surrounding questionable interpretations of scientific data. Earlier this year, the antiviral drug Tamiflu came under fire when the Cochrane Collaboration, a British organizationthat reviews clinical trial data, found trial results actually showed the drug offered “small benefit and increased risk of harms” when used to treat the flu. The nonprofit reached a similar conclusion about Relenza, another flu drug, based on a reexamination of data from the manufacturer’s clinical trials.

“The process [of analyzing clinical trials] is so subjective, you can twist it any way you want,” Cochrane Collaboration researcher Tom Jefferson told Science in response to the Stanford study. He added drug companies are often particularly unwilling to publish their raw data so that independent researches can reanalyze it: It took Cochrane Collaboration four years and a lawsuit to win the right to access data from clinical trials of Tamiflu from the drug’s manufacturer.

According to Shanil Ebrahim, a clinical epidemiologist at Stanford and a co-author of the study, differing interpretations of the data don’t always mean that the research was biased, or even that the original conclusion was incorrect. But the fact that data is so open to interpretation means that studies of clinical trials ought to be examined more than once to assess their validity, he told The Washington Post.

But, you know, that’s just one study. Shouldn’t someone take a second look at Ebrahim’s conclusions?

“Go ahead,” he said. “We will make all of our data available if anyone wants to reanalyze it.”