Follow Me on Pinterest

Thursday, September 27, 2012

The cost of clinical trial data bias/loss, FDA's new job and the need for bold leadership.


The scandal of clinical trial data loss is eroding the fundamentals of evidence-based research and clinical medicine.

Before you right this post off as the stuff of conspiracy theories, fear-mongering, and 'alternative world views' consider that this view is shared by the likes of the FDA, the International Committee of Medical Journal Editors, the Cochrane Collaboration, and researchers at institutions like Johns Hopkins School of Medicine.

Here's the underlying premise as succinctly described by author Ben Goldacre:
"Drugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analysed using techniques that are flawed by design, in such a way that they exaggerate the benefits of treatments. Unsurprisingly, these trials tend to produce results that favour the manufacturer.
When trials throw up results that companies don't like, they are perfectly entitled to hide them from doctors and patients, so we only ever see a distorted picture of any drug's true effects. Regulators see most of the trial data, but only from early on in a drug's life, and even then they don't give this data to doctors or patients, or even to other parts of government. This distorted evidence is then communicated and applied in a distorted fashion."
Authors M. Todwin and J. Abramson summarize it thusly:
"Trials with positive results generally are published more frequently than studies that conclude that a new drug poses greater risks or is no more effective than standard therapy or a placebo. Furthermore, some articles may distort trial findings by omitting important data or by modifying prespecified outcome measures. Lack of access to detailed information about clinical trials can undermine the integrity of medical knowledge."
Here is a great list of very recent resources that may convince you of the merits of this concern:
Yesterday, the US Secretary of Health and Human Services announced (in an FR notice) that the FDA was now charged with ensuring all organizations comply with the heretofore enacted but relatively unenforced  requirement to submit all relevant clinical trial data to

For further commentary on this move see the following reports from:
What is abundantly clear to me is that the FDA is left almost powerless - and if not powerless than certainly without sufficient resources - to successfully enforce its new power.  This requires collective industry leadership.  Bold, industry-initiated standards, infrastructure and old-fashioned peer pressure.

Here's what I wish.  

I wish that as a cell therapy industry we - through organizations like ISSCR, ARM, ISCT, etc and leading publishers of some of our leading journals like Regenerative Medicine, Cytotherapy, Cell Stem Cell, Stem Cells, etc - would take a leadership position on an issue like this.

I believe that as a relatively small and nascent sector of the biopharma industry we are more likely capable of collaborating on something important like this than larger, more established [entrenched] and diverse sectors.  Of course it requires the political will and cajones.

The payoff from our sector in taking a leadership role on this issue could potentially be enormous in terms of providing our sector with truly transparent and useful data.  Perhaps even more important would be the public profile such leadership would provide the sector.  Such a move requires bold leadership, pain, and cost but this is the kind of stuff that moves the needle and goes down as critical pivot points in history. 

Just my thought for the day...



Alexey said...

Thanks for sharing your thoughts Lee! I completely agree on the existence of problem and on the necessity to tackle it in cell therapy.

As you mentioned, cell therapy is relatively nascent field, so we don't have a lot of problems with heavy biases, non reporting data and hiding failures. Not yet. But, we as a new field are getting to it and under the risk. I think, we can take advantage of these ongoing discussions and learn from the problems of Pharma and Biotech trials. We still have a chance to be a "nice and neat" field.

I agree that FDA can't keep up with everything. That's why a leadership by professional organizations is important. The problem is that (i) nobody discuss these kind of issues on the meetings and (ii) societies barely communicate to each other in any productive and beneficial way.

I think, the registration of clinical trials should be mandatory. Reporting all of data should be mandatory and in open access. Professional communities, should widely and openly discuss results of trials, track all failed trials and analyze it. Small professional organizations (neurosurgeons, plastic surgeons, orthopedic...) should give recommendations for appropriate design and data analysis. Also, they should give recommendations about the standards (for example for cell product preparation) and terminology (for example: can we call infusion of unseparated total mononuclear cells as "stem cell therapy").

There are few proposals to reform current clinical trials system. Currently there is a proposal on a table about exemption from Phase 3 (S. Korea). All these things should be widely discussed between professionals.

Lee Buckler said...

Thanks for the comment Alexey.

I think the cell therapy sector does have a problem with non-reporting of negative trial outcomes. I think the result is that we see very similar trials being run because outcome data from previous trials are not available. Not only might this avoid redundancy but it also might help us do better comparative studies on certain elements.

For example, even if one is studying the same cell type from the same source in the same indication as was studied in a failed trial, we could study a trial design using a different way of administering the cells to determine if that might make a difference with efficacy.