A Growing Crisis in Schizophrenia Drug Development: Failing Signal Detection at North American and Other Clinical Trial Sites

See article by Gopalakrishnan et al

The article by Gopalakrishnan and colleagues1 is an important update on an issue the US Food and Drug Administration (FDA) first reported on in 2011.2 There was a trend of increasing placebo response in schizophrenia trials during the pre-2009 period, and that has worsened even more during the post-2009 period. During these same periods, the effect in the drug arms has largely remained the same. The result has been a steep decline in the treatment effect and an associated steep decline in schizophrenia trial success rates. The article breaks the findings down by roughly 7- to 10-year epochs over the entire time period, further highlighting the steady trend for worsening results for this 24-year period.

A critical issue to emphasize is that these dismal findings are coming predominantly from North American studies. While the decline in trial success rates from pre-2009 to post-2009 falls from 78% to 57% for the cohorts overall, the decline for North American trials is a striking 81% pre-2009 to 25% post-2009. What is missing from this article is a breakdown by program of results for trials done in North America versus multiregional trials, which would have been useful. In any case, during these 2 broad epochs (pre-2009 and post-2009), the percentage of trials that are North American has reversed from 68% to 29%. Is it any wonder that pharmaceutical companies are abandoning North American sites with such dismal results coming from these sites? It is true, of course, that these most recent findings are coming from only 3 programs involving North American trials (a total of 14 trials); however, at the present rate of decline, there may soon be no schizophrenia programs conducted at North American sites. The FDA may have to abandon its prerogative to demand that at least some data come from US sites or, alternatively, choose to refrain from approving any new drugs for schizophrenia in the United States.

Other groups have also reported on this problem of increasing placebo response and decreasing effect size in schizophrenia trials,3-7 but, except for Kemp et al,3 these were limited to published data. The FDA has access to all of the studies done in drug development programs and is, therefore, uniquely positioned to report on this problem. Only the FDA has been able to fully characterize the increasing problem in North American schizophrenia studies. Meta-regression analysis of summary data from published articles has been the main approach to trying to understand what is driving these changes in schizophrenia trials,5-7 but this approach has significant limitations. Most agree that a better approach would be to create a large, shared patient-level database from the now accumulating data from hundreds of placebo-controlled schizophrenia studies to do the kind of in depth exploration that may yield more definitive answers to this serious problem. Such an effort would require a willingness of pharmaceutical companies to contribute their data for such explorations and contribute funding for such an endeavor. Thus far, it would not appear there is sufficient support for this to happen.

The conclusion by the FDA that “close attention to trial conduct and a reexamination of design elements may be warranted”1 because of these trends understates the problem, in my view. These findings should be alarming, not only to pharmaceutical companies and regulators, but also to clinicians, the academic community, and of course patients and their families. To follow up on the FDA’s suggestion that there is a need to look at trial conduct and reexamine design elements, which aspects of trial conduct and design need to be looked at? There are several aspects of drug study conduct that need to be explored in general as an approach to improving study outcome.8 Of the various potential factors to examine, patient selection is perhaps most important for the placebo response findings revealed in this articles. Who are these acutely exacerbated schizophrenic patients at North American sites who improve so dramatically on placebo? These current fairly robust placebo responses in schizophrenia studies in North America seem to be inconsistent with what clinicians usually consider to be patients with acutely exacerbated schizophrenia. There are of course many potential factors that might contribute to this problem. Clinical trial sites are incentivized to recruit patients, but without any particular reward for getting the right patient (payment is generally for randomized patients and not for quality of patient selection). The same is largely true for contract research organizations. There is no reason why incentives for improved quality in patient selection couldn’ t be included in contracts. One would hope that oversight by companies and regulators would help to ensure quality over quantity in this process, but pharmaceutical company staff also face recruitment pressures that may outweigh the incentive to get the right patients. Regulators are probably too far removed from the actual process to have any meaningful impact. There has long been an impression that patients added late in trials as recruitment deadlines approach may have larger placebo responses and perhaps contribute to trial failure, but it would be useful to have actual data accumulated across trials to support this view.

One approach that has been utilized to improve selection of the right patients is to have some kind of outside adjudication of patients selected by sites to ensure these actually are patients who meet entry criteria. Various vendors have developed approaches to doing this, ranging all the way from oversight of interviews and ratings done by site investigators to a completely independent interview of the patient by an outside expert. Unfortunately, as with most approaches to try to improve the efficiency and precision of clinical trials, there are only soft data to rely on to make a judgment regarding whether or not such approaches actually improve signal detection.

Some companies have tried blinding site investigators to threshold severity criteria as a way of reducing score inflation. However, doing so would not address the possibility that patients being recruited might not even have the correct diagnosis. Another growing problem in psychiatric drug trials has been that of fraudulent patients (those individuals who pretend to be patients to gain entry into trials for financial gain). It would seem unlikely this would be happening for schizophrenia trials, but it isn’ t clear if this issue has been looked at carefully enough for schizophrenia trials. Registries have emerged for trying to detect fraudulent patients.

One thing that might be done is to create a set of principles for good clinical practice in conducting schizophrenia trials and set up a process for certifying sites that want to compete in the clinical trials in schizophrenia effort. Such a process might help to root out problematic sites that are lacking in what is needed to engage in productive research in this area. We might look to other therapeutic areas to see what others are doing to address this problem of the proper conduct of clinical trials. Psychopharmacology cannot be the only therapeutic area that faces this problem.

It seems to me that something the FDA might do is to drill down into trial sites that contribute data to any particular study to determine, if possible, which ones are contributing most to the placebo response problem. Pharmaceutical companies could also do such explorations, and they undoubtedly do. But if the data aren’ t shared among companies for common learning, it is a lost opportunity. If it turns out that certain sites are consistently problematic in this regard, there should be exploration to try to determine the reasons for such deviation. Sites that are consistently lacking in adhering to principles of good trial conduct, or even possibly engaging in fraudulent activity, could be subjected to for cause inspections by the FDA and other regulatory agencies.

In the absence of a better understanding of this trend for deteriorating signal detection in schizophrenia studies done in North America, pharmaceutical companies will become increasingly reluctant to expend resources on using North American sites, and in fact many companies are leaving neuroscience research altogether. In essence, all North American sites seem to be getting a bad reputation. This reputation is clearly not warranted. The one thing that is clear is that this is a serious problem that is going to have a negative impact on schizophrenia research and ultimately on public health. We should all be concerned about that.

Published online: March 3, 2020.

Potential conflicts of interest: Dr Laughren is a part time employee of Massachusetts General Hospital Clinical Trials Network and Institute and has active consulting relationships with the following pharmaceutical companies: Acadia, Almatica, Arbor, Axsome, Balance Therapeutics, Beckley Canopy, Biohaven, Concert, Cox Biosciences, Evecxia, Exciva, Greenwich, Harmony, Janssen, Karuna, KemPharm, LB Pharmaceuticals, Lyndra, MAPS, Marinus, Minerva, Newron, Noven, Ovid, Roche, Tonix, and Vanda.

Funding/support: None.

REFERENCES

1.Gopalakrishnan M, Zhu H, Farchione TR, et al. The trend of increasing placebo response and decreasing treatment effect in schizophrenia trials continues: an update from the US Food and Drug Administration. J Clin Psychiatry. 2020;81(00):19r12960.

2.Khin NA, Chen YF, Yang Y, et al. Exploratory analyses of efficacy data from major depressive disorder trials submitted to the US Food and Drug Administration in support of new drug applications. J Clin Psychiatry. 2011;72(4):464-472. PubMed CrossRef

3.Kemp AS, Schooler NR, Kalali AH, et al. What is causing the reduced drug-placebo difference in recent schizophrenia clinical trials and what can be done about it? Schizophr Bull. 2010;36(3):504-509. PubMed CrossRef

4.Mallinckrodt CH, Zhang L, Prucka WR, et al. Signal detection and placebo response in schizophrenia: parallels with depression. Psychopharmacol Bull. 2010;43(1):53-72. PubMed

5.Agid O, Siu CO, Potkin SG, et al. Meta-regression analysis of placebo response in antipsychotic trials, 1970-2010. Am J Psychiatry. 2013;170(11):1335-1344. PubMed CrossRef

6.Rutherford BR, Pott E, Tandler JM, et al. Placebo response in antipsychotic clinical trials: a meta-analysis. JAMA Psychiatry. 2014;71(12):1409-1421. PubMed CrossRef

7.Leucht S, Leucht C, Huhn M, et al. Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, Bayesian meta-analysis, and meta-regression of efficacy predictors. Am J Psychiatry. 2017;174(10):927-942. PubMed CrossRef

8.Marder SR, Laughren T, Romano SJ. Why are innovative drugs failing in phase III? Am J Psychiatry. 2017;174(9):829-831. PubMed CrossRef