In observational studies of patients with respiratory disease, the use of statins was found to be associated with improved respiratory outcomes.1 Three large randomized controlled trials (RCTs) were conducted to determine whether statins were indeed beneficial in such patients:
- Papazian et al2 studied adjunctive simvastatin in 300 patients with ventilator-associated pneumonia.
- The National Heart, Lung, and Blood Institute ARDS Clinical Trials Network3 studied adjunctive rosuvastatin in 745 patients with sepsis-associated acute respiratory distress syndrome.
- Criner et al4 studied simvastatin for the prevention of exacerbations in moderate-to-severe chronic obstructive pulmonary disease in 885 patients with this diagnosis.
All 3 RCTs were prematurely halted for futility; statins were ineffective for the indications studied.
Introduction
Observational studies are common in medical research. These may be prospective, as in planned cohort studies, or retrospective, as in analyses of information from insurance, health care, or other databases. Observational studies help generate hypotheses about possible relationships between, for example, a drug and a favorable or adverse outcome. When significant relationships are identified, these are sometimes assumed to indicate causality; that is, the drug is suggested to cause the favorable or adverse outcome.
Observational studies are useful for the identification of uncommon effects, such as bleeding with selective serotonin reuptake inhibitor (SSRI) antidepressants.5 Observational studies may also be useful when RCTs are unavailable or cannot be performed, such as to examine the risk of major congenital malformations following the use of SSRIs to treat depression during pregnancy. Unfortunately, observational studies may sometimes suggest incorrect conclusions, as shown by the 3 statin RCTs referred to above.2-4
The next sections contain examples of parallels to these statin studies in psychiatric literature. The first example is similar to the statin trials discussed above; RCTs refuted the results of observational studies. The second example illustrates an efficacy outcome, identified in observational studies, that remains unproven in RCTs. The third example illustrates adverse effect outcomes, identified in observational studies, that influence clinical practice because RCT data are unavailable.
Parallels in Psychiatric Literature: Example 1
About 1 to 2 decades ago, a large body of observational research suggested that hormonal replacement therapy (HRT) might improve cardiovascular, cognitive, and other outcomes in postmenopausal women; there were strong biological mechanisms to support this possibility.6 HRT was even suggested for the possible reduction of dementia risk.7 Subsequently, the Women’s Health Initiative Trial found that, at least in women 65 years and older, and contrary to expectations, HRT actually increased the risk of mild cognitive impairment and dementia.8 There is some possibility that HRT may benefit women if introduced earlier rather than later, after the onset of menopause, but the matter remains to be resolved.9
Parallels in Psychiatric Literature: Example 2
A large number of observational studies suggest that SSRIs improve cardiovascular outcomes in patients with ischemic heart disease (IHD). There is strong biological plausibility for these observational results.10 However, individual SSRI RCTs have not had adequate sample size and adequate trial duration to definitively support or refute these results. One meta-analysis of RCTs11 found that SSRIs were associated with a lower risk of readmission for IHD events and with lower mortality risk, but bias may have been present in this analysis. Therefore, the situation is unresolved, and one cannot as yet recommend the use of SSRIs to treat depression with the additional goal of primary or secondary prevention of IHD events in patients at risk of such events.
Parallels in Psychiatric Literature: Example 3
Depression is common during pregnancy, and antidepressants may be advised when the depression is severe. In observational studies, antidepressant use has been shown to increase the risk of different adverse outcomes, such as spontaneous abortion,12 preterm birth,13 low birth weight,13 major congenital malformations,14 postpartum hemorrhage,15 poor neonatal adaptation syndrome,16 persistent pulmonary hypertension of the newborn,17 autism,18 and others. There are no RCTs available that examine the safety of antidepressant use in pregnant depressed women, and it is unlikely that such RCTs will be conducted in the foreseeable future. As a result, the situation is unresolved, and the use of antidepressant medication during pregnancy continues to be viewed with concern.
Confounding
The results of observational studies may be compromised by confounding. Confounding is said to exist when a third variable explains part or all of an observed relationship between 2 variables. Here are a few easy-to-understand examples:
- Infants acquire teeth as they age. They also acquire larger vocabularies as they age. However, it isn’ t the teeth that are responsible for the larger vocabulary, even though the teeth are located in the same place from which the words emerge. Rather, it is greater age that is responsible for both more teeth and larger vocabulary.
- Schizophrenia is more prevalent in poorer sections of society. It is indeed possible that the psychosocial stresses, poorer nutrition, greater exposure to sources of infection, and other correlates of poverty increase the risk of schizophrenia. However, it is also possible that because schizophrenia compromises social and cognitive skills, there is a diminished capacity to earn, leading to a drift into poverty.19 Thus, the diminished capacity to earn might at least in part explain the observed association between poverty and the prevalence of schizophrenia.
- Children aged < 2 years who were exposed to dim night lighting were found to be more likely to develop myopia in later life.20 This does not mean that leaving lights on at night stimulates eyeball growth, increasing the risk of axial myopia. Rather, parents who are nearsighted are more likely to leave lights on at night, and heredity may explain why their offspring are more likely to develop myopia.21
Adjusting for Confounding
Regression analyses may help adjust for confounding. For example, observational studies show that greater education is associated with a lower risk of dementia.22 It is possible that greater education stimulates neuroplasticity changes in the brain, thereby increasing the cognitive reserve and diminishing the pathoplastic effect of the neurodegeneration that inevitably accompanies aging. However, it is also possible that intelligence is a confounding variable, and that more intelligent persons are more likely to undertake higher studies as well as enjoy a lower risk of dementia because higher intelligence is associated with greater cognitive reserve. Regression analyses can adjust for the effect of intelligence if the subject’s IQ score is included as an additional independent variable in the analysis. That is, the effect of years of education on dementia risk can be estimated after adjusting for intelligence.
Adjustment for confounding variables is possible only if the confounding variables are known and measured. This is not always possible, as shown by the following examples. Drinking green tea is associated with a lower risk of cognitive decline.23 It is possible that green tea contains neuroprotective compounds that reduce the risk of cognitive decline. It is also possible that certain kinds of people are more likely to prefer green tea as part of their lifestyle behaviors, and these same people, by virtue of the same set of lifestyle behaviors, may be less likely to develop cognitive decline. Analysis of the observational data would therefore adjust for age, gender, family history of Alzheimer’s disease, history of head injury, smoking and drinking behavior, exercise levels, and a host of other variables that are known to influence the risk of cognitive decline with aging. After adjusting for all known confounds, the effect of green tea on the risk of cognitive decline would be more clearly quantifiable. However, what if there are explanatory variables that were not thought of and hence not measured? What if there are explanatory variables (eg, IQ score) that were known but unavailable in the study? What if there were explanatory variables (eg, quantity of alcohol consumed daily, number of cigarettes smoked daily) that could not be accurately measured, with only a diagnosis of alcohol or nicotine dependence available in the database from which the records were drawn?
What if explanatory variables were accurately measured, but at only 1 point in time? For example, smoking, drinking, exercise, and cognitive activities may have been recorded at baseline in a prospectively studied cohort; however, these behaviors may have changed substantially as time passed, resulting in an unmeasurable influence on cognitive risks. Thus, adjustment for confounding is not necessarily efficient in observational studies.
Matching
In case-control studies, subjects with an outcome of interest may be matched with controls who do not show this outcome; matching may be based on age, gender, residence, and other important variables. However, matching is generally performed on a few important variables because it could be hard if not impossible to find controls who match cases on all important variables.
Propensity score matching is a special situation in which, for example, patients who do and do not receive a drug are matched for the likelihood of receiving that drug. Such matching is based on several variables, including indices of past illness severity and indices of current illness severity. Propensity score matching has its advantages, but the procedure has several requirements and limitations24 and cannot be equated with the quality of an RCT.
Confounding in Psychopharmacologic Studies in Pregnancy
One of the most vexing questions in psychopharmacology addresses the safety of antidepressants during pregnancy. Depression is common during pregnancy,25 and hormonal changes as well as illness behavior during depression can compromise maternal as well as fetal health.26-28 Logically, more severe depression would therefore be expected to be associated with worse maternal and fetal outcomes. However, more severe depression is also more likely to be treated with antidepressants, creating a situation in which antidepressant use is associated with worse pregnancy outcomes. So, do antidepressants worsen outcomes, or is it the depression for which the antidepressants were prescribed that worsens the outcomes?
Most observational studies of antidepressants in pregnancy are based on records drawn from health care, insurance, or other databases. Analyses of these records can adjust for age, known medical illness, known use of drugs for medical illness, past obstetric history, and other confounding variables. However, these databases do not contain information on occurrence and severity of stressful life events, severity of depression, presence and magnitude of substance use behaviors, nutrition, folate supplementation, adherence to obstetric guidance, or unusual variables such as exposure to air pollution during pregnancy, all of which can influence pregnancy outcomes. So, even though observational studies may attempt to adjust for measured confounds, unmeasured and inadequately measured variables may be responsible for residual confounding, creating spurious relationships between antidepressant exposure and pregnancy outcomes.
In this context, besides hormonal (internal) environment and illness behavior, genetic influences may also act as confounding variables. That is, genetic factors may predispose to depression (and hence antidepressant use) as well as to worse pregnancy outcomes. It is impossible to adjust for such genetic influences because what these genetic influences are (if any) is presently unknown.
If antidepressants worsen pregnancy outcomes, they should be prescribed to depressed pregnant women only when unavoidable. However, if hormonal environment or illness behavior predispose to worse outcomes, then antidepressant use becomes desirable because pregnancy outcomes might actually improve through reduced severity of illness. If genetic influences are responsible for both depression and worse pregnancy outcomes, then antidepressant prescription during pregnancy would reduce maternal suffering without compromising pregnancy outcomes.
Conclusions
When evaluating the results of observational studies, clinicians should remember that whereas observational studies are useful for hypothesis generation, RCTs are the gold standard for testing associations between predictor variables and outcomes. This is because observational studies are limited by known and unknown confounds that may not have been adequately adjusted for in analyses. As a specific example, significant associations identified between drugs and favorable or unfavorable outcomes in observational studies do not necessarily mean that the drugs are responsible for those outcomes.
Parting Notes
In observational studies, causal explanations for identified associations may be supported in various ways. These include the presence of biological plausibility for the association, existence of a dose-dependent relationship between the suggested cause and the effect of interest, and replication of the finding across studies.29 However, none of these supports is foolproof.
Many of the caveats expressed here in connection with observational studies also apply to subgroup analyses in RCTs. The reader is referred to Sun et al30 for a discussion of the subject. RCTs are a gold standard only for analyses that are based on randomization.
Each month in his online column, Dr Andrade considers theoretical and practical ideas in clinical psychopharmacology with a view to update the knowledge and skills of medical practitioners who treat patients with psychiatric conditions.
Department of Clinical Psychopharmacology and Neurotoxicology, National Institute of Mental Health and Neurosciences, Bangalore, India ([email protected]).
Financial disclosure and more about Dr Andrade.
REFERENCES
1. Drazen JM, Gelijns AC. Statin strikeout. N Engl J Med. 2014;370(23):2240-2241. PubMed doi:10.1056/NEJMe1405032
2. Papazian L, Roch A, Charles PE, et al; STATIN-VAP Study Group. Effect of statin therapy on mortality in patients with ventilator-associated pneumonia: a randomized clinical trial. JAMA. 2013;310(16):1692-1700. PubMed doi:10.1001/jama.2013.280031
3. Truwit JD, Bernard GR, Steingrub J, et al; National Heart, Lung, and Blood Institute ARDS Clinical Trials Network. Rosuvastatin for sepsis-associated acute respiratory distress syndrome. N Engl J Med. 2014;370(23):2191-2200. PubMed doi:10.1056/NEJMoa1401520
4. Criner GJ, Connett JE, Aaron SD, et al; COPD Clinical Research Network; Canadian Institutes of Health Research. Simvastatin for the prevention of exacerbations in moderate-to-severe COPD. N Engl J Med. 2014;370(23):2201-2210. PubMed doi:10.1056/NEJMoa1403086
5. Andrade C, Sandarsh S, Chethan KB, et al. Serotonin reuptake inhibitor antidepressants and abnormal bleeding: a review for clinicians and a reconsideration of mechanisms. J Clin Psychiatry. 2010;71(12):1565-1575. PubMed doi:10.4088/JCP.09r05786blu
6. Andrade C, Gelenberg AJ. Themes in psychiatry: a pot-pourri. In: Andrade C, ed. Advances in Psychiatry, No.1. New Delhi: Oxford University Press; 2000:241-278.
7. McBee WL, Dailey ME, Dugan E, et al. Hormone replacement therapy and other potential treatments for dementias. Endocrinol Metab Clin North Am. 1997;26(2):329-345. PubMed doi:10.1016/S0889-8529(05)70250-4
8. Shumaker SA, Legault C, Kuller L, et al; Women’s Health Initiative Memory Study. Conjugated equine estrogens and incidence of probable dementia and mild cognitive impairment in postmenopausal women: Women’s Health Initiative Memory Study. JAMA. 2004;291(24):2947-2958. PubMed doi:10.1001/jama.291.24.2947
9. Maki PM. Critical window hypothesis of hormone therapy and cognition: a scientific update on clinical studies. Menopause. 2013;20(6):695-709. PubMed
10. Chittaranjan A, Chethan KB, Sandarsh S. Cardiovascular mechanisms of SSRI drugs and their benefits and risks in ischemic heart disease and heart. Int Clin Psychopharmacol. 2013;28(3):145-155. PubMed doi:10.1097/YIC.0b013e32835d735d
11. Pizzi C, Rutjes AWS, Costa GM, et al. Meta-analysis of selective serotonin reuptake inhibitors in patients with depression and coronary heart disease. Am J Cardiol. 2011;107(7):972-979. PubMed doi:10.1016/j.amjcard.2010.11.017
12. Hemels ME, Einarson A, Koren G, et al. Antidepressant use during pregnancy and the rates of spontaneous abortions: a meta-analysis. Ann Pharmacother. 2005;39(5):803-809. PubMed doi:10.1345/aph.1E547
13. Ross LE, Grigoriadis S, Mamisashvili L, et al. Selected pregnancy and delivery outcomes after exposure to antidepressant medication: a systematic review and meta-analysis. JAMA Psychiatry. 2013;70(4):436-443. PubMed doi:10.1001/jamapsychiatry.2013.684
14. Grigoriadis S, VonderPorten EH, Mamisashvili L, et al. Antidepressant exposure during pregnancy and congenital malformations: is there an association? a systematic review and meta-analysis of the best evidence. J Clin Psychiatry. 2013;74(4):e293-e308. PubMed doi:10.4088/JCP.12r07966
15. Palmsten K, Hernández-D×az S, Huybrechts KF, et al. Use of antidepressants near delivery and risk of postpartum hemorrhage: cohort study of low income women in the United States. BMJ. 2013;347:f4877. PubMed doi:10.1136/bmj.f4877
16. Grigoriadis S, VonderPorten EH, Mamisashvili L, et al. The effect of prenatal antidepressant exposure on neonatal adaptation: a systematic review and meta-analysis. J Clin Psychiatry. 2013;74(4):e309-e320. PubMed doi:10.4088/JCP.12r07967
17. Occhiogrosso M, Omran SS, Altemus M. Persistent pulmonary hypertension of the newborn and selective serotonin reuptake inhibitors: lessons from clinical and translational studies. Am J Psychiatry. 2012;169(2):134-140. PubMed doi:10.1176/appi.ajp.2011.11040553
18. Rai D, Lee BK, Dalman C, et al. Parental depression, maternal antidepressant use during pregnancy, and risk of autism spectrum disorders: population based case-control study. BMJ. 2013;346:f2059. PubMed doi:10.1136/bmj.f2059
19. Cooper B. Schizophrenia, social class and immigrant status: the epidemiological evidence. Epidemiol Psichiatr Soc. 2005;14(3):137-144. PubMed doi:10.1017/S1121189X00006382
20. Quinn GE, Shin CH, Maguire MG, et al. Myopia and ambient lighting at night. Nature. 1999;399(6732):113-114. PubMed doi:10.1038/20094
21. Andrade C. Confounding. Indian J Psychiatry. 2007;49(2):129-131. PubMed doi:10.4103/0019-5545.33263
22. Meng X, D’ Arcy C. Education and dementia in the context of the cognitive reserve hypothesis: a systematic review with meta-analyses and qualitative analyses. PLoS ONE. 2012;7(6):e38268. PubMed doi:10.1371/journal.pone.0038268
23. Noguchi-Shinohara M, Yuki S, Dohmoto C, et al. Consumption of green tea, but not black tea or coffee, is associated with reduced risk of cognitive decline. PLoS ONE. 2014;9(5):e96013. PubMed doi:10.1371/journal.pone.0096013
24. Freemantle N, Marston L, Walters K, et al. Making inferences on treatment effects from real world data: propensity scores, confounding by indication, and other perils for the unwary in observational research. BMJ. 2013;347:f6409. PubMed doi:10.1136/bmj.f6409
25. Lancaster CA, Gold KJ, Flynn HA, et al. Risk factors for depressive symptoms during pregnancy: a systematic review. Am J Obstet Gynecol. 2010;202(1):5-14. PubMed doi:10.1016/j.ajog.2009.09.007
26. Davalos DB, Yadon CA, Tregellas HC. Untreated prenatal maternal depression and the potential risks to offspring: a review. Arch Women Ment Health. 2012;15(1):1-14. PubMed doi:10.1007/s00737-011-0251-1
27. Grigoriadis S, VonderPorten EH, Mamisashvili L, et al. The impact of maternal depression during pregnancy on perinatal outcomes: a systematic review and meta-analysis. J Clin Psychiatry. 2013;74(4):e321-e341. PubMed doi:10.4088/JCP.12r07968
28. Szegda K, Markenson G, Bertone-Johnson ER, et al. Depression during pregnancy: a risk factor for adverse neonatal outcomes? a critical review of the literature. J Matern Fetal Neonatal Med. 2014;27(9):960-967. PubMed doi:10.3109/14767058.2013.845157
29. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002;359(9302):248-252. PubMed doi:10.1016/S0140-6736(02)07451-2
30. Sun X, Ioannidis JP, Agoritsas T, et al. How to use a subgroup analysis: users’ guide to the medical literature. JAMA. 2014;311(4):405-411. PubMed doi:10.1001/jama.2013.285063
