Publishing Statistically Significant Results With Questionable Clinical Importance: Focus on Antidepressant Use in Pregnancy

Publishing Statistically Significant Results With Questionable Clinical Importance: Focus on Antidepressant Use in Pregnancy

Adrienne Einarson, RN

Many more women than men are diagnosed with depression, most often between 25 and 44 years of age when women are of childbearing age,1 and approximately 10% to 15% will experience depression during pregnancy.2 Therefore, a substantial number of women could be taking an antidepressant when they become pregnant.

The use of antidepressants has increased in the past decade, as reported by a group using data from the National Birth Defects Prevention Study,3 an ongoing case-control study of risk factors for birth defects covering 10 US states. The frequency of reported antidepressant use at any time during pregnancy increased from 2.5% in 1998 to 8.1% in 2005 (P < .001) in 4 states. Among 6,582 mothers included in the study, 298 (4.5%) reported use of an antidepressant from 3 months before pregnancy through the end of pregnancy.3 A statistically significant decline, from 3.1% to 2.3% (P < .001), was observed in reported use of antidepressants between the first and second month after conception. This decline in use between the first and second trimester is not because pregnancy caused these women to become euthymic and no longer require antidepressants, but because of fear of teratogenicity associated with fetal exposure to antidepressants, perpetuated by both health care providers and the general public (personal communication with Motherisk callers, unpublished data, 2012).

As there are no randomized controlled trials conducted on pregnant women for obvious ethical reasons, they and their health care providers rely on observational studies published in the peer-reviewed literature to evaluate the safety of antidepressant medication use during pregnancy. Prior to 2005, research using observational designs conducted on the use of selective serotonin reuptake inhibitors (SSRIs) in pregnancy reported no association between SSRI use and congenital malformations. A meta-analysis was conducted of the available literature in 2005, with only 18 identified studies (1,774 outcomes) that met the inclusion criteria (relative risk = 1.01 [95% CI, 0.57-1.80]).4 At that time, antidepressants were considered relatively safe to take in pregnancy and no one appeared to be unduly concerned judging from the lack of warnings in either the scientific literature or lay press. In December 2005, the US Food and Drug Administration (FDA), on the basis of unpublished data from GlaxoSmithKline5 and preliminary data from 2 abstracts presented at conferences, published a warning on their Web site that paroxetine use in pregnant women may increase the risk for fetal heart defects by 2-fold, which has not been updated despite the numerous studies that have been published in the ensuing 7 years.6 However, an update on SSRIs and persistent pulmonary hypertension in newborns (PPHN) stated: "There have been conflicting findings from new studies evaluating this potential risk, making it unclear whether use of SSRIs during pregnancy can cause PPHN. FDA has reviewed new study results and has concluded that it is premature to reach any conclusion about a possible link between SSRI use in pregnancy and PPHN. FDA will update the SSRI drug labels to reflect the new data and the conflicting results."7

It is unfortunate that the FDA did not reexamine the paroxetine and cardiovascular defect studies since this association has not been proven and even "experts" in the field disagree as to whether the association is real. Two commentaries were published along with a meta-analysis presenting opposing opinions.8,9 Scialli8 concluded that the scientific evidence does not support the conclusion that paroxetine causes cardiovascular defects, while Bérard9 maintained that evidence-based literature shows consistent epidemiologic evidence that paroxetine use during pregnancy increases the risk of cardiac malformations in newborns. From these statements, one is prompted to question how it could be that 2 experts in the same field have offered such opposing conclusions based on their evaluation of the same data. In addition, cardiovascular malformations occur in 1/100 live births in the general population, so some women gave birth to an infant with a cardiovascular malformation that would have occurred whether or not the mother took paroxetine in her pregnancy. Subsequently, lawyers encouraged these women to sue the manufacturer by advertising on numerous Web sites.10 In October 2009, a jury awarded a family $2.5 million in the first Paxil lawsuit filed against the drug manufacturer GlaxoSmithKline, alleging that the drug was responsible for their son’s cardiovascular malformations due to exposure during pregnancy. By July 2010, the company reportedly settled about 800 Paxil birth defect lawsuits for approximately $1 billion.11

With the use of large administrative databases such as prescription databases, which were for the most part not designed for research since it is unknown if the woman actually took the drug, the number of studies has increased exponentially and currently totals more than 30,000 pregnancy outcomes following exposure to antidepressants during pregnancy.12 There would seem to be enough evidence-based information accumulated by now, but apparently this is not so, and studies are continuing to be conducted and sent to peer-reviewed journals for publication on the topic of antidepressant use during pregnancy. The probable reason is that, despite this sizeable number of studies and by far the most information on any drug taken in pregnancy, there remains the perception that these results are conflicting, when in reality they are not. When individual studies are published, much is made of very small increased odds ratios (ORs), usually less than 2 (which most epidemiologists consider relatively unimportant). The ORs are frequently explained in a way that they appear much more significant than they really are, and it is rare to see a statement regarding the absolute risk, especially in abstracts,13 when, in realistic terms, the abstract is often the only part of the article that most clinicians read.

These studies are frequently picked up by the lay media and much is made of these marginally significant results, especially in headlines. In addition, it is uncommon to see studies that found no increased risk reported in the media, an inconsistency that creates a substantial bias in favor of studies associated with adverse effects. Small but statistically significant risks are important at the population level but may be less so when considering an individual, such as a woman who is pregnant and taking an antidepressant. However, many health care providers and their pregnant patients do not understand this concept and use these results to influence their treatment choices.14 Thus, some women may be influenced to abruptly discontinue their medication, which may have serious consequences to both the mother and her unborn child, or terminate a wanted pregnancy.15

THE PEER-REVIEW PROCESS

The aim of peer-reviewed research is to publish results of studies that have been conducted using the most rigorous methodology in order to add to the evidence-based information to assist in the treatment of patients. It should be noted that the review process for publication of scientific papers started not long after Johannes Gutenberg invented the printing press in 1440, when a universal method for the generation and assessment of new science was announced by Francis Bacon in the early 1600s. However, it was not until academic societies were founded in the 1700s that a more formal approach was initiated. In 1752, the Royal Society of London took over the editorial responsibility for the production of the Philosophical Transactions, at which time it adopted a review procedure that had been used previously by the Royal Society of Edinburgh as early as 1731. Manuscripts sent to the Society for publication were now subject to inspection by a hand-picked group of members who were considered to be knowledgeable in the subject matter and whose recommendation to the editor was influential in the possible publication of the manuscript. Many scholars consider this the beginning of the peer-review process, which is basically still in practice today. This process continued almost to the mid-19th century, when due to the increasing specialization of medicine and the diversity of scientific studies sent to journals, it was necessary for journal editors to seek assistance outside the group of knowledgeable reviewers who could be found in their individual academic societies.16

Use of outside experts occurred at different times at different journals. For example, The Journal of the American Medical Association did not use outside reviewers until after 1940, which was facilitated by another machine, the Xerox, commercially available in 1959 and used to make multiple copies of papers to be sent out for peer review.12 Prior to the advent of the Internet, older individuals may remember when one had to send 5 copies of their manuscript by mail to the journal for consideration. In those days, the average time from sending the manuscript to a journal to eventual print publication if accepted was at minimum a year. As authors are allowed to send a manuscript to only one journal at a time, and, if there are several rejections, by the time the article is finally accepted and in print, it could be several years after the study was completed and the information could be out of date.

With the advent of the computer and Internet technology, the process has accelerated at an amazing rate, to the extent that today, at some of the larger journals, a manuscript can be reviewed, revised, accepted, and published online ahead of print within 6 to 8 weeks. The number of medical journals has also increased to more than 20,000, which means there is a requirement for a huge number of reviewers with scientific expertise who are able to critically evaluate manuscripts and pass on their comments to assist the editor in determining whether or not the journal should publish the manuscript.

As peer review is usually an unpaid task that can be very time consuming, it is prudent to ask where all these "experts" are coming from. How and from where do journals recruit reviewers and what are their qualifications? In researching for this commentary, I could find no documentation of how reviewers are recruited and what qualifies them as experts. As an individual who is frequently asked to be a reviewer, I have never been asked by any journal to state how I am qualified to be an expert. Conversely, as a frequently published author, many times I have been amazed at how totally opposite the opinions of 2 reviewers can be, as it appears that sometimes they have not evaluated the same manuscript. In scientific journals, the decision to publish studies with marginally significant results and questionable clinical significance is the domain of the editors and their editorial boards. These individuals rely heavily on the opinions of their reviewers, who are chosen for their "expertise" in the field, so as to make a decision whether to accept a particular manuscript for publication.

STUDIES REPORTING ON SAFETY OF ANTIDEPRESSANT USE IN PREGNANCY

Perinatal mental health research is a subspecialty, and studying the use of antidepressants in pregnancy is an even smaller subspecialty. However, information disseminated regarding results of studies conducted on the safety of antidepressants in pregnancy can have a huge impact on a vulnerable population. In addition, pregnancy stories are interesting reading for the general public, and, as everyone knows, "medications should not be taken during pregnancy," it makes interesting reading when some women do and a study is published associating harm with the drug. Unfortunately, stories about psychotropic drugs are especially interesting, as there continues to be stigma surrounding mental illness, especially when pregnancy is supposed to be the happiest time in a woman’s life. The truth is some women do require pharmacologic treatment for depression in this period. However, many discontinue their medication following pregnancy diagnosis for reasons that include negative information they have heard from their health care providers, who have informed them of studies that have been published without a thorough understanding of the data and results.13

Many of the studies published recently regarding antidepressant use in pregnancy that report an association with adverse effects, albeit with small increased ORs, have been conducted using large administrative databases and involve extremely complex statistics, which often only an epidemiologic expert is able to understand. As many reviewers are clinicians, it behooves editors to recruit not only clinical experts in the field, but also someone with statistical skills and knowledge. This recruitment may at times involve sending the manuscript to a statistical expert, which some journals do, but as far as I know, statistical review is not a common practice in all fields. I am considered an expert in the use of psychotropic drugs during pregnancy (probably because I have published many research papers on this topic in the peer-reviewed literature) and consequently am sent many manuscripts to review. However, I am not a statistician and, at times, do ask the editor to send a paper with extremely complex methodology and statistical analysis to someone who is.

In conclusion, with the use of highly advanced computer technology, the process of conducting epidemiologic studies has become so complex that editors of scientific journals have to rely on their reviewers more than ever. Judicious use of both clinical and statistical experts will ensure that the primary focus is on not only the statistical significance, particularly if marginal, but also the clinical importance, if any, of the study results. This will allow empowered decision making on the part of women and their health care providers when deciding whether or not to take an antidepressant during pregnancy.

Drug names: paroxetine (Paxil, Pexeva, and others).

Author affiliation: The Motherisk Program, The Hospital for Sick Children, Toronto, Canada.

Potential conflicts of interest: The Motherisk Program received an unrestricted educational grant from Eli Lilly Canada for studying the safety of duloxetine in pregnancy.

Funding/support: None reported.

REFERENCES

1. Grigoriadis S, Robinson GE. Gender issues in depression. Ann Clin Psychiatry. 2007;19(4):247-255. PubMed doi:10.1080/10401230701653294

2. Marcus SM. Depression during pregnancy: rates, risks and consequences: Motherisk Update 2008. Can J Clin Pharmacol. 2009;16(1):e15-e22. PubMed

3. Alwan S, Reefhuis J, Rasmussen SA, et al; National Birth Defects Prevention Study. Patterns of antidepressant medication use among pregnant women in a United States population. J Clin Pharmacol. 2011;51(2):264-270. PubMed doi:10.1177/0091270010373928

4. Einarson TR, Einarson A. Newer antidepressants in pregnancy and rates of major malformations: a meta-analysis of prospective comparative studies. Pharmacoepidemiol Drug Saf. 2005;14(12):823-827. PubMed doi:10.1002/pds.1084

5. Clinical Study Register. Epidemiology study EPIP083: preliminary report on bupropion in pregnancy and the occurrence of cardiovascular and major congenital malformation; 2007. GlaxoSmithKline Web site. www.gsk-clinicalstudyregister.com/ Accessed October 1, 2012.

6. FDA News Release. FDA advising of risk of birth defects with Paxil. Agency requiring updated product labeling. http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/2005/ucm108527.htm. US Food and Drug Administration Web site. Accessed October 1, 2012.

7. Selective serotonin reuptake inhibitor (SSRI) antidepressants: drug safety communication-use during pregnancy and potential risk of persistent pulmonary hypertension of the newborn.

http://www.fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlertsforHumanMedicalProducts/ucm283696.htm. US Food and Drug Administration Web site. Accessed October 8, 2012.

8. Scialli AR. Paroxetine exposure during pregnancy cardiac malformations. Birth Defects Res A Clin Mol Teratol. 2010;88(3):175-177. PubMed

9. Bérard A. Paroxetine exposure during pregnancy and the risk of cardiac malformations: what is the evidence? Birth Defects Res A Clin Mol Teratol. 2010;88(3):171-174. PubMed

10. Paxil lawsuit. www.schmidtandclark.com/paxil. Accessed October 1, 2012.

11. Paxil birth defects.www.lawyersandsettlements.com/case/paxil-heart-defects-newborn.html. Accessed October 1, 2012.

12. Diav-Citrin O, Ornoy A. Selective serotonin reuptake inhibitors in human pregnancy: to treat or not to treat [published online ahead of print December 10, 2011]? Obstet Gynecol Int. PubMed

13. Einarson TR, Lee C, Smith R, et al. Quality and content of abstracts in papers reporting about drug exposures during pregnancy. Birth Defects Res A Clin Mol Teratol. 2006;76(8):621-628. PubMed doi:10.1002/bdra.20289

14. Mulder E, Davis A, Gawley L, et al. Negative impact of non-evidence-based information received by women taking antidepressants during pregnancy from health care providers and others. J Obstet Gynaecol Can. 2012;34(1):66-71. PubMed

15. Einarson A, Selby P, Koren G. Abrupt discontinuation of psychotropic drugs during pregnancy: fear of teratogenic risk and impact of counselling. J Psychiatry Neurosci. 2001;26(1):44-48. PubMed

16. Spier R. The history of the peer-review process. Trends Biotechnol. 2002;20(8):357-358. PubMed doi:10.1016/S0167-7799(02)01985-6

Editor’s Note: We encourage authors to submit papers for consideration as a part of our Focus on Women’s Mental Health section. Please contact Marlene P. Freeman, MD, at [email protected].