Psychoeducation for Patients With a Psychotic Disorder: Effects on Knowledge and Coping

Psychoeducation for Patients With a Psychotic Disorder: Effects on Knowledge and Coping

Ercolie R. Bossema, PhD; Cynthia A. J. de Haar, MSc; Willemijn Westerhuis, MSc; Bas P. F. Beenackers, MSc; Bernadette C. E. M. Blom, MSc; Melanie C. M. Appels, PhD; and Chris J. van Oeveren, MSc

Objective: Psychoeducation is an essential and promising element in the nonpharmacologic treatment of patients with a psychotic disorder. This study examined the effects of patient-directed psychoeducation on knowledge and coping.

Method: This study included 99 primary care patients with a psychotic disorder according to DSM-IV-TR criteria who completed a knowledge questionnaire before and a knowledge and coping questionnaire halfway through, immediately after, and 6 months after a 20-session group psychoeducation program. The first time the program was given was between April and October 2007, and the final time the program was given was between October 2009 and April 2010. Results were analyzed with multilevel analysis.

Results: Knowledge increased significantly from the beginning of the program to halfway through the program (P < .001), even after correction for baseline scores, but not any further thereafter. Coping improved from halfway through the program to the end of the program (P = .02), also after correction for baseline scores, but not thereafter. Only at 6 months after the program was knowledge related to coping (P = .01). There were no differences in knowledge and coping between male and female patients. Halfway through (P = .001) and at the end of the program (P = .02), the increase in knowledge was significantly lower for patients taking atypical antipsychotic medication than for patients taking typical antipsychotic medication.

Conclusions: In patients with a psychotic disorder, psychoeducation results in more knowledge immediately and several months after the program and contributes to better coping only immediately after the program. Patients with more knowledge several months after psychoeducation may also be patients who then cope better with the disorder.

Prim Care Companion CNS Disord 2011;13(4):doi:10.4088/PCC.10m01116

© Copyright 2011 Physicians Postgraduate Press, Inc.

Submitted: November 22, 2010; accepted March 2, 2011.

Published online: August 4, 2011 (doi:10.4088/PCC.10m01116).

Corresponding author: Ercolie R. Bossema, PhD, Department of Clinical and Health Psychology, Utrecht University, Heidelberglaan 1, PO box 80.140, 3508 TC Utrecht, The Netherlands ([email protected]).

Schizophrenia is a clinical disorder of disruptive, variable psychopathology that involves cognition, emotion, perception, and other aspects of behavior. The disorder persists throughout life and usually begins before the age of 25 years. One or more major areas of functioning, such as work, interpersonal relations, and self-care, are markedly below the level achieved prior to its onset. The annual incidence is 0.5 to 5.0 per 10,000 people. The prevalence is equal in men and women, but there are gender differences. Men have an earlier age at onset, a less favorable course of illness, and worse social functioning than women.1,2

Because schizophrenia has a large impact on almost every aspect of life,2 and people rarely completely recover, treatment is necessary.3 Patients often receive antipsychotic medication, but cognitive-behavioral therapy and psychosocial therapy can also be given.4 Psychoeducation is an essential and promising element in the nonpharmacologic treatment of patients with a psychotic disorder.5 It can be described as systematic, structured, didactic information on the illness and its treatment, integrating emotional aspects to enable patients to cope with the illness.6

A large number of positive effects of psychoeducation have been reported in patients with a psychotic disorder, although sometimes not consistently, including less symptoms, less rehospitalization, better treatment adherence, better quality of life, and better social functioning.7-14 It is assumed that these effects result from an increase in knowledge and better coping, but of the recent studies, only a few have examined effects on knowledge, and effects on coping have not yet been specifically addressed. Furthermore, psychoeducation has often been evaluated in combination with additional interventions, making it hard to specify its contribution to the outcome.10 Moreover, most studies focused on family psychoeducation rather than patient-directed psychoeducation, which does not correspond with clinical practice.10 Finally, since patients with a psychotic disorder often did not finish a treatment program,15 it is useful to examine the effect of psychoeducation with a multilevel approach, which does not assume equal numbers of observations across participants.16

The aim of this study was to examine the effects of patient-directed psychoeducation on knowledge and coping by using a multilevel approach. In addition, the influence of baseline knowledge and coping was investigated as well as whether knowledge was related to coping. Furthermore, differences in knowledge and coping between male and female patients and between patients who take either typical or atypical antipsychotic medication were explored.

METHOD

Participants

We included 99 primary care patients with a psychotic disorder according to DSM-IV-TR criteria.17 The diagnosis was made by a psychiatrist after a thorough examination by the psychiatrist and another doctor, psychologist, or psychiatric nurse, all specialized in psychotic disorders. Diagnoses included schizophrenia (n = 60; paranoid subtype, n = 48; disorganized subtype, n = 4; catatonic subtype, n = 3; undifferentiated subtype, n = 4; and residual subtype, n = 1), schizoaffective disorder (n = 17), schizophreniform disorder (n = 1), delusional disorder (n = 2), psychotic disorder not otherwise specified (n = 18), and depression with severe psychotic features (n = 1).

The sample included 56 men and 43 women, with a mean age of 39.8 years (SD = 9.2; range, 23-60). Education level (primary school or lower vocational secondary education) was low in 5 patients, middle (intermediate general secondary education or intermediate vocational education) in 57 patients, and high (higher general secondary education, higher vocational education, or university education) in 37 patients. In addition to typical (n = 32) or atypical (n = 54) antipsychotic medication, patients were treated with biperiden (n = 18) or benzodiazepines (n = 23) at some time during the psychoeducation program. The degree of symptomatology of each patient was not formally assessed. The scientific research committee of the Altrecht Institute for Mental Health Care indicated that this study did not need their approval. Written informed consent was obtained from all participants.

Procedure

The psychiatrists asked their patients to participate in a psychoeducation program developed by Peperstraten and colleagues.18 Criteria for participation were checked by the psychiatrist and another doctor, psychologist, or psychiatric nurse and included being diagnosed with a psychotic disorder, not being psychotic at the time of the program, and being able to concentrate for at least 1 hour during the sessions. The second and third criterion were not formally assessed but were based on the clinical views and experiences of the professionals.

The psychoeducation program was given 19 times. The first time the program was given was between April and October 2007, and the final time the program was given was between October 2009 and April 2010. The number of participants at the start of each program was 6 to 12. A program consisted of 20 group sessions. Sessions 1 through 5 focused on providing knowledge on psychotic disorders and their symptoms and sessions 6 through 10 on providing knowledge on the treatment of and coping with psychotic disorders. Sessions 11 through 20 were about exchanging experiences and integrating the knowledge obtained during the first 10 sessions with the patient’s own experiences to finally accept the disorder and its consequences. One-hour sessions were given weekly by 2 trainers who were social psychiatric nurses, psychologists, psychiatrists, or case managers. When the trainers thought that a participant was psychotic at the start of a session or not able to concentrate, the session was stopped for this patient.

Knowledge was measured 4 times: at the beginning of the first session (baseline), at the beginning of session 10 (halfway through the program), at the beginning of session 20 (end of the program), and about 6 months after the program (follow-up). Coping measurements (which were added later on during the study) were taken 3 times: halfway through the program, at the end of the program, and at follow-up. Patients were only included if they had been present for at least 5 of the first 10 sessions and at least 5 of the second 10 sessions, except for the baseline measurement.

Measures

The knowledge questionnaire consisted of 32 statements about psychotic disorders, its causes, and its symptoms, such as “With having a psychiatric illness, such as schizophrenia, it is unnecessary to adjust expectations about the future” and “Delusions and/or hallucinations can get worse by using alcohol and/or drugs.” These statements could be answered with “true,” “false,” or “do not know.” The total number of correct responses determined the knowledge score, which could range from 0 to 32. Cronbach’s α was 0.83 when based on the 93 patients who completed the questionnaire at the first knowledge measurement at the beginning of the program.

The coping questionnaire consisted of 8 statements, such as “I will punish myself less for my negative symptoms” and “I am now putting up with having a psychotic disorder.” These statements could be answered at the beginning of the study with false or true, but later on using a 5-point Likert scale from 1 to 5. In order to compare the results at the beginning and later on, the Likert scale scores were recoded so that the scores 1 “totally disagree” and 2 “disagree” became 0 (false) and the scores 4 “agree” and 5 “totally agree” became 1 (true). A score of 3 “neither agree nor disagree” was recoded as 0.5. The sum of scores determined the coping score, which could range from 0 to 8, with higher scores indicating more coping. Cronbach α was 0.77 when based on the 36 patients who completed the questionnaire with the false and true answer options at the first coping measurement that occurred halfway through the program.

Statistics

The Statistical Package for the Social Sciences, version 16.0 (SPSS Inc, Chicago, Illinois), along with Pearson correlation coefficients, was used to examine whether knowledge was related to coping at each measurement time. Multilevel regression modeling, as implemented in the program MLwiN, version 2.18 (Centre for Multilevel Modelling, University of Bristol, Bristol, United Kingdom), was used to examine whether knowledge and coping changed over time and whether these changes differed between patients with low baseline knowledge (ie, lower than versus equal to or higher than the median score of 21.00) or coping (ie, lower than versus equal to or higher than the median score of 4.00), between male and female patients, and between people taking typical or atypical antipsychotic medication after correction for age, education level, use of biperiden, and use of benzodiazepines. An important advantage of multilevel regression modeling is that it does not assume equal numbers of observations across participants, which means that participants with missing data can remain in the analyses, thereby increasing the precision of the estimates and the power of the statistical tests.16

In the multilevel regression modeling, there were 2 levels of nesting: the repeated measurements of knowledge (206-236) and coping (99-108) at the within-subject level were nested within the patients (n = 93 and n = 51, respectively) at the between-subject level. Preceding multilevel regression modeling, age was centered on its grand mean (ie, the overall mean was subtracted from the values of the variable). Measurement time, gender, type of antipsychotic medication, education, use of biperiden, and use of benzodiazepines were dummy coded. The significance of the effects was determined with the Wald test: z = estimate ×· SD of estimate, where z refers to the standard normal distribution.16

The adequacy of distinguishing the between- and within-subject level in the analysis was examined in a first step by testing an empty model without any explanatory variables included (Model 1). In a second step, differences in knowledge and coping across the various measurement times were examined (Model 2). The effects of the covariates and baseline knowledge/coping, gender, and type of antipsychotic medication on knowledge and coping were tested in a third step (Model 3). Finally, interaction effects between measurement time (random) and baseline knowledge/coping, gender, and type of antipsychotic medication on knowledge and coping were tested (Model 4). Separate multilevel analyses were performed for knowledge and coping and for the 3 subgroup distinctions. P values < .05 were considered statistically significant.

RESULTS

Table 1 shows the number of patients and their mean scores on the knowledge and coping questionnaire at each measurement. Many patients (n = 41, 41.0%) did not complete the program; ie, they were not present for at least 5 of the first 10 sessions and at least 5 of the second 10 sessions of the program.

Pearson’s correlation coefficients showed a moderate positive relationship between knowledge and coping at 6 months after the program (r = 0.68, P = .01, n = 12) but not halfway through (r=0.09, P = .54, n = 50) and at the end of the program (r = 0.11, P = .52, n = 39).

With regard to knowledge, the amount of variance at the between- and within-subject level was significant (both P < .001, Table 2, Model 1), justifying the specification of a 2-level model with patients at a between-subject level and the repeated knowledge measurements at a within-subject level. Knowledge increased significantly from the first measurement (baseline) to the second measurement (halfway through the program) (P < .001) but not any further thereafter (Table 2, Model 2).

Of the covariates, only education level was significantly associated with overall knowledge, with patients with middle or high education having more overall knowledge than patients with low education (P = .01 and .005, respectively; Table 2; Model 3-I). Furthermore, the degree of knowledge at the beginning of the program (at baseline) was significantly related to overall knowledge, with patients with high baseline knowledge having higher overall knowledge (P < .001, Table 2, Model 3-II). However, after correction for education level, patients with high baseline knowledge increased significantly less in knowledge halfway through the program, at the end of the program, and at the 6-month follow-up (all P < .001) than patients with low baseline knowledge (Table 2, Model 4-I).

The type of antipsychotic medication was not significantly associated with overall knowledge, but after correction for education level, patients taking atypical medication increased significantly less in knowledge halfway through (P = .001) and at the end of the program (P = .02) than did patients taking typical antipsychotic medication (Table 2, Model 4-II). Gender was not significantly related to overall knowledge, and the interaction effects of gender with measurement times were also not significant.

With regard to coping, the amount of variance at the between- and within-subject level was also significant (both P < .001, Table 3, Model 1), justifying the specification of a 2-level model with patients at a between-subject level and the repeated coping measurements at a within-subject level. As was the case with knowledge, coping improved significantly from the first measurement (halfway through the program) to the second measurement (at the end of the program) (P = .02) but not thereafter (Table 3, Model 2). None of the covariates and neither gender nor type of antipsychotic medication was significantly associated with overall coping, and the interaction effects between either gender or type of antipsychotic medication with measurement time were also not significant. However, the degree of coping halfway through the program was significantly related to overall coping, with patients with high baseline coping having better overall coping (P < .001, Table 2, Model 3). In addition, patients with high coping halfway through the program improved significantly more in coping at the end of the program (P < .001) than patients with low coping halfway through the program (Table 2, Model 4).

DISCUSSION

This study examined whether knowledge of and coping with a psychotic disorder improve during and after a psychoeducation program in patients with a psychotic disorder.

The results showed that knowledge increased significantly from the beginning of the program to halfway through the program and that this was retained until at least 6 months after the end of the program, even after correction for baseline knowledge. This finding corresponds with previous research evaluating the effect of psychoeducation on knowledge in schizophrenia and other psychotic disorders.7-10,13 The first part of our psychoeducation program focused on providing knowledge.18 It is therefore not surprising that knowledge increased after this portion of the program. Cognitive deficits, especially problems with learning and retrieval, are common in people with schizophrenia,19,20 but apparently this did not influence gaining the knowledge, possibly because it concerned information that was highly relevant for our patients. Furthermore, knowledge increased more in patients with middle or high education than in patients with low education. These patients may have better cognitive functioning and attention21 and therefore profited more from the psychoeducation program.

Coping improved from halfway through the program to the end of the program and also after correction for baseline coping, but not thereafter. The second half of the program was about exchanging experiences and integrating the knowledge obtained during the first 10 sessions with the patient’s own experiences to finally accept the disorder and its consequences.18 Apparently, this improved coping. As far as we know, effects of psychoeducation on coping have not yet been specifically addressed, although the positive effects of psychoeducation that have been reported, such as less rehospitalization and better treatment adherence,7-12 suggest better coping with the disorder.

Only at 6 months after the program had ended was knowledge related to coping. More knowledge may enable patients to cope better with their disorder. However, in the current study no relation was found between knowledge and coping halfway through the program. This may be due to the structure of the program, with a focus on providing knowledge in the first half of the program and on integrating this knowledge with one’s own experiences in the second half. Patients who at 6 months after the program still have knowledge may also be the patients who are better able to cope with the disorder. This finding cannot be explained by the patients included at the follow-up assessment, since a relation between knowledge and coping during the program was not shown for these patients (data not shown).

The degree of knowledge at the beginning of the program was associated with overall knowledge, and the degree of coping halfway through the program was associated with overall coping. The increase in knowledge from the beginning of the program to halfway through the program and the improvement in coping from halfway through the program to the end of the program even remained significant after correction for baseline scores. However, patients with high baseline scores increased less in knowledge and improved less in coping at the subsequent measurement times than patients with low baseline scores. This finding probably reflects that there is less room for improvement in patients who already have higher scores.

There were no differences in knowledge and coping between male and female patients. Although men and women may differ in cognitive functioning,22,23 this did not affect knowledge and coping after psychoeducation. Perhaps gender does influence certain cognitive functions but not memory for disorder-related knowledge and (consequently) not coping with the disorder, as was also observed in a recent study.24 For clinical practice, this finding may indicate that the psychoeducation program is equally suitable in male and female patients.

There was a difference in knowledge retention halfway through and at the end of the program between patients taking either typical or atypical antipsychotic medication. Patients who took atypical antipsychotic medication at these times had significantly less knowledge of the psychotic disorder than those who took typical antipsychotic medication. This is in contrast with a previous study,25 which found a positive effect of atypical antipsychotic medication on executive functioning, visual memory, and delayed recall. Another study26 found improved cognition after treatment with antipsychotic medication but no difference between patients taking either typical or atypical antipsychotic medication. Effects on knowledge and coping were not found for biperiden and benzodiazepines in the current study. Earlier studies did find an effect of these medication types.27,28

A strength of this study is that the effect of psychoeducation was examined without additional interventions and was directed at the patient, which corresponds with clinical practice.10 Moreover, effects were examined over a longer time period using a multilevel approach, which is especially useful when dropout either during or after the program is high.15 However, especially at the final measurement, only a small number of people completed the measures of knowledge and coping. The comparison of male and female patients and of patients taking either typical or atypical antipsychotic medication should therefore be interpreted with caution. Furthermore, the knowledge and coping questionnaires used in this study were not existing and standardized questionnaires. Nevertheless, the internal consistency of both questionnaires was satisfying according to the criteria reported by Nunnally and Bernstein,29 and the questions seem to have face validity. In addition, the coping questionnaire was added later on during the study, meaning that there was no measurement of coping at the beginning of the program. Thus, it could not be examined whether coping improved halfway through the program compared to before the program. Finally, rather than an effect of education, the observed improvements in knowledge and coping may be due to other elements of the psychoeducation program rather than the content itself, such as peer contact or sharing experiences. Future research may focus on the reliability and validity of the knowledge and coping questionnaires used in this study as well as the elements of psychoeducation contributing to increased knowledge and better coping after psychoeducation. It may also be useful to assess and control for the degree of symptomatology of each patient at each measurement time.

Finally, in patients with a psychotic disorder, psychoeducation results in more knowledge immediately and several months after the program and contributes to better coping only immediately after the program. Patients with more knowledge several months after psychoeducation may also be patients who then cope better with the disorder.

Drug names: biperiden (Akineton).

Author affiliations: Department of Clinical and Health Psychology, Utrecht University (Drs Bossema and Appels; Mss de Haar, Blom, and Westerhuis; and Mr Beenackers); and Department of Social Psychiatric Treatment, Altrecht Institute for Mental Health Care (Mr van Oeveren), Utrecht, The Netherlands.

Potential conflicts of interest: None reported.

Funding/support: Support for the study was provided by the Department of Clinical and Health Psychology, Utrecht University, Utrecht, The Netherlands, and the Altrecht Institute for Mental Health Care, Department of Social Psychiatric Treatment, Utrecht, The Netherlands.

Previous presentation: Part of this work was presented at an informal meeting at the Altrecht Institute for Mental Health Care; July 1, 2010; Utrecht, The Netherlands.

Acknowledgments: The authors thank staff members and patients from the Altrecht Institute for Mental Health Care in Utrecht, Nieuwegein, Woerden, and Zeist, The Netherlands.

REFERENCES

1. Sadock BJ, Sadock VA. Kaplan and Sadock’s Synopsis of Psychiatry. Tenth edition. Philadelphia, PA: Lippincott Williams & Wilkins; 2007.

2. Thornicroft G, Tansella M, Becker T, et al; EPSILON Study Group. The personal impact of schizophrenia in Europe. Schizophr Res. 2004;69(2-3):125-132. PubMed doi:10.1016/S0920-9964(03)00191-9

3. Barlow DH, Durand VM. Abnormal Psychology. An Integrative Approach. London, UK: Thomson Wadsworth; 2005.

4. Van Duin D, Boerema I, Hofstra K, et al. Multidisciplinaire Richtlijn Schizofrenie. Utrecht, The Netherlands: Trimbos-instituut; 2005.

5. Brenner HD, Pfammatter M. Psychological therapy in schizophrenia: what is the evidence? Acta Psychiatr Scand suppl. 2000;102(s407):74-77. PubMed doi:10.1034/j.1600-0447.2000.00014.x

6. Rummel-Kluge C, Kissling W. Psychoeducation in schizophrenia: new developments and approaches in the field. Curr Opin Psychiatry. 2008;21(2):168-172. PubMed doi:10.1097/YCO.0b013e3282f4e574

7. Pitschel-Walz G, Leucht S, Bäuml J, et al. The effect of family interventions on relapse and rehospitalization in schizophrenia: a meta-analysis. Schizophr Bull. 2001;27(1):73-92. PubMed

8. Pekkala ET, Merinder LB. Psychoeducation for schizophrenia (review). Cochrane Database Syst Rev. 2002;2(2):CD002831. PubMed

9. Mueser KT, Corrigan PW, Hilton DW, et al. Illness management and recovery: a review of the research. Psychiatr Serv. 2002;53(10):1272-1284. PubMed doi:10.1176/appi.ps.53.10.1272

10. Lincoln TM, Wilhelm K, Nestoriuc Y. Effectiveness of psychoeducation for relapse, symptoms, knowledge, adherence and functioning in psychotic disorders: a meta-analysis. Schizophr Res. 2007;96(1-3):232-245. PubMed doi:10.1016/j.schres.2007.07.022

11. Xia J, Merinder LB, Belgamwar MR. Psychoeducation for schizophrenia. Schizophr Bull. 2011;37(1):21-22. PubMed doi:10.1093/schbul/sbq138

12. Bäuml J, Pitschel-Walz G, Volz A, et al. Psychoeducation in schizophrenia: 7-year follow-up concerning rehospitalization and days in hospital in the Munich Psychosis Information Project Study. J Clin Psychiatry. 2007;68(6):854-861. PubMed doi:10.4088/JCP.v68n0605

13. Sibitz I, Amering M, Gössler R, et al. One-year outcome of low-intensity booster sessions versus care as usual in psychosis patients after a short-term psychoeducational intervention. Eur Psychiatry. 2007;22(4):203-210. PubMed doi:10.1016/j.eurpsy.2006.09.008

14. Bechdolf A, Knost B, Nelson B, et al. Randomized comparison of group cognitive behaviour therapy and group psychoeducation in acute patients with schizophrenia: effects on subjective quality of life. Aust N Z J Psychiatry. 2010;44(2):144-150. PubMed doi:10.3109/00048670903393571

15. Nosé M, Barbui C, Tansella M. How often do patients with psychosis fail to adhere to treatment programmes? a systematic review. Psychol Med. 2003;33(7):1149-1160. PubMed doi:10.1017/S0033291703008328

16. Hox J. Multilevel Analysis: Technique and Applications. London, UK: Lawrence Erlbaum Associates; 2002.

17. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association; 2000.

18. Peperstraten JJ, Aalders HCWM, van der Gaag M, et al. Psycho-Educatie Voor Mensen met Schizofrenie of Verwante Psychose. Den Haag, The Netherlands: Kennis Centrum Schizofrenie; 2003.

19. Mesholam-Gately RI, Giuliano AJ, Goff KP, et al. Neurocognition in first-episode schizophrenia: a meta-analytic review. Neuropsychology. 2009;23(3):315-336. PubMed doi:10.1037/a0014708

20. Heinrichs RW. The primacy of cognition in schizophrenia. Am Psychol. 2005;60(3):229-242. PubMed doi:10.1037/0003-066X.60.3.229

21. Mueller BW, Timmers R, Thienel R, et al. Effects of age, gender and education on cognitive performance in schizophrenia patients. Schizophr Res. 2008;102(1-3, suppl 2):143. doi:10.1016/S0920-9964(08)70435-3

22. Bora E, Yucel M, Pantelis C. Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: meta-analytic study. Br J Psychiatry. 2009;195(6):475-482. PubMed doi:10.1192/bjp.bp.108.055731

23. Seidman LJ, Goldstein JM, Goodman JM, et al. Sex differences in olfactory identification and Wisconsin Card Sorting performance in schizophrenia: relationship to attention and verbal ability. Biol Psychiatry. 1997;42(2):104-115. PubMed doi:10.1016/S0006-3223(96)00300-9

24. Reichhart T, Pitschel-Walz G, Kissling W, et al. Gender differences in patient and caregiver psychoeducation for schizophrenia. Eur Psychiatry. 2010;25(1):39-46. PubMed doi:10.1016/j.eurpsy.2009.08.001

25. Hori H, Noguchi H, Hashimoto R, et al. Antipsychotic medication and cognitive function in schizophrenia. Schizophr Res. 2006;86(1-3):138-146. PubMed doi:10.1016/j.schres.2006.05.004

26. van Veelen NMJ, Grootens KP, Peuskens J, et al. Short term neurocognitive effects of treatment with ziprasidone and olanzapine in recent onset schizophrenia. Schizophr Res. 2010;120(1-3):191-198. PubMed doi:10.1016/j.schres.2010.04.011

27. Wezenberg E, Verkes RJ, Sabbe BG, et al. Modulation of memory and visuospatial processes by biperiden and rivastigmine in elderly healthy subjects. Psychopharmacology (Berl). 2005;181(3):582-594. PubMed doi:10.1007/s00213-005-0083-7

28. Westra HA, Stewart SH, Teehan M, et al. Benzodiazepine use associated with decreased memory for psychoeducation material in cognitive behavioral therapy for panic disorder. Cognit Ther Res. 2004;28(2):193-208. doi:10.1023/B:COTR.0000021540.15135.45

29. Nunnally J, Bernstein I. Psychometric Theory. New York, NY: McGraw Hill; 1994.