The Predictive Value of the Memory Impairment Screen in Patients With Subjective Memory Complaints: A Prospective Study

The Predictive Value of the Memory Impairment Screen in Patients With Subjective Memory Complaints: A Prospective Study

Pedro J. Modrego, MD, and José Gazulla, MD

Amnestic mild cognitive impairment is a common condition in the elderly, with an approximate prevalence of 6%.1 These patients complain of memory loss and score lower on memory tests than is expected for age and educational level. Their general cognition is in the range of normality, and their daily living activities are preserved; however, about 12% of these patients convert to Alzheimer’s disease (AD) annually.2 It has also been observed that, after 3-year follow-up, around 50% of these patients have developed AD.3 A broad array of tools have been used at baseline to predict conversion to AD: neuropsychological tests (short tests such as the Mini-Mental State Examination [MMSE], 3-word delayed recall, and category naming tests and longer tests such as the Wechsler Memory Scale and verbal learning tests), magnetic resonance imaging (MRI) and related techniques (volumetry, diffusion/perfusion, spectroscopy), single-photon emission computed tomography, positron emission tomography with glucose metabolism and amyloid plaque radiotracers, amyloid and tau protein determination in the cerebrospinal fluid (CSF), apolipoprotein E (APOE) genotype, and others.3 Despite the encouraging results from use of these biomarkers, there is no clear consensus on the best biomarker of early AD, but the amyloid markers seem to be the earliest indicators, since they can be detected even in the preclinical stages of AD.4 Although radiologic biomarkers are expensive and not widely available, they have drawn much more attention than the simple tests, which may be underused.

In the era of modern and sophisticated biomarkers, one has the impression that the value of simple neuropsychological tests is underestimated or neglected. Some studies disclosed excellent predictive values for neuropsychological tests, with sensitivities and specificities around 90%.5,6 However, in a cohort of 195 patients with questionable dementia, some neuropsychological tests produced good predictions of conversion to dementia, but these predictions were not clinically useful, as predictions were at most at 64% sensitivity and 76% specificity for verbal recognition and verbal fluency.7 In a Canadian cohort of 263 subjects who were initially nondemented, the short delayed recall test predicted dementia at 73% sensitivity and 70% specificity after 10 years of follow-up.8

The Memory Impairment Screen (MIS)9 was developed to screen for AD in elderly subjects, but there has been no assessment of its value in predicting conversion to AD in elderly subjects with memory complaints but normal global cognitive function. Brief neuropsychological instruments are welcome in clinical practice since a complete neuropsychological examination is time-consuming, and outpatient clinics are frequently overloaded with patients.

According to what is already known on this issue, biomarkers are expensive and not widely available. The MMSE is frequently used but only evaluates memory with a 3-word delayed recall task. Given that memory loss is the most prominent symptom in early AD, we hypothesize that a simple and brief memory test (MIS) may be even more useful than the MMSE to predict the development of AD in subjects with subjective memory loss.

METHOD

Patients

In this prospective study, we recruited a consecutive cohort of patients with memory complaints who were referred by family physicians and examined in the outpatient clinics of a university hospital in Zaragoza, Spain. The patients gave their written consent, and the project was approved by the regional ethics committee. The study was conducted from December 2007 to April 2011. The criteria for study inclusion were subjective memory complaints corroborated by an informant (relative or caregiver), preservation of daily living activities, no behavioral symptoms, and normality in global cognitive function. At baseline, patients underwent neuropsychological examination with the Mini-Examen-Cognoscivo10 (MEC; Spanish adaptation of the MMSE), MIS,9 Clinical Dementia Rating (CDR) scale,11 Blessed Dementia Rating Scale (BDRS),12 and Geriatric Depression Scale.13

• Many tools for evaluating memory and other cognitive functions are time-consuming, so brief instruments are welcomed by general practitioners and general neurologists.
• The Memory Impairment Screen (MIS) is a brief instrument that can be easily administered, even to illiterate patients, as a first evaluation of recent memory in those with memory complaints who do not fulfill criteria for dementia.
• A score of 0 or 1 on the MIS suggests eventual conversion to dementia of Alzheimer’s type. However, a normal score does not exclude such a conversion. Therefore, the MIS should be used in combination with other more sensitive tools.

Global cognitive function was measured with the MEC, with a maximum possible score of 35 points; for study inclusion, patients aged 65 years and older had to score > 23 points and patients aged < 65 years had to score > 26 points.10 For study inclusion, patients had to have a score of 0.5 (very mild dementia) on the CDR11 and had to score < 4 points on the BDRS. The BDRS was used to exclude patients with behavioral symptoms and deteriorated daily living activities, which could suggest mild or moderate dementia.12 Those patients who scored ≥ 11 points on the Geriatric Depression Scale13 were reevaluated after antidepressant treatment to confirm that the memory problems were not secondary to depressive symptoms. Patients who met National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) Work Group criteria14 for dementia or those with hydrocephalus, chronic psychiatric conditions, and large infarcts of the brain on MRI were excluded.

In the MIS, 4 written words of different categories are presented to the patient to be read aloud. Immediately after reading the words, the patient is warned that, after 5 minutes, he/she should recall those words. Two points are given for each word recalled spontaneously and 1 point for each word recalled with clues (eg, “It was an animal.”). The maximum possible score is 8 points. This test was originally designed to screen for dementia, with a cutoff score of 4 points, but, operationally, we established a cutoff point of ≤ 5 to diagnose amnestic mild cognitive impairment. The patients who screened positive for amnestic mild cognitive impairment also received the Rey Auditory Verbal Learning Test (RAVLT).15

As part of the protocol, and to exclude pathologies other than AD, MRI of the brain was performed on every patient in addition to standard blood tests including levels of vitamin B12, thyroid hormones, electrolytes, creatinine, transaminases, glucose, and lipids; white and red blood cell counts; erythrocyte sedimentation rate; and syphilis screening. APOE genotype was also obtained in most patients.

After baseline examinations, the patients were evaluated in the morning in the outpatient clinics by the same neurologist every 6 months or earlier if needed for a median period of 2 years (range, 1-4 years). Reexamination included screening with the MIS, MEC, CDR, and BDRS. The final endpoint was the conversion to dementia, mostly of probable AD type according to the NINCDS-ADRDA work group criteria.14

Statistical analysis was based on receiver operating characteristic (ROC) curves analysis to provide sensitivity, specificity, and predictive values of conversion to dementia. The t test was used to compare means of test scores in converters and nonconverters. Since age and educational level have effects on memory,16 we constructed a Cox proportional hazards model to adjust for these variables. Age was added to the model as a continuous variable. Educational level was computed in 4 levels as follows: low level, elementary school, high school, and university education. Calculations were done with the SPSS package software, version 10 (IBM Corporation, Armonk, New York).

RESULTS

The initial cohort comprised 110 consecutive patients with subjective memory complaints; 71 were women. We excluded a patient with an incidental brain stem tumor on MRI, 2 patients who had a pacemaker, and 2 patients who refused to undergo MRI. Finally, 105 patients were followed up, with a mean (SD) age of 74.3 (8.3) years.

The mean (SD) baseline scores for neuropsychological tests were as follows: 28.65 (2.9) for the MEC and 2.84 (1.99) for the MIS. In 83 patients diagnosed as having mild cognitive impairment, the MIS score was ≤ 5 points; in these patients, the mean (SD) baseline score for the RAVLT delayed recall was 2.83 (1.37). After a mean follow-up of 2 years (range, 1-4 years), 57 patients converted to probable AD and 48 did not. We did not see outcomes other than AD (eg, parkinsonism, hallucinations), at least at the point of the last revision of this article. The mean (SD) MIS baseline score for converters was 2.0 (1.8) and for nonconverters was 3.8 (1.7, t = 4.82, P ≤ .0001). There were 17 patients who scored 0 points, 7 patients who scored 1 point, and 25 patients who scored 2 points on the MIS. The difference in the mean baseline MEC score between converters and nonconverters (27.6 points versus 29.8, respectively) was also significant (t = 3, P = .0002). The mean decline on the MEC was 4.7 points for converters and 0.8 for nonconverters. For the MIS, changes from baseline were small and not significant in nonconverters. Table 1 presents the main variable values for converters and nonconverters.

The APOE genotype status was assessed in 70 patients, and 21 of them had 1 or 2 APOE4 alleles. The mean (SD) MIS score in patients with 1 or 2 APOE4 alleles was 1.28 (1.38) in comparison to 2.79 (2.63) in those without the allele (P = .0004). However, in terms of prediction to conversion to dementia, the sensitivity of the MIS was only 33%, with a specificity of 72%.

A score ≤ 30 points at baseline on the MEC predicted conversion to AD, with a 46.1% sensitivity and 86.8% specificity (area under the curve: 0.72; 95% CI, 0.62-0.8). For the MIS, a cutoff score ≤ 2 points predicted the development of dementia at 68% sensitivity and 73.5 specificity. The result of the ROC curve for the MIS was 0.76 (95% CI, 0.66-0.83). However, a score of 0 or 1 point on the MIS predicted conversion to dementia at 42.9% sensitivity and 98% specificity, with a positive predictive value of 96%. Figure 1 presents the ROC curve for the MIS. In the subset of 72 patients who underwent the RAVLT 20-minute delayed recall test, a score ≤ 3 points predicted conversion to dementia at 83% sensitivity and 69% specificity, with a positive predictive value of 73%.

In the survival curves, we observed that patients who scored 3-8 points on the MIS at baseline remained free of conversion to AD longer than those who scored 0, 1, or 2 points. At the end of follow-up, most patients with the lowest scores had converted to dementia (Figure 2). In the Cox proportional hazards model, we adjusted the effect of the MIS for age, sex, and educational level. The adjusted hazard ratio for the MIS was 0.71 (95% CI, 0.57-0.88; P = .001).

On MRI at baseline, we saw diffuse cortical atrophy of the brain in 54 patients, and, in 13 patients, the atrophy was limited to the hippocampus, but both were only evaluated in a visual way. The variable atrophy did not improve predictions when added to the multivariate analysis model.

The combination of the MEC and MIS increased sensitivity to 59% but with a specificity of 84%, so the final predictive value was not better than that of the MIS alone.

DISCUSSION

This is the first report of the use of the MIS as a tool for predicting the development of dementia. The MIS performed well in identifying those patients at high risk of developing dementia within at least a year. This test only takes 10 minutes to conduct, a brief time compared to longer tests often used for this purpose. Given that many outpatient clinics are overloaded, clinicians need brief instruments to evaluate patients. Of course, longer and more complete tests can give more information, but they are difficult to perform in the primary and secondary clinical settings.

In our cohort, a simple memory test (MIS) demonstrated an excellent specificity (98%) for predicting conversion to dementia, which is very helpful in the follow-up of patients with mild cognitive impairment to decide if further evaluations are needed to confirm the diagnosis of AD or to start the appropriate treatment for AD in patients with very low scores for whom all other causes of dementia have been excluded. According to our results, a score of 0 or 1 on the MIS supports the initiation of treatment because the specificity and positive predictive value are high.

Tierney et al8 developed a simple 6-item model to predict conversion to dementia that included 2 items from the MMSE, 2 from a patient rating scale, and 2 from an informant rating scale reaching 90% sensitivity and 94% specificity. The excellent values found in studies with fluorodeoxyglucose positron emission tomography were not confirmed in the cohort of 85 patients with mild cognitive impairment in the AD Neuroimaging Initiative (ADNI), wherein the positive predictive value was 41% and the negative predictive value was 79%.17 The remaining biomarkers (APOE genotype, hippocampal volume, and CSF biomarkers such as AB42-amyloid and tau proteins) disclosed good negative predictive values, with the RAVLT being of similar or better value than the mentioned biomarkers. However, none of the predictive variables reached positive predictive values higher than 50%.17 Of course, it may be argued that the follow-up (2 years) was shorter than that of some patients in our cohort (range, 1-4 years) and that predictive values found in other cohorts were better than those of the ADNI group. Notwithstanding, from a cost-effectiveness analysis viewpoint, and given the lack of effective treatments in AD, the use of simple memory tests could be even more reasonable than the use of expensive biomarkers for prediction purposes.

In this study, there are some limitations to bear in mind. First, all patients in the cohort were referred by family physicians because of memory complaints and were not recruited from the general population, so a bias toward increased risk of conversion to dementia should be noted. Second, it may be argued that some patients may already be demented at baseline despite not meeting the NINCDS-ADRDA criteria for probable AD. Although these criteria are sensitive only in relatively advanced stages of the disease, no better criteria have been validated so far, although this may change with the development of valid biomarkers. Third, there was heterogeneity in the follow-up, with some patients followed for only 1 year, which mitigates the predictive value in the patients with shorter follow-up for the final outcome. The follow-up should ideally be longer than a year.

Another concern is the appropriateness of one determined screening test over another. To date and among many tests, there is no demonstrated superiority of one test over the others.18 Furthermore, longer tests are not necessarily better than short tests.19 The MIS was originally designed to screen for AD and proved superior to the 3-word delayed recall used in the MMSE in terms of sensitivity and specificity.19 Given that memory loss alone does not necessarily mean that AD is present, at least in the short- and mid-term, we extended the use of the MIS to predict early conversion to AD. In its original version, the MIS was given a cutoff score of 4 points for screening for dementia, but the cutoff in this study was 5 points for screening of mild cognitive impairment. In a previous cohort of 106 patients with questionable dementia, the MIS plus a brief verbal fluency test yielded a sensitivity of prediction of 74% and a specificity of 81%.20

The usefulness of single domain tests for detecting dementia was evaluated in a meta-analysis of robust studies in several settings (community, primary care, and specialist). The optimal method in the community and specialist settings was the alteration in memory tests, with higher specificity values (85% and 82%, respectively) but lower sensitivity values.21

In conclusion, the MIS score at baseline (0 or 1) was a simple, single domain biomarker capable of providing useful information on the risk of conversion to AD or other dementias in patients with subjective memory complaints. On the basis of its brevity, the MIS may be a useful tool in the primary care and general neurology settings for predicting conversion to AD in patients with memory complaints. Although the MIS yielded a somewhat better curve of prediction than the MEC, both tests are far from sufficient to predict early conversion to dementia. Ideally, the MIS would be more useful when combined with a higher sensitivity test. We also have to bear in mind that the sample was not of a size to support a split sample analysis in which one part of the sample is used to determine the cutoff score that is then validated in the second part of the sample. This study, at this point, requires replication.

Author affiliations: Department of Neurology, Hospital Miguel Servet, Zaragoza, Spain.

Potential conflicts of interest: None reported.

Funding/support: None reported.

REFERENCES

1. López OL, Jagust WJ, DeKosky ST, et al. Prevalence and classification of mild cognitive impairment in the Cardiovascular Health Study Cognition Study, pt 1. Arch Neurol. 2003;60(10):1385-1389. PubMed doi:10.1001/archneur.60.10.1385

2. Petersen RC, Smith GE, Waring SC, et al. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999;56(3):303-308. PubMed doi:10.1001/archneur.56.3.303

3. Modrego PJ. Predictors of conversion to dementia of probable Alzheimer type in patients with mild cognitive impairment. Curr Alzheimer Res. 2006;3(2):161-170. PubMed doi:10.2174/156720506776383103

4. Petersen RC, Roberts RO, Knopman DS, et al. Mild cognitive impairment: ten years later. Arch Neurol. 2009;66(12):1447-1455. PubMed doi:10.1001/archneurol.2009.266

5. Kluger A, Ferris SH, Golomb J, et al. Neuropsychological prediction of decline to dementia in nondemented elderly. J Geriatr Psychiatry Neurol. 1999;12(4):168-179. PubMed doi:10.1177/089198879901200402

6. Grober E, Lipton RB, Hall C, et al. Memory impairment on free and cued selective reminding predicts dementia. Neurology. 2000;54(4):827-832. PubMed doi:10.1212/WNL.54.4.827

7. Tian J, Bucks RS, Haworth J, et al. Neuropsychological prediction of conversion to dementia from questionable dementia: statistically significant but not yet clinically useful. J Neurol Neurosurg Psychiatry. 2003;74(4):433-438. PubMed doi:10.1136/jnnp.74.4.433

8. Tierney MC, Yao C, Kiss A, et al. Neuropsychological tests accurately predict incident Alzheimer disease after 5 and 10 years. Neurology. 2005;64(11):1853-1859. PubMed doi:10.1212/01.WNL.0000163773.21794.0B

9. Buschke H, Kuslansky G, Katz M, et al. Screening for dementia with the memory impairment screen. Neurology. 1999;52(2):231-238. PubMed doi:10.1212/WNL.52.2.231

10. Lobo A, Ezquerra J, Gómez Burgada F, et al. Cognocitive mini-test (a simple practical test to detect intellectual changes in medical patients). Actas Luso Esp Neurol Psiquiatr Cienc Afines. 1979;7(3):189-202. PubMed

11. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43(11):2412-2414. PubMed doi:10.1212/WNL.43.11.2412-a

12. Blessed G, Tomlinson BE, Roth M. The association between quantitative measures of dementia and of senile change in the cerebral grey matter of elderly subjects. Br J Psychiatry. 1968;114(512):797-811. PubMed doi:10.1192/bjp.114.512.797

13. Spreen O, Strauss E. A Compendium of Neuropsychological Tests. New York, NY: Oxford University Press; 1998:326-340.

14. McKhann G, Drachman D, Folstein M, et al. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34(7):939-944. PubMed doi:10.1212/WNL.34.7.939

15. Rey A. L’ Examen Clinique En Psychologie. Paris, France: Presse Universitaires de France; 1964.

16. Ashford JW, Gere E, Bayley PJ. Measuring memory in large group settings using a continuous recognition test. J Alzheimers Dis. 2011;27(4):885-895. PubMed

17. Landau SM, Harvey D, Madison CM, et al; Alzheimer’s Disease Neuroimaging Initiative. Comparing predictors of conversion and decline in mild cognitive impairment. Neurology. 2010;75(3):230-238. PubMed doi:10.1212/WNL.0b013e3181e8e8b8

18. Ashford JW. Screening for memory disorders, dementia, and Alzheimer’s disease. Aging Health. 2008;4(4):399-432. doi:10.2217/1745509X.4.4.399

19. Kuslansky G, Buschke H, Katz M, et al. Screening for Alzheimer’s disease: the memory impairment screen versus the conventional three-word memory test. J Am Geriatr Soc. 2002;50(6):1086-1091. PubMed doi:10.1046/j.1532-5415.2002.50265.x

20. Chopard G, Vanholsbeeck G, Tio G, et al. Rapid screening of cognitive change in patients with questionable dementia using the Memory Impairment Screen and the Isaacs Set Test. J Am Geriatr Soc. 2009;57(4):703-708. PubMed doi:10.1111/j.1532-5415.2008.02170.x

21. Mitchell AJ, Malladi S. Screening and case-finding tools for the detection of dementia, pt 2: evidence-based meta-analysis of single-domain tests. Am J Geriatr Psychiatry. 2010;18(9):783-800. PubMed doi:10.1097/JGP.0b013e3181cdecd6