Objective: Develop a brief, patient-reported screening tool designed to identify individuals with probable binge-eating disorder (BED) for further evaluation or referral to specialists.
Methods: Items were developed on the basis of the DSM-5 diagnostic criteria, existing tools, and input from 3 clinical experts (January 2014). Items were then refined in cognitive debriefing interviews with participants self-reporting BED characteristics (March 2014) and piloted in a multisite, cross-sectional, prospective, noninterventional study consisting of a semistructured diagnostic interview (to diagnose BED) and administration of the pilot Binge-Eating Disorder Screener (BEDS), Binge Eating Scale (BES), and RAND 36-Item Short-Form Health Survey (RAND-36) (June 2014-July 2014). The sensitivity and specificity of classification algorithms (formed from the pilot BEDS item-level responses) in predicting BED diagnosis were evaluated. The final algorithm was selected to minimize false negatives and false positives, while utilizing the fewest number of BEDS items.
Results: Starting with the initial BEDS item pool (20 items), the 13-item pilot BEDS resulted from the cognitive debriefing interviews (n = 13). Of the 97 participants in the noninterventional study, 16 were diagnosed with BED (10/62 female, 16%; 6/35 male, 17%). Seven BEDS items (BEDS-7) yielded 100% sensitivity and 38.7% specificity. Participants correctly identified (true positives) had poorer BES scores and RAND-36 scores than participants identified as true negatives.
Conclusions: Implementation of the brief, patient-reported BEDS-7 in real-world clinical practice is expected to promote better understanding of BED characteristics and help physicians identify patients who may have BED.
ABSTRACT
Objective: Develop a brief, patient-reported screening tool designed to identify individuals with probable binge-eating disorder (BED) for further evaluation or referral to specialists.
Methods: Items were developed on the basis of the DSM-5 diagnostic criteria, existing tools, and input from 3 clinical experts (January 2014). Items were then refined in cognitive debriefing interviews with participants self-reporting BED characteristics (March 2014) and piloted in a multisite, cross-sectional, prospective, noninterventional study consisting of a semistructured diagnostic interview (to diagnose BED) and administration of the pilot Binge-Eating Disorder Screener (BEDS), Binge Eating Scale (BES), and RAND 36-Item Short-Form Health Survey (RAND-36) (June 2014-July 2014). The sensitivity and specificity of classification algorithms (formed from the pilot BEDS item-level responses) in predicting BED diagnosis were evaluated. The final algorithm was selected to minimize false negatives and false positives, while utilizing the fewest number of BEDS items.
Results: Starting with the initial BEDS item pool (20 items), the 13-item pilot BEDS resulted from the cognitive debriefing interviews (n = 13). Of the 97 participants in the noninterventional study, 16 were diagnosed with BED (10/62 female, 16%; 6/35 male, 17%). Seven BEDS items (BEDS-7) yielded 100% sensitivity and 38.7% specificity. Participants correctly identified (true positives) had poorer BES scores and RAND-36 scores than participants identified as true negatives.
Conclusions: Implementation of the brief, patient-reported BEDS-7 in real-world clinical practice is expected to promote better understanding of BED characteristics and help physicians identify patients who may have BED.
Prim Care Companion CNS Disord 2016;18(2):doi:10.4088/PCC.15m01896
© Copyright 2016 Physicians Postgraduate Press, Inc.
aShire Development, LLC, Lexington, Massachusetts
bRTI Health Solutions, Durham, North Carolina
*Corresponding author: Barry K. Herman, MD, MMM, Shire Development, LLC, 300 Shire Way, Lexington, MA 02421 ([email protected]).
Binge-eating disorder (BED) was formally included as a distinct eating disorder in the DSM-5.1 BED is characterized by recurrent episodes of binge eating accompanied by feeling a lack of control and marked distress over one’s eating behaviors. The binge episodes must occur on average at least once per week over a 3-month period, not occur exclusively during the course of bulimia nervosa or anorexia nervosa, and not be associated with recurrent inappropriate compensatory behaviors.1 The 12-month prevalence rates of BED in the United States for adult women and men are estimated at 1.6% and 0.8%, respectively.2 The estimated lifetime prevalence in the US population is approximately 2.62 and is expected to increase.3,4 However, recognition of BED within the general medical community is most likely limited due to a lack of awareness of and familiarity with this newly categorized eating disorder.
In a recent systematic literature review, ×gh and colleagues5 concluded that BED is a serious condition that impairs health-related quality of life (HRQL) and increases health care costs. ×gh et al5 presented several studies finding lower levels of HRQL for patients with BED in comparison to general population norms based on Medical Outcomes Study 36-item Short-Form Health Survey physical and mental component summary scores.6,7 BED has been linked with several comorbid health conditions, including diabetes, hypertension, stroke, and heart disease,2 and other psychiatric illnesses such as anxiety and depression.8
Effective treatments for BED have the potential to reduce the burden of BED on patients and the health care system. Because general practitioners have the most contact with patients overall and psychiatrists see patients at higher risk for BED (eg, patients with eating disorders and associated mental health issues), general medicine and psychiatric practices may be ideal settings for assessment of BED. A BED screening tool could improve general awareness and knowledge of BED and facilitate patient-physician communication in both general and specialty settings. However, to our knowledge, no existing tool reflects the DSM-5 criteria for BED and is brief enough for physicians to easily incorporate into their practices. The focus of this article was to describe the qualitative and quantitative research conducted to develop the 7-item Binge-Eating Disorder Screener (BEDS-7), a patient-reported screening tool designed to identify individuals with probable BED for further evaluation or referral.
METHODS
The BEDS-7 was developed in 3 phases: development of an initial item pool based on DSM-5 diagnostic criteria, existing tools, and input from clinical experts (January 2014); cognitive debriefing interviews to test and refine draft items (March 2014); and quantitative evaluation to finalize and develop a scoring algorithm for the screener (June 2014-July 2014).
This study was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki 2008 and reviewed and approved by RTI International’s Institutional Review Board (Durham, North Carolina).
Initial Item Development
Potential concepts for the screening tool were identified through a review of DSM-5 diagnostic criteria (Table 1) and targeted literature search of existing patient-reported tools designed to assess the signs and symptoms of BED. On the basis of identified concepts, a preliminary item pool was developed by 3 psychologists (D.B.D., S.E.F., and T.M.B.) and instrument development experts. Phone interviews were conducted with 3 BED clinical experts to obtain feedback on candidate items and ensure that the DSM-5 criteria were accurately represented.
- Binge-eating disorder (BED) is a new and distinct diagnosable eating disorder in the DSM-5; as such, physicians are less familiar with recognizing its signs and symptoms.
- The 7-item Binge-Eating Disorder Screener (BEDS-7), a brief screener for BED, can assist physicians in identifying patients who may have BED and making the necessary follow-up decisions related to patient referrals or additional assessment and potential diagnosis of BED.
- While the BEDS-7 could be incorporated into a general screening assessment for various health and psychiatric conditions, physicians may need additional education and insight into identifying specific patients at highest risk for BED or those who would benefit most from its identification and potential treatment.
Cognitive Debriefing Interviews
To pretest and refine the draft BEDS item pool, 2 iterative sets of face-to-face cognitive debriefing interviews were conducted with 13 participants self-reporting BED characteristics in 2 US locations. Interview participants, recruited and screened by qualitative research firms in each location, were aged ≥ 18 years with a normal or greater body mass index (BMI ≥ 19) based on self-reported height and weight. Each firm recruited from their established database of general community residents previously agreeing to be contacted for research opportunities.
Participants were asked to answer and provide feedback on the draft BEDS items using a “think-aloud” process to identify any problems with question phrasing or response options. The interviewers also asked participants how they would revise the BEDS items (if needed) to make them clearer and easier to answer. The initial pool of 20 items (tested in the first set of interviews) addressed 13 unique concepts. A reduced item pool of 15 items was tested in the remaining interviews; alternative items were tested for several concepts. Results from these interviews were used to create a 13-item draft version of the BEDS for quantitative evaluation and further refinement. The 13 draft items represented each DSM-5 diagnostic criterion for BED.
Quantitative Evaluation
Data were collected from 97 participants in a multisite, cross-sectional, prospective, noninterventional study to finalize the content and develop a scoring algorithm for the screener.
Sample
Similar to the pool of general community residents in the cognitive debriefing interviews, participants with and without self-reported BED characteristics were recruited and screened by qualitative research firms in 4 US locations. Participants were aged ≥ 18 years with a normal or greater BMI (≥ 19). Due to the low prevalence of BED, individuals more likely to meet diagnostic criteria for BED were overrecruited. Specifically, screening items consistent with the DSM-5 criteria for BED1 were developed and used to target a study sample with half of the participants self-reporting all BED characteristics.
Procedure
Individual data collection sessions consisted of a semistructured diagnostic interview administered by 1 of 2 PhD-level clinical psychologists (D.B.D. and T.M.B.) (to diagnose BED) and administration of the 13-item BEDS pilot version, the Binge Eating Scale (BES),9 and the RAND 36-Item Short-Form Health Survey (RAND-36).10 Interviewers were blinded to participants’ responses to the recruiting items and self-report instruments. The order of the diagnostic interview and instrument administration was counterbalanced.
Given the absence of any known clinical interviews updated to reflect the DSM-5 criteria (at the time of this study), the BED criteria from the Structured Clinical Interview for DSM-IV-TR Axis I Disorders-Research Version, Non-Patient Edition (SCID-I/NP)11 were updated. Study participants were diagnosed as having BED (yes) or not having BED (no) using the modified BED portion of the SCID-I/NP. As part of the informal training to conduct the diagnostic interview, both interviewers diagnosed an initial set of participants and discussed any differences related to the use and interpretation of interview feedback until consensus was obtained.
The 16-item BES measures the presence and severity of binge-eating behaviors that may be indicative of an eating disorder. Total scores range from 0 to 46 points. BES cut scores proposed by Timmerman12 were used to classify study participants into 3 severity subgroups: none or mild (≤ 17), moderate (18 to 26), and serious (≥ 27). The RAND-36, a 36-item generic HRQL measure, assesses 8 concepts and yields subscale scores for each, ranging from 0 to 100, with higher values denoting more favorable health.
Analysis
The observed item-level response distributions of the 13 BEDS items by BED diagnosis were reviewed to guide the selection of an optimal algorithm for identifying participants’ BEDS screening status (positive or negative).
A reference BEDS classification algorithm was defined by a specific response pattern to the 13 items on the basis of DSM-5 criteria. Numerous alternative classification algorithms were also formed by altering the responses required for the 13 items under the reference algorithm and eliminating nondiscriminating items (between those who did and did not meet the BED diagnostic criteria from the clinical interview). Each classification algorithm resulted in 4 diagnostic accuracy subgroups (true positive, false positive, true negative, false negative) on the basis of structured clinical interview (BED diagnosis: yes or no) and BEDS algorithm screening status.
The accuracy of the reference and alternative BEDS screening status subgroups in predicting BED diagnosis (ie, sensitivity and specificity) was tabulated to compare algorithm performance. The optimal algorithm was expected to produce a reasonable number of false positives while minimizing the number of false negatives so that most or all individuals with BED are detected for follow-up and few, if any, are missed.
RESULTS
Initial BEDS Item Pool
The preliminary BEDS item pool developed for clinical expert review included 18 items addressing 12 unique concepts relevant to the diagnosis of BED (alternative items were developed for 6 concepts).
While all 3 clinical experts deemed these 18 items to be appropriate candidates for a BED-specific screener, several improvements resulted, including minor modification of several items and the addition of new items addressing the concept of distress (due to bingeing). The experts also recommended the implementation of a stopping rule at the beginning of the screener if respondents indicated a lack of binge eating in the past 3 months. The resulting pool of 20 items addressed 13 unique behaviors pertaining to binge eating and the lack of recurrent compensatory behaviors (eg, vomiting) and included alternative items for 7 of these concepts.
Cognitive Debriefing Interviews
The cognitive debriefing participants (n = 13) had a mean age of 41.6 years (range, 21-59 years) and a mean BMI of 36.2 (range, 23.3-53.2), and 53.8% (n = 7) were female. Approximately two-thirds (69.2%, n = 9) were white, and 92.3% (n = 12) had completed at least some college. While interview participants reported many BED characteristics, none had received a diagnosis of BED from a health care professional.
Interview participants easily understood the BEDS instructions, questions, and response scales. As noted previously, alternative items were tested; the items deemed clearer or easier to answer by interview participants were retained for further testing.
On the basis of results of the 2 iterative rounds of cognitive debriefing interviews, and representing all DSM-5 BED criteria, a 13-item pilot version of the BEDS was developed (Table 2). Items 1 and 2 establish the presence of participants’ excessive overeating during the past 3 months. If a participant reports at least 1 episode of excessive overeating per week during that time period, he or she is directed to items 3-13. If a participant reports no episodes of excessive overeating (no to both items 1 and 2), the participant is directed to stop.
Quantitative Evaluation
Ninety-seven adults participated in the BEDS quantitative evaluation. Table 3 shows the participant characteristics by BED diagnosis status and overall. Of the 97 participants, the ratio of females to males was relatively similar—approximately 60%:40%—across BED diagnosis status. Nearly half (n = 47, 48.5%) endorsed all the recruitment screening items consistent with the DSM-5 criteria for BED (data not shown), and 16 (16.5%) of these participants were ultimately diagnosed with BED via the clinical interviews. The proportions of females and males diagnosed with BED were similar (10/62 female, 16%; 6/35 male, 17%). While the small sample size prohibits generalization, compared to those not diagnosed with BED, participants with BED tended to be slightly older (median age: 46 years), have higher BMIs (median: 34.7), be black or Hispanic, be employed part-time, and have participated in a weight-loss program.
The mean and median item scores tended to be similar in magnitude, irrespective of the order of the diagnostic interview and instrument administration. Table 2 shows the item-level response distributions of the 13-item pilot BEDS.
BEDS Classification Algorithms
The left side of Table 4 shows the item response pattern of the reference BEDS classification algorithm. The right side of Table 4 shows that only 3 of the 16 participants diagnosed with BED during the clinical interview responded in a manner wholly consistent with the reference algorithm, yielding a low sensitivity of 18.8% (true positive rate) and an unacceptably high false-negative rate of 81.2% (ie, 13 of 16). However, a specificity of 89.3% indicates that the majority of participants not diagnosed with BED on the basis of the clinical interview were also not identified as probable BED by the reference algorithm.
Because a primary goal was to develop a screener that would identify all (or nearly all) individuals with probable BED, instrument developers considered discarding items that reduced sensitivity, while retaining items that maximized specificity. Originally, item 1 (excessive overeating in last 3 months) and item 2 (excessive overeating at least once per week in last 3 months) were viewed as key items because their content is important in a clinical interview. However, the inclusion of item 2 in the BEDS-7 algorithm (for a total of 8 items) yielded a sensitivity of 87.5% and specificity of 56.0%. All 16 participants diagnosed with BED responded yes to item 1 and only 14 responded yes to item 2. While the specificity of the algorithm including item 2 was moderately better than the BEDS-7 algorithm (38.7%), sensitivity was reduced by 12.5%. Measurement error associated with item 2 was also a concern, because respondents must recall the total number of episodes during the past 3 months and divide by 12 to ensure an accurate answer. Ultimately, the developers decided to remove item 2 rather than sacrifice some of the screener’s sensitivity by retaining a question with a significant potential for inaccurate response.
Researchers explored over 15 alternative BEDS classification algorithms without including responses on item 2 (data not shown); item 1 and item 3 (distress) were included to maintain maximum sensitivity (100% using only items 1 and 3). Response distributions of the remaining 10 items were reviewed, and each item was added to gauge its impact on specificity. The response patterns for item 5 (eat faster than normal), item 6 (eat until uncomfortably full), and item 7 (continue eating) were similar with overlapping content. To create the briefest screener, only item 7 was retained. Additionally, while the items addressing compensatory behaviors (items 10, 11, 12, and 13) did not contribute substantially to the screener’s predictive value, to maintain face validity, item 10 (vomiting) was retained.
Ultimately, 7 items (final BEDS-7 items shown in Supplementary Appendix 1) were retained in the algorithm that maximized sensitivity and obtained the highest possible specificity. The response pattern of the BEDS-7 required for a diagnosis of BEDS is shown in the left-hand side of Table 4. The BEDS-7 algorithmic scheme yielded 100% sensitivity and 38.7% specificity when applied to the study sample. Specifically, with use of the BEDS-7, 100% of the participants who received a diagnosis of BED via the clinical interview screened positive (true positives). Among those without a diagnosis of BED (n = 75), 38.7% (n = 29) screened negative on the BEDS-7 (true negatives). Forty-six of the 62 participants who screened positive using the BEDS-7 (74.2%) were not diagnosed with BED via clinical interviews. Table 4 shows that the sensitivity (at 100%) and specificity (33.3%) of the 16-item BES subgroup12 ranges were similar to those observed using the BEDS-7.
Table 5 shows select characteristics for participants identified by the BEDS-7 algorithm as true positives, true negatives, and false positives (there were no false negatives due to the 100% sensitivity). A higher proportion of women (71%, 44 of 62) than men (51%, 18 of 35) were identified with probable BED. Of the participants identified with probable BED, those who were not diagnosed with BED tended to be younger (median age: 33.5 years) and have a lower BMI (median: 30.9) than those diagnosed with BED (median age: 46.0 years, median BMI: 34.7). Notably, participants who were classified as unlikely to have BED had the lowest median BMI (29.9). Participants who were correctly identified tended to have higher BES scores and lower mean RAND-36 subscale scores, both indicating poorer health status, than participants identified as true negatives.
DISCUSSION
The goal of this study was to develop a brief, valid, patient-reported screening tool for use in primary care and general psychiatry settings to identify individuals most likely to have BED and to facilitate further evaluation or referral to specialists. The 13-item BEDS pilot version, developed with a review of existing instruments and input from BED clinical experts, was included in a cross-sectional study of 97 participants, including a semistructured clinical interview to diagnose BED. Although the presence of all 13 concepts addressed in the BEDS pilot version is required to clinically diagnose BED (per DSM-5 criteria), analyses were conducted to investigate the feasibility of eliminating items for the briefest screening tool possible without sacrificing its face and content validity or predictive ability. Maximizing the sensitivity of the BEDS was considered critical to ensure that future respondents with BED receive further evaluation and care. Additionally, a moderate level of specificity was considered reasonable because the consequences of further evaluating a modest number of individuals without BED are outweighed by those of missing individuals needing treatment.
The sensitivity and specificity of numerous candidate classification algorithms were evaluated. The BEDS-7 maximized sensitivity (100%) while closely preserving the content of the DSM-5 criteria. In comparison to the 16-item BES, the BEDS-7 is shorter with comparable sensitivity and specificity. Contributing to the high rate of false positives from the BEDS-7 was a large proportion of individuals who did not meet diagnostic criteria for BED during the clinical interview but reported that they were regularly engaging in excessive overeating episodes, had a lack of control over this behavior, and were distressed about the behavior.
A potential weakness of this study is the small number of individuals diagnosed with BED (n = 16) from the clinical interviews. Obtaining a larger sample of individuals diagnosed with BED was the original intent, but differences between individuals’ initial self-report of food and eating behaviors and perceptions via the phone screening versus those obtained in the clinical interview proved meaningful. As such, consistent with the high rate of false positives from the BEDS-7, many individuals not diagnosed with BED self-reported BED behaviors and beliefs.
A strength of this study is the rigorous design, which allowed for the development of a brief screening tool based on existing instruments, clinician and patient input, and quantitative evaluation to inform item reduction and scoring in a sample of participants including a subset diagnosed with BED. Additionally, the BEDS-7 would appear to have broader utility in clinical settings beyond that of primary care and general psychiatry, for example, in the allied health practitioner clinic or office setting. Implementation of the BEDS-7 in real-world clinical practice is expected to promote better understanding of BED characteristics and help health care professionals identify patients who may have BED for appropriate follow-up.
Submitted: October 16, 2015; accepted February 1, 2016.
Published online: April 28, 2016.
Potential conflicts of interest: Dr Herman is an employee of Shire Development, LLC. Ms Deal is currently an employee of Pfizer, Inc but was employed at Shire Development, LLC during the conduct of this project. Drs DiBenedetti, Nelson, Fehnel, and Brown are employees of RTI Health Solutions.
Funding/support: Funding for this study and article was provided by Shire Development, LLC. RTI Health Solutions was contracted by Shire to design and implement this project.
Role of the sponsor: Authors from Shire Development, LLC, the sponsor of this study, made substantial contributions to the study design and interpretation of data, as well as the drafting and revision of the manuscript.
Previous presentation: Content from this original research was presented at the following annual meetings and conferences: American Psychiatric Association 168th Annual Meeting, Toronto, Canada, May 16-20, 2015 ▪ XXIst Annual Meeting of the Eating Disorders Research Society, Taormina, Sicily, September 17-19, 2015 ▪ Canadian Psychiatric Association 65th Annual Conference, Vancouver, Canada, October 1-3, 2015.
Acknowledgments: Special thanks are extended to Susan G. Kornstein, MD (Virginia Commonwealth University, Richmond, Virginia); Rebecca Puhl, PhD (University of Connecticut, Storrs, Connecticut); and Denise E. Wilfley, PhD (Washington University, St Louis, Missouri) for reviewing and providing feedback on the initial version of the Binge-Eating Disorder Screener. They are all paid consultants to Shire Development, LLC.
Additional information: Shire develops and manufactures treatments for psychiatric disorders, including binge-eating disorder.
Supplementary material: See accompanying pages.
REFERENCES
1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Fifth Edition. Washington, DC: American Psychiatric Association; 2013.
2. Kessler RC, Berglund PA, Chiu WT, et al. The prevalence and correlates of binge eating disorder in the World Health Organization World Mental Health Surveys. Biol Psychiatry. 2013;73(9):904-914. PubMed doi:10.1016/j.biopsych.2012.11.020
3. Hudson JI, Coit CE, Lalonde JK, et al. By how much will the proposed new DSM-5 criteria increase the prevalence of binge eating disorder? Int J Eat Disord. 2012;45(1):139-141. PubMed doi:10.1002/eat.20890
4. Trace SE, Thornton LM, Root TL, et al. Effects of reducing the frequency and duration criteria for binge eating on lifetime prevalence of bulimia nervosa and binge eating disorder: implications for DSM-5. Int J Eat Disord. 2012;45(4):531-536. PubMed doi:10.1002/eat.20955
5. ×gh T, Kovács G, Pawaskar M, et al. Epidemiology, health-related quality of life and economic burden of binge eating disorder: a systematic literature review. Eat Weight Disord. 2015;20(1):1-12. PubMed doi:10.1007/s40519-014-0173-9
6. Masheb RM, Grilo CM. Quality of life in patients with binge eating disorder. Eat Weight Disord. 2004;9(3):194-199. PubMed doi:10.1007/BF03325066
7. Padierna A, Quintana JM, Arostegui I, et al. The health-related quality of life in eating disorders. Qual Life Res. 2000;9(6):667-674. PubMed doi:10.1023/A:1008973106611
8. Sallet PC, de Alvarenga PG, Ferr×£o Y, et al. Eating disorders in patients with obsessive-compulsive disorder: prevalence and clinical correlates. Int J Eat Disord. 2010;43(4):315-325. PubMed
9. Gormally J, Black S, Daston S, et al. The assessment of binge eating severity among obese persons. Addict Behav. 1982;7(1):47-55. PubMed doi:10.1016/0306-4603(82)90024-7
10. Hays RD, Sherbourne CD, Mazel RM. The RAND 36-Item Health Survey 1.0. Health Econ. 1993;2(3):217-227. PubMed doi:10.1002/hec.4730020305
11. First MB, Spitzer RL, Gibbon M, et al. Structured Clinical Interview for DSM-IV-TR Axis I Disorders-Research Version, Non-Patient Edition (SCID-I/NP). New York, NY: Biometrics Research, New York State Psychiatric Institute; 2002.
12. Timmerman GM. Binge Eating Scale: further assessment of validity and reliability. J Appl Biobehav Res. 1999;4(1):1-12. doi:10.1111/j.1751-9861.1999.tb00051.x
