Social Media Images Can Predict Suicide Risk Using Interpretable Large Language-Vision Models

Yael Badian, MSc; Yaakov Ophir, PhD; Refael Tikochinski, MA; Nitay Calderon, MSc; Anat Brunstein Klomek, PhD; Eyal Fruchter, MD; Roi Reichart, PhD

J Clin Psychiatry 2024;85(1):23m14962

ABSTRACT

Background: Suicide, a leading cause of death and a major public health concern, became an even more pressing matter since the emergence of social media two decades ago and, more recently, following the hardships that characterized the COVID-19 crisis. Contemporary studies therefore aim to predict signs of suicide risk from social media using highly advanced artificial intelligence (AI) methods. Indeed, these new AI-based studies managed to break a longstanding prediction ceiling in suicidology; however, they still have principal limitations that prevent their implementation in real-life settings. These include “black box” methodologies, inadequate outcome measures, and scarce research on non-verbal inputs, such as images (despite their popularity today).

Objective: This study aims to address these limitations and present an interpretable prediction model of clinically valid suicide risk from images.

Methods: The data were extracted from a larger dataset from May through June 2018 that was used to predict suicide risk from textual postings. Specifically, the extracted data included a total of 177,220 images that were uploaded by 841 Facebook users who completed a gold-standard suicide scale. The images were represented with CLIP (Contrastive Language-Image Pre-training), a state-of-the-art deep-learning algorithm, which was utilized, unconventionally, to extract predefined interpretable features (eg, “photo of sad people”) that served as inputs to a simple logistic regression model.

Results: The results of this hybrid model that integrated theory-driven features with bottom-up methods indicated high prediction performance that surpassed common deep learning algorithms (area under the receiver operating characteristic curve [AUC] = 0.720, Cohen d = 0.82). Further analyses supported a theory-driven hypothesis that at-risk users would have images with increased negative emotions and decreased belongingness.

Conclusions: This study provides a first proof that publicly available images can be leveraged to predict validated suicide risk. It also provides simple and flexible strategies that could enhance the development of real-life monitoring tools for suicide.

J Clin Psychiatry 2024;85(1):23m14962

Author affiliations are listed at the end of this article.

Continue Reading...

Did you know members enjoy unlimited free PDF downloads as part of their subscription? Subscribe today for instant access to this article and our entire library in your preferred format. Alternatively, you can purchase the PDF of this article individually.

Subscribe Now

Already a member? Login

Purchase PDF for $40

Members enjoy free PDF downloads on all articles. Join today

References (59)

Turecki G, Brent DA, Gunnell D, et al. Suicide and suicide risk. Nat Rev Dis Primers. 2019;5(1):74. PubMed CrossRef
Hedegaard H, Curtin SC, Warner M. Suicide Rates in the United States Continue to Increase. Hyattsville, MD: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics; 2018.
Moutier C. Suicide prevention in the COVID-19 era: transforming threat into opportunity. JAMA Psychiatry. Published online October 16, 2020. PubMed
Klomek AB. Suicide prevention during the COVID-19 outbreak. Lancet Psychiatry. 2020;7(5):390. PubMed CrossRef
Banerjee D, Kosagisharaf JR, Sathyanarayana Rao TS. ‘The dual pandemic’ of suicide and COVID-19: a biopsychosocial narrative of risks and prevention. Psychiatry Res. 2021;295:113577. PubMed CrossRef
Gunnell D, Appleby L, Arensman E, et al; COVID-19 Suicide Prevention Research Collaboration. Suicide risk and prevention during the COVID-19 pandemic. Lancet Psychiatry. 2020;7(6):468–471. PubMed CrossRef
Samji H, Wu J, Ladak A, et al. Review: mental health impacts of the COVID‐19 pandemic on children and youth–a systematic review. Child Adolesc Ment Health. 2022;27(2):173–189. PubMed
Zaninotto P, Iob E, Demakakos P, et al. Immediate and longer-term changes in the mental health and well-being of older adults in england during the COVID-19 pandemic. JAMA Psychiatry. 2022;79(2):151–159. PubMed CrossRef
Vizheh M, Qorbani M, Arzaghi SM, et al. The mental health of healthcare workers in the COVID-19 pandemic: a systematic review. J Diabetes Metab Disord. 2020;19(2):1967–1978. PubMed CrossRef
Yao H, Chen J-H, Xu Y-F. Patients with mental health disorders in the COVID-19 epidemic. Lancet Psychiatry. 2020;7(4):e21. PubMed CrossRef
Li LZ, Wang S. Prevalence and predictors of general psychiatric disorders and loneliness during COVID-19 in the United Kingdom. Psychiatry Res. 2020;291:113267. PubMed CrossRef
Ren X, Huang W, Pan H, et al. Mental health during the Covid-19 outbreak in China: a meta-analysis. Psychiatr Q. 2020;91(4):1033–1045. PubMed CrossRef
Taquet M, Geddes JR, Husain M, et al. 6-month neurological and psychiatric outcomes in 236 379 survivors of COVID-19: a retrospective cohort study using electronic health records. Lancet Psychiatry. 2021;8(5):416–427. PubMed CrossRef
Ellis WE, Dumas TM, Forbes LM. Physically isolated but socially connected: psychological adjustment and stress among adolescents during the initial COVID-19 crisis. Canadian Journal of Behavioural Science/Revue canadienne des sciences du comportement. 2020;52(3):177–187. CrossRef
Murata S, Rezeppa T, Thoma B, et al. The psychiatric sequelae of the COVID-19 pandemic in adolescents, adults, and health care workers. Depress Anxiety. 2021;38(2):233–246. PubMed CrossRef
Zhang L, Zhang D, Fang J, et al. Assessment of mental health of Chinese primary school students before and after school closing and opening during the COVID-19 pandemic. JAMA Netw Open. 2020;3(9):e2021482. PubMed CrossRef
Sakamoto H, Ishikane M, Ghaznavi C, et al. Assessment of suicide in Japan during the COVID-19 pandemic vs previous years. JAMA Netw Open. 2021;4(2):e2037378. PubMed CrossRef
Dubé JP, Smith MM, Sherry SB, et al. Suicide behaviors during the COVID-19 pandemic: a meta-analysis of 54 studies. Psychiatry Res. 2021;301:113998. PubMed CrossRef
Bruffaerts R, Demyttenaere K, Hwang I, et al. Treatment of suicidal people around the world. Br J Psychiatry. 2011;199(1):64–70. PubMed CrossRef
Franklin JC, Ribeiro JD, Fox KR, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol Bull. 2017;143(2):187–232. PubMed CrossRef
Sejnowski TJ. The deep learning revolution. Mit Press; 2018.
Roy A, Nikolitch K, McGinn R, et al. A machine learning approach predicts future risk to suicidal ideation from social media data. NPJ Digit Med. 2020;3(1):78. PubMed CrossRef
Coppersmith G, Leary R, Crutchley P, et al. Natural language processing of social media as screening for suicide risk. Biomed Inform Insights. 2018;10:1178222618792860. PubMed CrossRef
Ophir Y, Tikochinski R, Asterhan CSC, et al. Deep neural networks detect suicide risk from textual Facebook posts. Sci Rep. 2020;10(1):16685. PubMed CrossRef
Zirikly A, Resnik P, Uzuner O, et al. CLPsych 2019 shared task: predicting the degree of suicide risk in Reddit posts. Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology. 2019 2019:24-33.
Zheng L, Wang O, Hao S, et al. Development of an early-warning system for high-risk patients for suicide attempt using deep learning and electronic health records. Transl Psychiatry. 2020;10(1):72. PubMed CrossRef
Bernert RA, Hilberg AM, Melia R, et al. Artificial intelligence and suicide prevention: a systematic review of machine learning investigations. Int J Environ Res Public Health. 2020;17(16):5929. PubMed CrossRef
Burke TA, Ammerman BA, Jacobucci R. The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: a systematic review. J Affect Disord. 2019;245:869–884. PubMed CrossRef
Resnik P, Foreman A, Kuchuk M, et al. Naturally occurring language as a source of evidence in suicide prevention. Suicide Life Threat Behav. 2021;51(1):88–96. PubMed
Ji S, Pan S, Li X, et al. Suicidal ideation detection: a review of machine learning methods and applications. IEEE Trans Comput Soc Syst. 2021;8(1)214–226.
Ophir Y, Tikochinski R, Brunstein Klomek A, et al. The Hitchhiker’s Guide to computational linguistics in suicide prevention. Clin Psychol Sci. 2022;10(2):212–235. CrossRef
Ophir Y, Rosenberg H, Lipshits-Braziler Y, et al. “Digital Adolescence”: The Effects of Smartphones and Social Networking Technologies on Adolescents’ Well-Being. Online Peer Engagement in Adolescence. Routledge; 2020:122–139.
Stravynski A, Boyer R. Loneliness in relation to suicide ideation and parasuicide: a population-wide study. Suicide Life Threat Behav. 2001;31(1):32–40. PubMed CrossRef
Bennardi M, Caballero FF, Miret M, et al. Longitudinal relationships between positive affect, loneliness, and suicide ideation: age-specific factors in a general population. Suicide Life Threat Behav. 2019;49(1):90–103. PubMed CrossRef
Rogers ML, Joiner TE. Rumination, suicidal ideation, and suicide attempts: a meta-analytic review. Rev Gen Psychol. 2017;21(2):132–142. CrossRef
Ribeiro JD, Joiner TE. The interpersonal-psychological theory of suicidal behavior: current status and future directions. J Clin Psychol. 2009;65(12):1291–1299. PubMed CrossRef
Joiner T. Why People Die by Suicide. Harvard University Press; 2009.
Bretherton I. The Origins of Attachment Theory: John Bowlby and Mary Ainsworth. Attachment Theory. Routledge; 2013:45–84.
Markowitz JC, Weissman MM. Interpersonal psychotherapy: principles and applications. World Psychiatry. 2004;3(3):136–139. PubMed
Ewing ESK, Diamond G, Levy S. Attachment-based family therapy for depressed and suicidal adolescents: theory, clinical model and empirical support. Attach Hum Dev. 2015;17(2):136–156. PubMed CrossRef
Diamond G, Diamond GM, Levy S. Attachment-based family therapy: theory, clinical model, outcomes, and process research. J Affect Disord. 2021;294:286–295. PubMed CrossRef
Chancellor S, De Choudhury M. Methods in predictive techniques for mental health status on social media: a critical review. NPJ Digit Med. 2020;3(1):43. PubMed CrossRef
Ophir Y, Asterhan CSC, Schwarz BB. The digital footprints of adolescent depression, social rejection and victimization of bullying on Facebook. Comput Human Behav. 2019;91:62–71. CrossRef
Ophir Y. SOS on SNS: adolescent distress on social network sites. Comput Human Behav. 2017;68:51–55. CrossRef
Brown TB, Mann B, Ryder N, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165. 2020.
Lee E, Lee J-A, Moon JH, et al. Pictures speak louder than words: motivations for using instagram. Cyberpsychol Behav Soc Netw. 2015;18(9):552–556. PubMed CrossRef
Herring S, Dainas A. “Nice Picture Comment!” Graphicons in Facebook Comment Threads. 2017.
Tang Y, Hew KF. Emoticon, emoji, and sticker use in computer-mediated communication: a review of theories and research findings. Int J Commun. 2019;13:27.
Bai Q, Dan Q, Mu Z, et al. A systematic review of emoji: current research and future perspectives. systematic review. Front Psychol. 2019;10:2221. PubMed CrossRef
Alishahi A, Chrupała G, Linzen T. Analyzing and interpreting neural networks for NLP: a report on the first BlackboxNLP workshop. Nat Lang Eng. 2019;25(4):543–557. CrossRef
Posner K, Brown GK, Stanley B, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266–1277. PubMed CrossRef
Ophir Y, Sisso I, Asterhan CSC, et al. The turker blues: hidden factors behind increased depression rates among Amazon’s Mechanical Turkers. Clin Psychol Sci. 2020;8(1):65–83. CrossRef
Radford A, Kim JW, Hallacy C, et al. Learning Transferable Visual Models From Natural Language Supervision. PMLR; 2021:8748–8763.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016:770–778.
Ramírez-Cifuentes D, Freire A, Baeza-Yates R, et al. Detection of suicidal ideation on social media: multimodal, relational, and behavioral analysis. J Med Internet Res. 2020;22(7):e17758. PubMed CrossRef
Ma Y, Cao Y. Dual Attention based Suicide Risk Detection on Social Media. IEEE; 2020:637–640.
Huang Y, Li W, Macheret F, et al. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc. 2020;27(4):621–633. PubMed CrossRef
Rodway C, Tham S-G, Turnbull P, et al. Suicide in children and young people: can it happen without warning? J Affect Disord. 2020;275:307–310. PubMed CrossRef
Ribeiro JD, Huang X, Fox KR, et al. Predicting imminent suicidal thoughts and nonfatal attempts: the role of complexity. Clin Psychol Sci. 2019;7(5):941–957. CrossRef