Published on in Vol 4, No 1 (2021): Jan-Mar

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26340, first published .
A New Tool for Detecting COVID-19 Psychological Burden Among Postacute and Long-term Care Residents (Mood-5 Scale): Observational Study

A New Tool for Detecting COVID-19 Psychological Burden Among Postacute and Long-term Care Residents (Mood-5 Scale): Observational Study

A New Tool for Detecting COVID-19 Psychological Burden Among Postacute and Long-term Care Residents (Mood-5 Scale): Observational Study

Original Paper

Mansbach Health Tools, LLC, Simpsonville, MD, United States

Corresponding Author:

William E Mansbach, PhD

Mansbach Health Tools, LLC

PO Box 307

Simpsonville, MD, 21150

United States

Phone: 1 4438967409

Email: wmansbach@thebcat.com


Background: Older adults are at high risk for developing serious somatic and psychological symptoms associated with COVID-19. Currently available instruments may not be sensitive to the concerns about COVID-19 in postacute and long-term care and their applications in telehealth remain to be clarified.

Objective: We investigated the psychometric properties of the Mood-5 Scale (M5) as a rapid self-assessment of the COVID-19 psychological burden among postacute and long-term care residents.

Methods: Residents (N=131), aged 50 years and above, from 20 postacute and long-term care facilities in Maryland, USA, were evaluated in-person or via telehealth (43/131, 32.8%) across a 4-week period (May 11 to June 5, 2020) during the COVID-19 pandemic. The COVID-19 psychological burden experienced by the residents was rated by geriatric psychologists who independently reviewed their clinical documentation. Psychometric analyses were performed on the M5 in relation to psychological tests, COVID-19 psychological burden, and diagnostic data collected during the evaluation.

Results: The M5 demonstrated acceptable internal consistency (Cronbach α=.77). M5 scores were not confounded by demographic variables or telehealth administration (P>.08). Convergent validity for the M5 was established via positive associations with anxiety (r=0.56, P<.001) and depressive (r=0.49, P<.001) symptoms. An M5 cutoff score of 3 demonstrated strong sensitivity (0.92) and adequate specificity (0.75) for identifying COVID-19 psychological distress among postacute and long-term care residents (area under the curve of 0.89, positive predictive value=0.79, negative predictive value=0.91).

Conclusions: The M5 is a reliable and valid tool for self-assessment of mood that can help identify postacute and long-term care residents with significant psychological burden associated with COVID-19. It can be completed in less than 1 minute and is appropriate for use in both in-person and virtual visits.

JMIR Aging 2021;4(1):e26340

doi:10.2196/26340

Keywords



The base rates of depression and anxiety are high among postacute and long-term care (PA/LTC) residents. Approximately one-third of all residents in PA/LTC facilities experience significant depressive symptoms [1,2], whereas an estimated 5%-10% experience anxiety-related disorders [3,4]. These numbers are significantly higher among residents referred for neurocognitive evaluations. For instance, in a sample of PA/LTC residents referred for evaluation of mood and/or cognitive symptoms, 55% met the criteria for a major depressive episode, and 36.6% met the criteria for generalized anxiety disorder [5]. Although we found no studies investigating the psychological burden associated with COVID-19 in PA/LTC settings, there is evidence suggesting that the pandemic has contributed to an increase in mental health concerns. In a community sample, the American Psychiatric Association found that 36% of Americans reported that COVID-19 has had a significant impact on their mental health, and 48% reported feeling anxious about potentially contracting the infection [6]. The Centers for Disease Control and Prevention has issued a warning that people over the age of 65 years, those with serious underlying medical conditions [7,8], and those living in residential care settings are at the highest risk for developing severe illness from COVID-19; therefore, it is expected that these groups experience an increased psychological burden, placing them at a considerable risk of the development or exacerbation of psychiatric symptoms.

To our knowledge, evidence supporting a rapid mood screening tool that can be used to capture psychological burden associated with COVID-19 is currently lacking. Rapid screening is especially critical in the context of COVID-19. Under normal circumstances, health care providers are highly limited for the time that they can spend on assessments, particularly for co-occurring medical conditions [9,10]. Time is even more limited when the duration and extent of face-to-face encounters is capped to prevent the spread of infection, and competition for resources restricts the duration of virtual visits. This almost rules out the use of multiple measures or instruments that require more than 2 minutes of the providers’ time to administer. The measures of depression and anxiety that are most commonly used in PA/LTC settings, including the Patient Health Questionnaire (PHQ-9) [11] and the Generalized Anxiety Disorder 7-item (GAD-7) scale [12], assess depression or anxiety but not both. Moreover, formal anxiety screening is not required in PA/LTC settings. The Geriatric Depression Scale–Short Form [13] is a mood instrument developed specifically for older adults, but it has limited psychometric properties and double the number of items in the PHQ-9 [14].

In this study, we developed and validated the Mood-5 Scale (M5) to address barriers to its practical use in the context of COVID-19 by minimizing administration time, allowing for self-administration, and combining the assessment of depression and anxiety. The scale is an adapted version of the Brief Anxiety and Depression Scale (BADS), a screening tool that assesses both depressive and anxiety symptoms and is widely used by health care professionals in PA/LTC settings [5]. The M5 was designed so that it can be (1) self-administered by residents in PA/LTC settings for a variety of conditions, ranging from normal cognitive functioning to mild dementia; (2) completed in less than 1 minute; and (3) completed as part of an in-person or telehealth visit, which is particularly relevant in the context of the COVID-19 pandemic. We hypothesized that the M5 will be able to rapidly identify COVID-19–associated psychological burden, as well as clinical anxiety and depression.


Participants and Procedures

Residents (N=131) aged 50 years and above from 20 PA/LTC facilities in Maryland, USA, were evaluated by a behavioral health interdisciplinary team comprising 10 psychologists, 1 psychiatrist, and 10 nurse practitioners via in-person or telehealth visits. Data were collected across a 4-week period during the COVID-19 pandemic (ie, May 11 to June 6, 2020) to obtain a snapshot of the possible psychological burden during the pandemic, with the intention of sharing actionable information with the providers who care for PA/LTC residents. The relatively small sample size reflects the effort to fast-track the research process and maximize impact in the context of the current COVID-19 pandemic. Institutional approval was obtained from each PA/LTC facility, and all residents or their responsible parties completed a consent agreement. Furthermore, all residents were deidentified for the analysis. M5 items were derived from the standard evaluation procedures so that the residents experienced no additional burden through its administration. A battery of psychological tests, including the M5, BADS [5], and the Brief Cognitive Assessment Tool (BCAT) [15], was administered as part of the usual evaluation. The International Classification of Diseases 10th Revision (ICD-10) [16] and Clinical Dementia Rating Scale [17] were used to assign psychiatric diagnoses and dementia stages, respectively. Residents were excluded from the study analyses if they had incomplete M5 data, moderate-to-severe dementia, or were aged below 50 years. Demographic and clinical characteristics of the study sample are presented in Table 1.

Table 1. Select demographics and clinical characteristics (N=131).
VariableValue, n (%)
Age (years), mean (SD)76.12 (11.05)
Gender
Female69 (52.67)
Male62 (47.33)
Race
White110 (83.97)
Black14 (10.69)
Other4 (3.05)
Missing3 (2.29)
Marital status
Single23 (17.56)
Married14 (10.69)
Widowed52 (39.69)
Separated6 (4.58)
Divorced34 (25.95)
Missing2 (1.53)
Education (years completed)
≤1118 (13.74)
1255 (41.98)
13-1522 (16.79)
1617 (12.98)
≥1714 (10.69)
Missing5 (3.82)
Facility
Skilled nursing87 (66.41)
Assisted living44 (33.59)
Cognitive level
No dementia10 (7.63)
MCIa67 (51.15)
Mild dementia54 (41.12)
Telehealth delivery43 (31.82)
COVID-19 distress67 (51.15)

aMCI: mild cognitive impairment.

Measures

Development of the M5

The M5 was adapted from BADS, which was chosen because it measures depression and anxiety factors separately, is used widely in PA/LTC settings, and has strong psychometric properties. Two items each from the depression and anxiety factors of BADS were selected for inclusion in the M5. A fifth item was added to address somatic or cognitive features. A panel of experts comprising 3 geriatric psychologists, 1 psychiatrist, and 2 PA/LTC medical directors vetted the instrument before data collection.

For standardized administration, residents were instructed as follows: “Think about how you have been feeling during the past month as you answer the following five questions. Please answer: ‘no’=0, ‘somewhat’=1, or ‘yes’=2.” The M5 items were written as follows:

  1. Have you lost interest in activities that you had found pleasurable?
  2. Do you worry about things more than usual?
  3. For at least two consecutive days, have you felt depressed, hopeless, or down?
  4. Are you feeling nervous, anxious, or “wound up” much of the time?
  5. Are you experiencing fatigue, headaches, stomach upset, or memory problems?

Multimedia Appendix 1 presents the M5 items and standardized scoring instructions.

COVID-19 Psychological Burden

The outcome binary variable “COVID-19 psychological burden” was based on a geriatric psychologist’s independent review of the behavioral health providers’ clinical documentation. While completing the medical records of PA/LTC residents, health care professionals were required to directly ask the patient whether they were experiencing psychological symptoms associated with fear of contracting COVID-19 and/or social distancing precautions to reduce disease transmission. An affirmative score was assigned if the documentation supported that the resident was queried about the COVID-19 psychological burden and the resident made direct statements about experiencing increased anxiety or depressive symptoms associated with COVID-19 or if the health care professional observed increased anxiety or depression associated with COVID-19.

Validity Measures

The convergent and discriminant validity of the M5 were evaluated using the BADS and BCAT, respectively. The BADS is an 8-item mood questionnaire designed to identify anxiety and depression (score range 0-16) among older adults. The BCAT is a 21-item, multi-domain cognitive instrument (score range 0-21) that distinguishes among normal cognition, mild cognitive impairment, and dementia [15,18].

Statistical Analysis

Analyses were performed in R (version 3.6.1; R Core Team) [19] using RStudio (version 1.2.5019; RStudio Team) software [20]. Descriptive statistics were used to report demographics, clinical characteristics, and study measures. Pearson correlations, independent sample t tests, and analysis of variance were performed to investigate the relationship between these variables and the M5. Cronbach α was used to estimate internal consistency. Receiver operator characteristic curve analyses examined the ability of the M5 to identify COVID-19 psychological burden. Despite the compressed data collection period, the sample size was sufficient for preliminary reliability [21,22]. Residents with missing study measures were removed pairwise to maximize the use of available M5 data.


Preanalysis

Table 2 reports descriptive statistics for the M5 and validity measures. M5 scores were not associated with gender, race, marital status, education, or provider discipline (P>.05). Residents in skilled nursing settings (87/131, 66.4%) reported higher M5 scores than residents in assisted living settings (diff=1.73; 95% CI 0.29-3.18; P=.01). Younger age was associated with higher M5 scores (r=−0.19, P=.03). M5 scores did not differ as a function of telehealth (43/131, 32.8%) or in-person evaluations (diff=0.08; 95% CI −0.97 to 1.13; P=.88).

Table 2. Descriptive statistics of the various study measures used.
Study measuren (%)Mean (SD)MinimumMaximumSkewnessKurtosis
M5a131 (100)3.60 (2.86)0100.57−0.59
BCATb70 (53.4)34.61 (6.38)2246−0.05−1.02
BADSc AFd110 (83.9)2.70 (1.78)060.47−0.75
BADS DFe110 (83.9)3.31 (3.01)0100.68−0.59

aM5: Mood-5 Scale.

bBCAT: Brief Cognitive Assessment Tool.

cBADS: Brief Anxiety and Depression Scale.

dAF: Anxiety Factor of BADS.

eDF: Depression Factor of BADS.

Psychometric Analyses

The M5 demonstrated acceptable internal consistency (Cronbach α=.77, 95% CI 0.71-0.83). Item-level statistics for the M5 are presented in Table S1 of Multimedia Appendix 1. Convergent validity for the M5 was established via positive and moderate associations with anxiety (r=0.56, P<.001) and depressive (r=0.49, P<.001) symptoms on the BADS (Table 3). Discriminant validity was confirmed for the M5 by negligible relationship with cognitive functions on the BCAT (r=0.17, P=0.15).

Table 3. Correlation analysis (Pearson r and two-tailed P values) among the study measures.
MeasureM5aBCAT bBADSc AFdBADS DFe
M5
r10.170.490.56
P valuef.15<.001<.001
BCAT
r0.1710.030.23
P value.15.81.06
BADS AF
r0.490.0310.52
P value<.001.81<.001
BADS DF
r0.560.230.521
P value<.001.06<.001

aM5: Mood-5 Scale.

bBCAT: Brief Cognitive Assessment Tool.

cBADS: Brief Anxiety and Depression Scale.

dAF: Anxiety Factor of BADS.

eDF: Depression Factor of BADS.

fNot applicable.

Residents with generalized anxiety disorder or anxiety disorder due to a known physiological condition reported significantly higher M5 scores (41/131, 31.3%) than the remaining residents without anxiety diagnoses (diff=1.94; 95% CI −0.92 to 2.95; t129=3.78; P<.001). The effect size for this difference was medium (Cohen d=0.71; 95% CI 0.33-1.09).

Residents with moderate or severe recurrent major depressive disorder (without psychotic symptoms) reported significantly higher M5 scores (22/131, 16.8%) than the remaining residents without these depression diagnoses (diff=3.65; 95% CI 2.49-4.82; t129=6.21; P<.001). The effect size for this difference was large (Cohen d=1.45; 95% CI 0.96-1.95).

COVID-19 Psychological Distress

An M5 cutoff score of 3 (ie, scores ≥3) maximized the product of sensitivity (0.92) and specificity (0.75) for detecting COVID-19 psychological distress among PA/LTC residents (positive predictive value=0.79, negative predictive value=0.91). Area under the curve was 0.89 (95% CI 0.83-0.95), and 16% (21/131) of the residents were incorrectly classified (16 false positive and 5 false negative). Table 4 presents the properties for alternative M5 cutoff scores. Figure 1 illustrates the M5 receiver operative characteristic curve.

Table 4. Predictive utility of several cutoff scores for the Mood-5 Scale.
Cutoff scoreValue (95% CI)
SensitivitySpecificityPPVaNPVb
21.00 (0.93, 1.00)0.56 (0.43-0.68)0.71 (0.60-0.79)1.00 (0.88-1.00)
30.93 (0.83-0.97)c0.75 (0.62-0.85)0.79 (0.69-0.87)0.91 (0.79-0.96)
40.78 (0.65-0.87)0.84 (0.73-0.92)0.84 (0.72-0.92)0.78 (.66-0.87)

aPPV: positive predictive value.

bNPV: negative predictive value.

cItalicized values in the table indicate the M5 cutoff scores with the optimal product of sensitivity, specificity, positive predictive value, and negative predictive value for identifying COVID-19 psychological distress.

Figure 1. Receiver operating characteristics and area under the curve (AUC) were calculated from sensitivity and (1−specificity) values for the Mood-5 Scale for identifying COVID-19 psychological distress among older adults.
View this figure

Our findings support the reliability and validity of the M5 as a mood scale that can identify PA/LTC residents with COVID-19 psychological burden. The M5 is a reliable and valid mood scale that can be completed rapidly, is appropriate for in-person or virtual visits, and can be self-administered. It can be facilitated by a staff member or completed by a resident prior to or during a visit with a health care professional. Given its brevity, the M5 fits easily into an attending physician’s assessment toolbox and can provide real-time information to guide the management of psychiatric medications. This may help rightsize psychotropic use, especially for PA/LTC settings wherein behavioral health specialists are lacking. We recommend a cutoff score of 3 (ie, scores ≥3) to identify those residents who are more psychologically vulnerable and may benefit from a formal mood evaluation. We selected a M5 cutoff score that emphasized sensitivity to identify residents who would benefit from specific counseling to address concerns about COVID-19. Such concerns could be associated with contracting the infection, reduced opportunities for meaningful engagement due to social distancing, and concerns about the health of loved ones.

Our study has several strengths, including (1) our use of the ICD-10 and the Clinical Dementia Rating scale for diagnoses, (2) determination of the COVID-19 psychological burden by an independent review performed by a geriatric psychologist, (3) feedback on the M5 from attending physicians and medical directors, and (4) a selection of widely used validity measures (eg, BCAT and BADS) developed specifically for PA/LTC settings. Owing to the urgency to develop a scale that could be applied to PA/LTC residents during the current COVID-19 pandemic, our sample size was relatively small. This is partly mitigated by the inclusion of residents from multiple settings. The next steps for our study should involve cross-validation, collecting additional data to investigate psychological burden over time as the prevalence of confirmed COVID-19 cases decline, and investigating the psychological burden and associated M5 scores assigned by health care professionals and staff who care for PA/LTC residents. The primary focus of this study was to establish a clinically relevant cutoff score for the M5. Future studies should compare the psychometric properties of the M5 to separate measures of anxiety and depression commonly used in PA/LTC settings, such as the GAD-7 and PHQ-9.

The most immediate implication of this study is that widespread deployment of the M5 in PA/LTC settings can identify residents who are at a higher risk for experiencing COVID-19–related psychological burden and facilitate timely intervention. However, the M5 has potential utility beyond its ability to identify residents with an increased psychological burden associated with COVID-19. For nursing homes, incorporating the M5 into standard screening practices would redress a shortcoming in the current Minimum Data Set (MDS 3.0), which mandates a depression screening but does not include an instrument sensitive to anxiety symptoms. The M5 is sensitive to both depressive and anxiety symptoms. The use of instruments that are sensitive to both anxiety and depression could help reduce rehospitalizations [23], thereby improving some quality measures. Finally, use of the M5 during postacute care can provide a mood baseline that can be used to track mood symptoms postdischarge, thus improving care transitions.

Acknowledgments

The authors would like to thank the health care professionals of CounterPoint Health Services for their assistance with data collection.

Conflicts of Interest

The corresponding author (WEM) has ownership rights of the Mood-5 Scale. RAM is partially employed by Mansbach Health Tools, LLC. The authors report no other conflicts of interest.

Multimedia Appendix 1

Item-level statistics for the M5.

DOCX File , 15 KB

References

  1. Parmelee PA, Katz IR, Lawton MP. Depression among institutionalized aged: assessment and prevalence estimation. J Gerontol 1989 Jan;44(1):M22-M29. [CrossRef] [Medline]
  2. Wagenaar D, Colenda CC, Kreft M, Sawade J, Gardiner J, Poverejan E. Treating depression in nursing homes: practice guidelines in the real world. J Am Osteopath Assoc 2003 Oct;103(10):465-469. [Medline]
  3. Beekman ATF, Bremmer MA, Deeg DJH, Van Balkom AJLM, Smit JH, De Beurs E, et al. Anxiety disorders in later life: a report from the longitudinal aging study Amsterdam. Int J Geriatr Psychiatry 1998 Oct;13(10):717-726. [CrossRef]
  4. Creighton AS, Davison TE, Kissane DW. The prevalence of anxiety among older adults in nursing homes and other residential aged care facilities: a systematic review. Int J Geriatr Psychiatry 2016 Jun;31(6):555-566. [CrossRef] [Medline]
  5. Mansbach WE, Mace RA, Clark KM. The Brief Anxiety and Depression Scale (BADS): a new instrument for detecting anxiety and depression in long-term care residents. Int Psychogeriatr 2014 Nov 18;27(4):673-681. [CrossRef]
  6. New Poll: COVID-19 Impacting Mental Well-Being: Americans Feeling Anxious, Especially for Loved Ones; Older Adults are Less Anxious. American Psychiatric Association - News Releases. 2020 May 20.   URL: https://tinyurl.com/2x6me2fk [accessed 2020-07-02]
  7. People at Increased Risk and Other People Who Need to Take Extra Precautions. Centers for Disease Control and Prevention - COVID-19.   URL: https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-at-increased-risk.html [accessed 2020-07-02]
  8. People Who Live in a Nursing Home or Long-Term Care Facility. Centers for Disease Control and Prevention - COVID-19.   URL: https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-in-nursing-homes.html [accessed 2020-07-02]
  9. Yarnall KSH, Pollak KI, Østbye T, Krause KM, Michener JL. Primary care: is there enough time for prevention? Am J Public Health 2003 Apr;93(4):635-641. [CrossRef] [Medline]
  10. Østbye T, Yarnall KSH, Krause KM, Pollak KI, Gradison M, Michener JL. Is there time for management of patients with chronic diseases in primary care? Ann Fam Med 2005;3(3):209-214 [FREE Full text] [CrossRef] [Medline]
  11. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
  12. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
  13. Yesavage JA, Brink T, Rose TL, Lum O, Huang V, Adey M, et al. Development and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res 1982;17(1):37-49. [CrossRef] [Medline]
  14. Chiang KS, Green KE, Cox EO. Rasch analysis of the Geriatric Depression Scale-Short Form. Gerontologist 2009 Apr 27;49(2):262-275. [CrossRef] [Medline]
  15. Mansbach WE, MacDougall EE, Rosenzweig AS. The Brief Cognitive Assessment Tool (BCAT): a new test emphasizing contextual memory, executive functions, attentional capacity, and the prediction of instrumental activities of daily living. J Clin Exp Neuropsychol 2012 Feb;34(2):183-194. [CrossRef] [Medline]
  16. ICD-10: international statistical classification of diseases and related health problems: tenth revision, 2nd ed. World Health Organization. 2004.   URL: https://apps.who.int/iris/handle/10665/42980 [accessed 2021-03-02]
  17. Morris JC. Clinical dementia rating: a reliable and valid diagnostic and staging measure for dementia of the Alzheimer type. Int Psychogeriatr 1997;9(S1):173-176. [CrossRef] [Medline]
  18. Mansbach WE, Mace RA, Clark KM. Differentiating levels of cognitive functioning: a comparison of the Brief Interview for Mental Status (BIMS) and the Brief Cognitive Assessment Tool (BCAT) in a nursing home sample. Aging Ment Health 2014 Sep 31;18(7):921-928. [CrossRef] [Medline]
  19. R: A Language and Environment for Statistical Computing. R Core Team.: R Foundation for Statistical Computing; 2017.   URL: https://www.r-project.org/ [accessed 2021-03-05]
  20. Integrated Development for R. RStudio; 2016.   URL: http://www.rstudio.com/ [accessed 2021-03-05]
  21. Bujang MA, Omar ED, Baharum NA. A review on sample size determination for Cronbach's alpha test: a simple guide for researchers. Malays J Med Sci 2018 Nov 28;25(6):85-99 [FREE Full text] [CrossRef] [Medline]
  22. Bujang MA, Baharum N. Sample size guideline for correlation analysis. WJSSR 2016 Mar 10;3(1):37. [CrossRef]
  23. Prina AM, Cosco TD, Dening T, Beekman A, Brayne C, Huisman M. The association between depressive symptoms in the community, non-psychiatric hospital admission and hospital outcomes: a systematic review. J Psychosom Res 2015 Jan;78(1):25-33 [FREE Full text] [CrossRef] [Medline]


BADS: Brief Anxiety and Depression Scale
BCAT: Brief Cognitive Assessment Tool
GAD-7: Generalized Anxiety Disorder 7-item
ICD-10: International Classification of Diseases 10th Revision
M5: Mood-5 Scale
PA/LTC: Post-acute and long-term care
PHQ-9: Patient Health Questionnaire


Edited by J Wang; submitted 07.12.20; peer-reviewed by L Costantini, P Tonn; comments to author 20.02.21; revised version received 26.02.21; accepted 28.02.21; published 10.03.21

Copyright

©William E Mansbach, Ryan A Mace, Melissa A Tanner. Originally published in JMIR Aging (http://aging.jmir.org), 10.03.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Aging, is properly cited. The complete bibliographic information, a link to the original publication on http://aging.jmir.org, as well as this copyright and license information must be included.