Asking For Help is Helpful: Validation of a Brief Lifestyle and Mood Assessment Tool in Primary Health Care

INTRODUCTION

There is a progressive trend for primary care to be a continuing health care process, improving community health through preventive services, disease prevention, screening, and generalist first-level interventions.¹ Many at-risk behaviors and mental health issues, however, remain unidentified in routine practice, with an estimated only 30% of those needing treatment receiving it.² To this end, we developed the short, self-administered, Case-finding and Help Assessment Tool (CHAT) for lifestyle and mental health assessment of adult patients (aged 16 years and older) in primary care.³ The tool, which assesses for inactivity, tobacco use, alcohol and other drug misuse, problem gambling, depression, anxiety and stress, abuse and anger problems, and insomnia, was designed by a multidisciplinary team from general practice, nursing, psychology, and university academia. For each item (1 or 2 questions), patients are asked whether they would like help with the issue, either during the consultation or at a later date. The tool has now undergone testing for acceptability, reliability, and validity. Initially the tool included questions assessing for eating disorders, but these were found to be imprecise and replaced by a question on insomnia.⁴ For the current CHAT see the Supplemental Appendix, available at http://annfammed.org/cgi/content/full/7/3/239/DC1.

Patients might object to being asked or be embarrassed by sensitive questions about their lives. For example, studies looking at whether women object to being asked about domestic violence report a wide variability, ranging from 15% to 57%, of patients who are unhappy about being asked.⁵ It was hypothesized that combining issues in a self-administered tool would increase acceptability, reduce the possibility of patients being offended by a particular question, and inform patients that their primary care practitioners are interested in these issues and can offer assistance.

The acceptability of the CHAT was evaluated in a study involving more than 2,500 patients from 20 randomly selected urban and 11 rural general practice physicians and 20 practice nurses.³ Patients wanting help the day of the consultation (0.5% to 13.5%) did not overwhelm the practitioners. The tool was well accepted by patients, with less than 1% objecting to any of the questions. Both physicians and nurses were keen to use the tool once it became available. It takes most patients less than 2 minutes to complete the CHAT.

In separate studies of the 2 depression questions^6,7 and of the anxiety question⁸ plus the help question, the addition of the question inquiring whether help is needed was found to increase test specificity (reducing false positives) while maintaining sensitivity.

Validation of the tool was conducted with 1,000 consecutive primary care patients completing both the CHAT and a composite reference standard.⁴ The aim of our current analysis was to assess the additional value of the help question for each of the individual items in terms of estimating diagnostic accuracy.

METHODS

The CHAT was validated in primary care practices serving socioeconomically deprived populations in South Auckland and socioeconomically advantaged populations in Auckland’s North Shore in 2006–2007.

Consecutive primary health care patients aged 16 years and older were recruited from waiting room situations to complete both the CHAT with the help questions and the composite reference standard (Table 1⇓).^9–15 Although the Diagnostic and Statistical Manual-IV diagnostic interviews might be ideal, selection of the reference standards was pragmatic after considering the time restraints of recruited patients in the waiting room and the prohibitively large sample size required if patients completed only 1 diagnostic instrument on a random basis.

The CHAT and the composite reference standard forms were self-administered by patients in the waiting room. There was a research assistant available to assist with consent and form collection, and they were advised not to look at the screening tool answers when the patients were completing the reference standard. Where the CHAT showed a risk factor that the patient wanted addressed, the family physician could either deal with the problem at the time of the consultation or schedule an appointment for a later date.

The validation was conducted according to the STAndards for Reporting of Diagnostic accuracy studies (STARD) statement for diagnostic tests.¹⁶ Approval for the study was obtained from the Auckland Ethics Committee (REF Study AKY/04/10/367).

Data analysis using Microsoft Excel was conducted on all cases where participants had completed both the base CHAT questions, including the help question, and the reference standard. Reference standard scores were dichotomized as ”case” or “not a case.” Sensitivities, specificities, positive and negative predictive values, and likelihood ratios were calculated using the Center for Evidence-Based Medicine online statistical calculator (http://www.cebm.utoronto.ca/practise/ca/statscal/).

The sensitivity and specificity of CHAT questions compared with those for the reference standards were calculated for items relating to tobacco use, alcohol and other drug misuse, problem gambling, depression, anxiety and stress, abuse, and anger problems. We omitted from this analysis the question relating to inactivity, because the initial format of the question in the validation study was reversed (a yes response meant that the person was physically active), which was confusing and led to inverted responses in some cases. The current version of CHAT has addressed this issue. Specificity was also calculated for those items for which patients answered yes to a help question (either wanting help today or later).

RESULTS

Although 1,000 patients were recruited for the validation study,⁴ only 755 completed the CHAT forms, including the help questions, and the reference standards. There was a 2% decline from consecutive eligible patients invited to participate. The case prevalence detected by the reference standards ranged from 0.3% for being a victim of verbal abuse or for having difficulty controlling one’s physical anger to 11.3% for problematic drinking (Table 1⇑).

Table 2⇓ displays the percentage of positive cases with a request for help. Requests for help ranged from 11% for those with problematic drinking to 57% for those with major depression. Most patients wanting help, however, did not request it for the current consultation but were prepared to return later to address the issue.

Sensitivity ranged from 80% to 98% for the more-common conditions (depression, nicotine dependency, anxiety, problematic drinking) (Table 3⇓). For less-common conditions, such as problematic gambling and drug use, abuse, and difficulty controlling anger, sensitivities are lower and confidence intervals are wider.

For some conditions, particularly depression and anxiety, specificity is relatively low (73% and 77%, respectively), meaning that along with true positives, responses to the CHAT included a number of false positives. Even so, because the CHAT in effect has 2 tiers (patients only respond to the help question if they have indicated problem condition), the help question can improve the specificity without reducing the sensitivity. Once a patient has indicated yes to a CHAT issue, that patient is identified as CHAT positive, and only CHAT-positive patients indicate whether they want help. Table 4⇓ shows that for each condition the specificity increases with the addition of the help question (patients who say they want help are much more likely to have the condition in question).

Table 5⇓ displays the likelihood ratio of the reference standard being positive when patients request help (either today or later) for the more-common conditions of nicotine dependency, problematic drinking, major depression, and anxiety. When calculating the likelihood ratios of being a positive case if the help question answer is positive, wanting help today indicates a very high likelihood of the person being positive for that particular condition. Positive likelihood ratios for all 4 conditions show a consistent downward progression from wanting help today to wanting help at a later date. When wanting help today or later are combined, the likelihood ratios range between 9.4 and 11.8.

DISCUSSION

The findings indicate that the help question serves as a second step for case finding and increases the specificity of the test without compromising its sensitivity. The help question serves to reduce the false positives in that patients who have a CHAT-positive response and who indicate they would like help are very likely to have the condition being assessed. This finding is consistent with those of previous studies in which we compared a composite international reference standard with the 2 screening questions for depression⁶ and the anxiety questions⁷ with and without the help questions.

Using the help question reduces the consulttion time needed to determine which patients require immediate intervention. The results also show a progressive increase in the likelihood ratio for positive case findings from wanting help some time in the future to wanting help today, which raises the posttest probability.

It should be noted that false-positive CHAT responses may represent patients with subsyndromal conditions. Time is a tool, and once an issue has been highlighted, physician and patient may explore the issue further at a later date.