Abstract
PURPOSE Most people with mental disorders receive treatment in primary care. The charts developed by the Dartmouth Primary Care Cooperative Research Network (COOP) and the World Organization of National Colleges, Academies, and Academic Associations of General Practitioners/Family Physicians (WONCA) have not yet been evaluated as a screen for these disorders, using a structured psychiatric interview by an expert or considering diagnoses other than depression. We evaluated the validity and feasibility of the COOP/WONCA Charts as a mental disorders screen by comparing them both with other questionnaires previously validated and with the assessment of a mental health specialist using a structured diagnostic interview.
METHODS We trained community health workers and nurse assistants working in a collaborative mental health care model to administer the COOP/WONCA Charts, the 20-item Self-Reporting Questionnaire (SRQ-20), and the World Health Organization–Five Well-Being Index (WHO-5) to 120 primary care patients. A psychiatrist blinded to the patients’ results on these questionnaires administered the SCID, or Structured Clinical Interview for the DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition).
RESULTS The area under the receiver operating characteristic curve was at least 0.80 for single items, a 3-item combination, and the total score of the COOP/ WONCA Charts, as well as for the SRQ-20 and the WHO-5, for screening both for all mental disorders and for depressive disorders. The accuracy, sensitivity, specificity, and positive and negative predictive values of these measures ranged between 0.77 and 0.92. Community health workers and nurse assistants rated the understandability, ease of use, and clinical relevance of all 3 questionnaires as satisfactory.
CONCLUSIONS One-time assessment of patients with the COOP/WONCA Charts is a valid and feasible option for screening for mental disorders by primary care teams.
- Mental disorders
- depressive disorders
- anxiety disorders
- screening
- primary health care
- collaborative care
- COOP/WONCA Charts
- SCID
INTRODUCTION
Epidemiologic studies of the adult population have found the prevalence of mental disorders is as high as 20% in 1 year and 40% in a lifetime.1–3 Disabilities, decreased quality of life, and the economic consequences associated with the presence of common mental disorders are similar to those associated with common nonpsychiatric diseases.1,4,5 Moreover, the presence of mental disorders increases the risk and worsens the prognosis of these nonpsychiatric diseases.6,7 Many people with mental disorders receive treatment in primary health care facilities; however, primary care teams often detect, correctly diagnose, and adequately treat less than 50% of mental disorder cases.8–10 Increasing evidence shows that there is a better chance of achieving good mental health outcomes when these teams work within collaborative care models that include mental health specialists who perform consultation/liaison activities and provide specialized care in primary care facilities.11–15
In primary care services with these organizational characteristics, questionnaires can be used to facilitate screening for mental disorders.16–18 Examples of such questionnaires include the 20-item Self-Reporting Questionnaire (SRQ-20),19 the General Health Questionnaire,20 the World Health Organization–Five Well-Being Index (WHO-5),21 and the first phase of interviews of the Symptom-Driven Diagnostic System for Primary Care22 and Primary Care Evaluation of Mental Disorders.17 The Dartmouth Primary Care Cooperative Research Network (COOP) and the World Organization of National Colleges, Academies, and Academic Associations of General Practitioners/ Family Physicians (WONCA) developed a 6-item questionnaire or set of charts to evaluate the physical, emotional, and social well-being of a person subjectively, called the COOP/WONCA Charts.23 These charts include a specific item to assess mental health (Feelings), as well as items of Physical Fitness, Daily Activities, Social Activities, Change in Health Status, and Overall Health. The WHO-5, which was originally developed to assess well-being in patients with chronic conditions, is an efficient screen for depressive and anxiety disorders.24 Accordingly, it is possible that the COOP/WONCA Charts, in addition to their original objectives, could also identify patients who are mentally ill. To date, previous studies showed significant concordance only between the COOP/WONCA Charts and a depression diagnosis made by general practitioners or rating scales of depressive symptoms.25–27
In this study, we compared, in a collaborative mental health care model, the validity and feasibility of the COOP/WONCA Charts, the SRQ-20, and the WHO-5 administered by Brazilian primary health care teams as screening tools, for mental disorders in general and also for depressive and anxiety disorders in specific, using the SCID, or Structured Clinical Interview for the DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition), clinical version,28,29 to definitively establish the presence or absence of these disorders.
METHODS
Context of the Study
Since 1999, primary health care teams linked to the Ribeirao Preto Medical School work according to the Brazilian Family Health Program (FHP)30–33 and have access to mental health specialists who perform weekly consultation/liaison activities and provide specialized care in FHP facilities. Additionally, family physicians and primary care nurses are trained in mental health in academic seminars during a period of 1 year (40 seminars in total, 2 hours each), with goals of integrating mental health in a patient-centered/biopsychosocial approach34 and enabling these professionals to provide continuous support in this area for community health workers and nurse assistants.
At the time of this study in 2003, there were 4 FHP teams, covering a population of approximately 15,000 inhabitants. Each team had 4 community health workers, 2 nurse assistants, 1 nurse, 1 senior family physician, and 2 or 3 family medicine residents.
Questionnaires and Psychiatric Interview
The SRQ-20 was developed by WHO in the 1970s for a multinational study on mental disorders in primary care. It consists of 20 questions with possible answers of only yes or no19 and can be self-administered or administered by an interviewer. It is widely used in studies in developing countries to screen for nonpsychotic mental disorders in primary care, and the Brazilian version has been validated.35 Higher scores indicate more symptoms.
The WHO-5 is a questionnaire that measures wellness21 and can discriminate between older adults with and without mental disorders.24 There are 2 versions, each having 5 items; Version 1 detects anxiety and depression, while Version 2 detects only depression.36 On the basis of these data and considering the importance of detecting various mental disorders in primary care, we translated the first version for this study. Higher scores indicate greater wellness.
The COOP/WONCA Charts were developed in the late 1980s and early 1990s under the auspices of WONCA, with the aim of subjectively evaluating an individual’s functional status, defined as assessing his or her physical, emotional, and social well-being. These charts measure functioning in 6 aforementioned domains. Each chart consists of a single question referring to the preceding 2 weeks and has 5 possible answers, illustrated by a simple picture. This tool has been translated into multiple languages and validated in several countries. We used the Portuguese version, with minor modifications for use in Brazil.23,37,38 Higher scores indicate a worse functional status.
The psychiatric diagnoses that we used as standards for validation were established using a structured psychiatric interview, the SCID, translated for use in Brazilian context.28,29
Feasibility Assessment
The community health workers and nurses assistants who administered the SRQ-20, WHO-5, and COOP/ WONCA Charts completed a questionnaire about the feasibility of their use. It had 9 items focused on understanding of the instructions, questions, and answers of each tool; their brevity, ease of use, and suitability for administration in routine care; and their contribution to identifying patients who needed help in dealing with emotional difficulties (ie, a questionnaire’s clinical relevance). There were 4 response options for each item: unsatisfactory, doubtfully satisfactory, reasonably satisfactory, and fully satisfactory.39,40
Participants
The study was approved by the Research Ethics Committee of the University Community Health Center of the University of Sao Paulo, and each patient and team member gave written consent to participate. The patients included had no previous interaction with the consultant psychiatrist (either through personal attendance or case discussion). The patients registered with teams were similar to the general population in terms of the sex ratio (with about 50% of each sex) and age (with about 50% aged between 20 and 39 years).
We selected a sample size of 120 patients based on a mathematic model for sample size calculation for studies assessing the sensitivity and specificity of diagnostic tests.41 Based on previous studies of prevalence of mental disorders and validity of screens for them in Brazilian primary health care,8,35 we considered the following expected parameters: a sensitivity of 85%; a specificity of 80%; a 95% confidence interval of 10% for sensitivity and specificity; and a prevalence of 40% of mental disorders in general. To achieve this sample size, a total of 168 patients were invited, of whom 21 did not participate because they had been previously personally assessed by the consultant psychiatrist or involved in the consultation/liaison discussions, 20 because they declined (the main reason given was lack of time), and 7 because they did not complete the psychiatric interview.
Data Collection
We trained the community health workers and nurse assistants in how to administer the 3 questionnaires in meetings lasting 2 hours with the consultant psychiatrist. The meetings also included discussion of the impact that psychological distress and mental disorders have on patients’ lives and how to facilitate patients’ communication of such difficulties. The training consisted of reading and dramatization of applications.
At random intervals (on average, once weekly), a researcher visited primary care facilities and invited 1 or 2 patients who were waiting to participate in the study. When patients agreed to participate, community health workers and nurse assistants administered the SRQ-20, the WHO-5, and the COOP/WONCA Charts in a random order during the routine care of these patients. Within a week, a psychiatrist who was trained in the use of the SCID42–44 and had participated in previous studies in which it was used administered the interview in person, remaining blinded to the patients’ results on the previous questionnaires. Patients who were determined to have a mental disorder received adequate care in the primary care facilities.
By the end of data collection, 14 community health workers and nurse assistants had administered the screening questionnaires 4 to 9 times each and had completed the questionnaire about feasibility.
Statistical Analysis
Statistical analyses were performed using the Statistical Package for Social Sciences (SPSS) version 13.0 (SPSS Inc, Chicago, Illinois). To evaluate the predictive validity of the screening questionnaires, considering the presence or absence of psychiatric disorders as measured by the SCID as the reference standard, we generated a receiver operating characteristic (ROC) curve for each questionnaire and calculated its sensitivity, specificity, positive and negative predictive values, and total accuracy. An area under the curve (AUC) of 1.0 was considered to indicate perfect discrimination; an AUC of at least 0.8, good discrimination; and an AUC of less than 0.7, poor discrimination.45,46 The 4 response options for each item of the feasibility questionnaire were grouped into 2 categories, satisfactory (combining reasonably satisfactory and fully satisfactory) and unsatisfactory (combining unsatisfactory and doubtfully satisfactory), and evaluated using absolute numbers and percentages.
RESULTS
The demographic and clinical characteristics of our patient sample are presented in Table 1⇓. Overall, 36.7% of patients had at least 1 mental disorder, and depressive and anxiety disorders were most prevalent. Regarding specific diagnoses, the most common were major depressive episode (15.8% of the sample), generalized anxiety disorder (10%), panic disorder (4.2%), and dysthymia (3.3%). A substantial proportion of patients (7.5% of the sample) had comorbid depressive and anxiety disorders.
Patient Characteristics (N = 120)
The AUCs for the SRQ-20, the WHO-5, and various COOP/WONCA Charts measures—the total score, the Feelings item, the Daily Activities item, and a 3-item combination of Feelings, Daily Activities, and Social Activities—indicated that these measures had good discrimination (AUC ≥0.8)46 for any mental disorder and for depressive disorder (Table 2⇓). The COOP/ WONCA Charts total score and the 3-item combination also had good discrimination for anxiety disorders.
Area Under the ROC Curve (95% Confidence Interval) for the Questionnaires Studieda
With regard to the diagnosis of any mental disorder by the SCID, the accuracy of the SRQ-20, the WHO-5, and the COOP/WONCA (the total score, the 3-item combination, and the single item Feelings) were high and similar, with values ranging from 0.85 to 0.87 (Table 3⇓).
Cutoff Scores and Performances of the Questionnaires for Screening for Any Mental Disordera
Almost all community health workers and nurse assistants (71.4%–100%) rated the understandability, ease of use, and clinical relevance of the SRQ-20, WHO-5, and COOP/WONCA Charts as satisfactory. When it came to brevity, most community health workers and nurse assistants considered these questionnaires to be unsatisfactory for use at every appointment for each patient (100% for SRQ-20, 71.4% for WHO-5, and 64.3% for COOP/WONCA), but suitable for use every 6 months for each patient (100% for each questionnaire).
DISCUSSION
We found a 36.7% prevalence of mental disorders in our study population, which agrees with the relatively high rates of these disorders in Brazilian primary care facilities found in other studies.35,47 Additionally, our moderate to good results obtained using the SRQ-20 and WHO-5 to screen for general mental disorders and for depression, respectively, are in accordance with results of previous studies conducted with primary care patients in other countries.48–52
The main findings of this study are the good psychometric characteristics of the COOP/WONCA Charts, when administered by community health workers and nurse assistants after a training program conducted within a collaborative care model, to screen both for mental disorders generally and for specific ones present in primary care. Previous studies comparing the Feelings item from the COOP/WONCA Charts with a diagnosis of depression made by general practitioners or rating scales of depressive symptoms showed significant concordance.25–27 This association had previously not been studied using a structured psychiatric interview administered by an expert or considering diagnoses other than depression, however.
The COOP/WONCA Charts are recommended for routine use by WONCA to monitor the health status of and to facilitate bonding with the patient.23,37 This questionnaire is friendly, brief, and easily understood, and can be administered and scored by all team members (or, alternately, can be self-administered). These features, coupled with the results on validity and feasibility as a screen for mental disorders, suggest this it is a good option to accomplish screening in this setting, considering the competing demands and time constraints of primary care services.53,54 Routine use of the COOP/WONCA Charts by community health workers and nurse assistants in primary care facilities could thus identify patients who may have mental disorders who could be further evaluated by more detailed interviews, enabling appropriate care sooner. Another important finding is that the Feelings item of the COOP/WONCA Charts has good validity in screening for depression and for mental disorders in general. With respect to depression, some studies have similarly found good results,55–58 although a recent meta-analysis considering all studies of the issue does not recommend the use of just a single question to detect depression.59 Yet, an earlier study found that a single question of the 5-item Mental Health Inventory (MHI-5) can have a moderate validity for tracking mental disorders in general in primary care.55 Regardless, we observed the best psychometric characteristics in the present study with combination of 3 items of the COOP/ WONCA Charts (Feelings, Daily Activities, and Social Activities).
This study has its limitations. Some diagnoses (eg, posttraumatic stress disorder and bipolar disorder) were absent or rare in our sample and could not be evaluated. Only 1 psychiatrist administered the SCID, and that professional was the same one who trained the community health workers and nurse assistants. The psychiatrist was blinded to the diagnoses obtained with the questionnaires until the end of the data collection phase, however. Hence, it is unlikely that the expectation of a good result from the training affected the results. It is also possible that diagnostic errors occurred systematically in application of the SCID, but such errors are also unlikely because the professional’s initial SCID training involved group interviews, with other specialized professionals, with appropriate inter-rater reliability.43,44 It is also questionable as to whether the study’s results can be generalized to staff who do not work in collaborative care programs and do not have consultation/liaison psychiatrists and specific training in the questionnaires studied.
In conclusion, we found that the COOP/WONCA Charts are a useful instrument to screen for mental disorders with good psychometric qualities in a real-world setting when administered by community health workers and nurse assistants. This finding presupposes that training in their use is incorporated into multifaceted learning strategies and that use occurs in the context of a collaborative care model having both mental health specialists and primary care teams. Future studies should examine the impact of using this questionnaire among patients with mental disorders treated in primary care, including its use in conjunction with structured diagnostic interviews conducted by physicians and nurses in primary care to diagnose specific mental disorders42 and as an instrument to assess the outcome of treatments for these disorders.
Footnotes
-
Conflicts of interest: authors report none.
-
Funding support: This study was supported by a grant from “Conselho Nacional de Desenvolvimento Científico e Tecnológico” (“CNPq”), Brazil, number 575094/2008-7. A.W.Z is recipient of a productivity fellowship (CNPq, Brazil).
- Received for publication November 3, 2010.
- Revision received February 26, 2011.
- Accepted for publication March 15, 2011.
- © 2011 Annals of Family Medicine, Inc.
