Abstract
PURPOSE Patients who regularly see more than one clinician for health problems risk discontinuity and fragmented care. Our objective was to develop and validate a generic measure of management continuity from the patient perspective.
METHODS Themes from 33 qualitative studies of patient experience with care from various clinicians were matched to existing instruments to identify potential measures and measurement gaps. Adapted and new items were tested cognitively, and the instrument was administered to 376 adult patients consulting in primary care for a variety of health conditions but seeing clinicians in a variety of settings. After initial psychometric analysis, the instrument was modified slightly and readministered after 6 months. The analysis identified reliable subscales and their association with indicators of continuity.
RESULTS Observed factors correspond to 8 intended constructs, with good reliability. Three subscales (12 items) relate to the principal clinician and cover management and relational continuity. Four subscales (13 items) are related to multiple clinicians and address team relational continuity and problems with coordination and gaps in information transfer. Two (11 items) pertain to the patient’s partnership in care. Subscales correlate well and in expected directions with indicators of discontinuity (wanting to change clinicians, suffering, and sense of being abandoned, medical errors) and degree of care organization.
CONCLUSION The instrument reliably assesses both positive and negative dimensions of continuity of care across the entire system, and the subscales correlate with continuity effects. It supports patient-centered and relationship-based care and can be used as a whole or in part to assess coordination and continuity in primary care.
- outcome and process assessment (health care)
- delivery of health care
- continuity of patient care
- health care evaluation mechanisms
INTRODUCTION
Today’s patients see multiple clinicians more often than not, so it is critical to have instruments that validly and reliably assess breaks in the connectedness or coherence in the health care trajectory (discontinuity) or, alternatively, how much patients experience health care services as connected and consistent with their evolving health needs.1
In family medicine, continuity of care usually denotes concentrating care with a single clinician or in clinician group. Repeated contacts that enhance whole-person knowledge, therapeutic alliance, and care consistency are called relational continuity. Elsewhere, continuity that mainly is about consistent and coherent management of health problems over time is called management continuity.1
A 2001 cross-disciplinary synthesis provided a common framework for continuity of care and reviewed the available measures.2 The few measures of management continuity focused on a specific step in the care process, such as compliance with follow-up visits3 or completeness of information transfer.4 Usually, continuity was inferred from compliance with disease-specific protocols, confounding quality of care with continuity. Measures were needed of the patient’s experience, especially of management continuity, and indeed, various measures were developed. Most, however, focus on a specific care context such as discharge from hospital,5,6 or on single problems such as diabetes7,8 heart disease,9 or mental health.10–12 Primary care requires a generic instrument, applicable to a broad range of health conditions (including multimorbidity) and patients in an ambulatory setting. Some subscales measure coordination in primary care,13–15 but these focus on the primary care physician and do not capture the patient’s continuity experience at the system level.16,17 An interesting measure is the overall coordination subscale in the Veterans Affairs National Outpatient Customer Satisfaction Survey.18 This work inspired our approach to measurement but does not cover all aspects of continuity important to patients.
Our objective was to develop and validate a generic measure of management continuity from the patient perspective, applicable in primary health care but capturing continuity across the entire system. To do so, we undertook a sequential mixed-method study. The first phase was a meta-summary of 33 qualitative studies of patient experience when care is received from various clinicians over time. A meta-summary19 uses study reports as the qualitative data that are independently coded by researchers to identify themes across studies. The examination of various study designs, patient groups, health conditions, and disciplinary perspectives is more suitable for identifying generic continuity-related issues than is a single qualitative study. Our initial intent was to measure management continuity, but we found that it could not be dissociated from relational and informational continuity. In addition, the qualitative studies evoked dimensions of partnership and patient role not previously emphasized.20 We identified a sense of security rather than smoothness as the hallmark of management continuity for patients. A secondary objective was to measure relational continuity with a team. Here we present the development and metric properties of the quantitative measures and association with continuity indicators.
METHODS
Item Development
Codes from the meta-summary of qualitative studies between 1950 and 2007 were reformulated as latent variables with suggested indicators. These indicators were mapped to relevant items in 23 instruments (mapping available on request). We built on developed items on the assumption that they had adequate metric properties with some level of validation. Where entire subscales mapped to a code and indicators, we adapted these subscales (eg, Primary Care Assessment Survey: contextual knowledge).13
We adapted relevant items to reflect the qualitative codes or to fit the framing and response options for a section; new items were developed where necessary. For instance, information to patients emerged as a dimension of information continuity, but relevant items focused only on transmission of information, not the impact on the patient’s role in care, so we adapted the items accordingly. Likewise, the importance of a coordinator is widely recognized, but no measures captured the scope of the coordinator’s actions, so we developed new items. Some qualitative dimensions were dropped as a result of instrument length.
Because the meta-summary includes researcher interpretations of qualitative data, we conducted extensive cognitive testing to ensure that patients understood the ideas suggested by researchers and found them relevant. Iterative cognitive testing and item modification were done on convenience samples (n = 7), after which we recruited patients in family practice waiting rooms (n = 20) and by telephone interviews of previously tested and new patients. Many items were excluded after cognitive testing, mostly aspects of coordination that patients do not observe or experience directly. Considerable refinement was needed to find lay synonyms for health care jargon, such as coordination and treatment plan. We used the word person for clinician and found that respondents identified the appropriate type of clinician without being confused by the more difficult term of health professional. We developed items simultaneously in French and English (Canadian) to achieve semantic equivalence in both languages.
The instrument includes a mix of evaluative and reporting items and indicators of both positive and negative continuity experiences. Questions offer 5-point Likert response options; some were collapsed to 3-point scales after initial analysis. Evaluative items required judgment about how much the phenomenon was present or achieved (hardly at all to a great deal); all reflected positive experiences of continuity. Item values were averaged to capture the subscale construct. Frequency response options for reporting (never to almost always) were dichotomized then summed for the subscale score. All subscales except care plan captured negative experience of continuity.
The initial instrument contained 84 questions that included 40 on health care use, functional health status, shared decision-making preferences, and socio-demographic characteristics. After initial psychometric analysis, we modified the instrument and added more indicators of discontinuity and good continuity to assess predictive validity. The refined instrument had 37 items on 9 dimensions of continuity, as well as indicators of continuity problems.
Instrument Administration
The self-administered instrument was applied twice, at 6-months intervals (T1 = baseline, T2 = 6 months after T1), to adult patients aged 25 to 75 years, recruited in waiting rooms of 6 primary care clinics. Eligible patients had received care for the same health condition at more than one place in the last year and expected to continue to do so during the next 6 months. To contrast expected continuity, 3 clinics were in a highly integrated health network and 3 in a poorly integrated network.
Analysis
In initial psychometric analysis, no items were excluded because of missing values (>4% criterion), lack of variance, or floor or ceiling effects. We conducted exploratory factor analysis to assess whether items loaded on expected factors. Within each factor, we examined the performance of individual items with item-total correlations and nonparametric and parametric item response theory analysis.21,22 We eliminated redundant or poorly functioning items, and added or collapsed response options to reduce the skewed distribution and enhance reliability.
After T2 we examined the goodness-of-fit of the factor resolutions and tested alternate configurations using structural equation modeling or confirmatory factor analysis.23 We estimated the correlations between the subscales to examine how subscales were related and whether they demonstrated distinctiveness. Finally, we used logistic regression modeling to examine how subscales were associated with binary indicators of discontinuity and other outcomes, such as the occurrence of medical errors, and whether the odds of outcomes increased with subscale scores in an ordinal, linear and interval manner. Adjustment for age-group, morbidity, and education did not affect predictive validity. Regression and basic psychometric analyses were conducted with SAS 9 (SAS Institute Inc).24
RESULTS
In the clinic waiting rooms we found 615 eligible patients of whom 61% consented to participate, and 79.6% (300 of 376) returned the baseline questionnaire (T1); of these 85.3% (256 of 300) responded at 6 months (T2), the sample for this article. The characteristics of the study population and comparison of respondents and nonrespondents at T2 (n = 256) are shown in Table 1. Respondents tended to be older and less likely to be in full-time employment or school, but there was no difference in education or health status. Though not statistically significant at P = .05, a higher proportion of respondents than nonrespondents reported having a family physician and care coordinator, suggesting that those without may have had difficulty answering the first questions about the main physician. Most respondents (93%) identified a principal physician and a person who coordinated their care. Almost 90% of respondents were French-speaking.
Instrument Content and Factor Analysis
Table 2 summarizes the instrument content. Subscales are grouped by those pertaining to the principal physician (management and relational continuity), various clinicians (team relational, management and information) and engagement of the patient as partner (management and information). The instruments that inspired the item content are indicated. The detailed instrument is also available as a Supplemental Appendix, at http://www.annfammed.org/content/10/5/443/suppl/DC1.
Table 3 presents the summary of the psycho-metric performance of the subscales. The item-scale correlations generally indicate high consistency within the subscales, with an internal reliability that was higher than the accepted criterion of 0.70, except for the role clarity and coordination within the clinic sub-scale (α= .66), for which a low value reflects the small number of respondents consulting various clinicians in their regular clinic in the last 6 months (102, 40%). A care plan item on being told whom to contact had a low item-scale correlation (0.38) and factor loadings (0.43) and was dropped, resulting in good internal reliability and increased uniqueness relative to the other subscales. We attempted to remove from the information gap subscale the item on repeating tests, but doing so did not improve internal consistency. We judged removing this item to result in a loss of information when we examined it against other subscales and indicators of discontinuity, so it was maintained in the subscale.
Exploratory factor analysis was performed separately within the report and evaluative format types, then overall. Table 3 shows the ranges of subscale-specific loadings. All items loaded principally within intended factors, except for items of relational continuity with the principal physician, which loaded in a single factor rather than 2. In confirmatory factor analysis, however, the 2-factor model fit substantially better than the 1-factor model.
Features of Subscales
The descriptive statistics and score distribution are shown in the last columns of Table 3. The full range of scores was present for all subscales, and the ceiling and floor effects are in keeping with each subscale. Generally, the subscales show sufficient variability to meaningfully permit assessment and detect change overtime.
The evaluative subscales, except for confidence in team, have a skewed distribution: most assessments are positive. The reporting subscales are skewed in the opposite direction, with most respondents reporting no problems. For role the clarity and coordination subscale, three-quarters reported no problems, whereas only 37% reported no problems with an information gap. Care plan, invoked consistently as an important support for coordination and continuity, proved difficult to translate for patients. After much cognitive testing and modification, the final subscale shows good reliability, with correlations between subscales that make sense conceptually.
Table 4 shows the Pearson correlation between the subscales. Correlation coefficients are appropriately negative between positive measures of continuity and discontinuity. The subscales pertaining to the action or knowledge of the principal clinician are most highly correlated (coordinator role, comprehensive knowledge, and confidence and partnership). The modest correlation between the confidence in team subscale and measures of relational continuity with the family physician (r ≈ 0.42) supports team relational continuity as a unique and distinct construct. Finally, Table 4 includes the correlation between subscales and a single-item question eliciting the patient’s assessment of how well-organized her or his health care seems to be. All subscales correlated significantly with this item, consistent with the intended function of the subscales as measures of continuity and coordination when patients see various clinicians.
Association With Indicators of Discontinuity
Table 5 shows the extent to which a unit increase in each subscale score significantly increases or decreases the odds of indicators of discontinuity, as an appreciation of the subscales’ predictive validity. The table also shows the percentage of respondents that experienced indicator problems. Odds ratios of less than 1 show the subscale construct is protective against discontinuity, as illustrated by a larger coordination role or stronger relationship with the patient. The desire to change the responsible physician decreased even more dramatically with better partnership and confidence than with comprehensive knowledge or the larger coordinator role. Having to organize care alone or sensing that no one was in charge decreased by one-third for every unit increase in the coordinator role or self-management information score and by one-half with better care plan and confidence in team subscale scores.
In contrast, problems with role clarity and coordination and with information gaps subscales are most strongly associated with increased risk of discontinuity, as well as inappropriate emergency department use and medical errors. So, for instance, among the majority who did not experience any information gap, only approximately 5% reported experiencing a medical error compared with 35% of those reporting the most problems. Likewise, less than 10% of those reporting no problems believed that their emotional or physical heath had suffered because their care was poorly organized, compared with more than 60% of those reporting the most problems.
DISCUSSION
The measure of continuity of care presented in this article captures 9 dimensions of continuity experienced by patients when they encounter multiple caregivers in various places. The instrument includes a measure of team relational continuity, recognizing that patients can and do establish relational continuity with more than 1 care clinician.30 Though the instrument builds on previous measures of continuity of care, this measure advances by integrating different types of continuity and refinements that emerge from qualitative studies and a variety of health conditions. For instance, the subscale of contextual knowledge from the Primary Care Assessment Survey13 captures an important dimension of relational continuity, but when patients receive care from various clinician, they not only want to have 1 clinician who knows them but also who applies that knowledge to designing solutions for their health problems31–33; our comprehensive knowledge subscale captures that finding. The subscales show good reliability and capacity to detect discontinuity. Overall, this generic measure of continuity of care is successful.
That our instrument measures aspects of care coordination within the direct experience or observation of the patient is a strength because it reduces guessing and confounding by respondent expectations. It may also seem a limitation, because much of care coordination occurs outside the purview of the patient and cannot be measured. For instance, our initial attempts to capture reports of information exchange between clinicians were unsatisfactory because, although we are confident that reported exchanges actually occurred, other exchanges not observed directly escaped measurement (false negatives or poor sensitivity). Likewise, our initial care plan measure failed, in part, because the elements were stated in terms familiar to clinicians, not patients (eg, treatment plan, monitoring). Indeed, the dimensions of care coordination in the recently published Care Coordination Atlas34 are all defined from the clinician’s perspective. Patients can accurately and validly judge perceived coordination failures, but evaluating the extent of coordination activities requires complementary measures from the clinician’s perspective.
This study has several other limitations. First, we depended heavily on the themes that emerge from a meta-summary of qualitative studies, which increased the contribution of researchers’ judgments and removed us from the patient voice. We compensated with extensive cognitive testing to ensure relevance to patients. Second, we were heavily inspired by previously developed items, which some argue is inappropriate. We cannot pretend ignorance of previous work and opted to be transparent about its influence in our instrument. We emphasize that we used items as indicators of a new construct or latent variable, not as proxies of the construct inferred by the original developer. This instrument still represents new knowledge, because no single instrument or subscale has combined these indicators to capture the dimensions suggested by qualitative studies. Finally, these results reflect predominantly French respondents in Quebec. Based on our cognitive testing and previous work in making French versions of English instruments,35 we are confident that French and English versions are mostly equivalent, though our small sample of English respondents (n = 27) limited statistical power to detect differential item functioning. More important is the limit to generalizability, because continuity is sensitive to health system structures. We may have excluded elements not currently prevalent in Quebec (e-mail contact, patient internet portals) or overrepresented others, such as our current crisis of access to family physicians. The ultimate test of validity will depend on how this tool performs across a variety of contexts.36
Continuity of care is a multidimensional construct, including at least 3 types of continuity and referring to both processes (information transfer, coordination) and intermediate outcomes (sense of security, fragmentation). This instrument does not capture every aspect of continuity that emerged from our qualitative analysis, but we believe it captures the most common elements when patients encounter more than one clinician. This has implications for how the tool might be applied. We encourage evaluators to identify aspects of continuity that are most important to them and select relevant subscales or indicators. Indicators of discontinuity or about the organization of care can be used as stand-alone indicators to provide a snapshot of continuity. Individual subscales could be then applied to help identify the source of the problems and to benchmark quality improvement initiatives. The period of reference for questions can be adapted as necessary, keeping in mind that longer time-frames may risk lack of precision. We recommend minimizing branching in the questionnaires, though items to identify relevant subgroups are useful. For instance, problems with role clarity and coordination within the clinic were answered only by those seeing multiple clinicians within their primary care clinic during the last 6 months, though our findings suggest that it is much more disturbing to patients to have poor coordination within the medical home than between organizations. It would be better to have all respondents answer all questions and then do subanalyses.
The ultimate test of this tool will be to show whether improving continuity translates into better quality of care and health outcomes. Our results with generic indicators suggest that such relationships are likely, but this association should now be tested. This instrument represents an important addition to tools that assess care from the patient’s perspective, addressing dimensions of information transfer, care plan and monitoring, support for self-management, and team-work focused on coordination. It further supports the primary care values of patient-centered and relationship based care.
Acknowledgments
Christine Beaulieu coordinated all aspects of the study operationalization. Cognitive testing and subject recruitment was done by Taika Baiergeron; survey administration also by Suzanne Descent, Martine Fournier; analysis, by Fatima Bouharaoui and Steven Sanche.
Footnotes
-
Conflicts of interest: authors report none.
-
Funding support: This study was supported by a research grant from the Funds for Health Research of Quebec (FRSQ) with research infrastructure provided by the Canadian Foundation for Innovation. During the conduct of this study Dr Haggerty held a salary award as a Canada Research Chair funded at the Université de Sherbrooke funded by the Canadian Institute for Health Research. Dr Breton holds a postdoctoral fellowship by the Canadian Health Services Research Foundation.
-
Prior presentations: Some of the material in this article was presented at the following peer-reviewed scientific conferences:
Haggerty J, Beaulieu C, Bouharaoui F, Fournier M, Freeman G, Roberge D. When patients see many providers: development and validation of a generic measure of management or cross-boundary continuity of care. North American Primary Care Research Group (NAPCRG), Seattle, Washington, November 13–17, 2010.
Breton M, Haggerty J, Roberge D, Freeman G. Is there a relationship between integrated services network and continuity of care perceived by patients? Canadian Association for Health Services and Policy Research (CAHSPR), Halifax, Nova Scotia, May 9–12, 2011.
Haggerty J, Freeman G, Roberge D, Beaulieu C, Breton M. When patients see many providers: a generic measure of management continuity of care [poster]. Society for Academic Primary Care, Bristol, July 6–8, 2011.
- Received for publication May 25, 2011.
- Revision received December 16, 2011.
- Accepted for publication December 30, 2011.
- © 2012 Annals of Family Medicine, Inc.