Measuring Continuity of Care in Diabetes Mellitus: An Experience-Based Measure

Development of the Measure

We held in-depth interviews with 25 type 2 diabetic patients to explore their experiences and values with respect to continuity of care.¹⁹ We found that patients’ experiences of continuity of diabetes care can be characterized in terms of 4 dimensions. (1) Experienced longitudinal continuity refers to the regular monitoring of the patient and his or her illness over time, with advice on self-management, and care from as few clinicians as possible.²⁰ This process provides the context in which a relationship may develop between the patient and a usual doctor or nurse based on familiarity, closeness, and trust. This is characteristic of (2) experienced relational continuity.²⁰ When patients experience problems with their diabetes, they may need an urgent consultation, or they may want to speak to their usual doctor or nurse to obtain advice. (3) Experienced flexible continuity characterizes the extent to which clinicians respond flexibly to patients’ changing needs over time. Finally, (4) experienced team and cross-boundary continuity concerns the degree of consistency and coordination of care between different care settings and between different individual clinicians. In our data, patients were unable to comment consistently on the availability of clinical information, so the dimension of informational continuity was omitted.

The 4 dimensions of experienced continuity of care, together with key experiences and values drawn from patients’ accounts, were used to develop questionnaire items. The precise wording of items was based on qualitative data and modified through a process of discussion and consensus among the members of the study team. Item development was also informed by considering items with similar content from established measures of patient satisfaction.^21,22 Prototype items were subject to cognitive testing on several occasions, with small samples of 3 or 4 diabetic patients who were attending a local diabetes clinic, to test that the wording was appropriate and items were understood as intended. A nearly final version of the instrument was formally tested in a pilot study that included a convenience sample of 40 patients attending local family practices. Minor changes to the wording of the instrument were made after the analysis of these pilot data, but these pilot data are not presented here.

Questionnaire Items, Coding, and Scoring

The questionnaire items are shown in Table 1⇓. There are 19 items with 4 items each for the subdomains of longitudinal continuity (LC) and flexible continuity (FC), 6 for relational continuity (RC), and 5 for team and cross-boundary continuity (TCB). Eight of the items asked about care received from the “usual doctor or nurse” in a given setting. The usual doctor or nurse was defined as the “doctor or nurse who knows you and your diabetes best.” The response sets for each item were coded using standard Likert-type scales, each of which had 6 response options. Differently worded response sets were used for different sections of the questionnaire. Six response options were preferred to avoid a bias toward the central option. When only 1 item was missing for a given subdomain, then the value was imputed by taking the mean of the items with complete data. The maximum number of cases with imputed data was 6 for item RC2. Each subdomain was scored by summing the items and then rescaling to give a score out of 25. The total score was obtained by summing the scores for the 4 subdomains, giving a score out of 100. Full details of the questionnaire and coding procedures are available in our full report at http://www.sdo.lshtm.ac.uk/files/project/14-final-report.pdf.

Diabetic patients sometimes receive medical care in more than 1 health care setting. When such was the case, patients completed the items for longitudinal, flexible, and relational continuity with reference first to the family practice setting and then with reference to hospital clinic care, whereas the items for team and cross-boundary continuity referred to patients’ overall experience of care. When patients received care in both family practice and hospital clinic settings, then the higher of the subdomain scores was included in the total score, because the patient was considered to experience the continuity in diabetes care that was offered by the more favorable setting.

Field Testing the Measure

The measure was tested in a cross-sectional survey of patients with type 2 diabetes who were registered with 19 family practices in London, England. The patients were older than 30 years when diabetes was diagnosed and did not require insulin within 6 months of diagnosis. In the United Kingdom, patients with type 1 diabetes generally receive diabetes care from hospital-based clinics. The survey questionnaire included the measure of experienced continuity of care and questions concerning the type of care received (family practice only, hospital only, or shared care), age, sex, ethnicity, duration of diabetes, and general health. The questionnaire was administered during an interview in the patients’ homes. All the main study interviews were conducted either by one author (SN) or an assistant, and both were trained to a common standard. Patient questionnaire data were also linked to data concerning the organization of diabetes care at the family practice; this information included an item concerning whether the family practice had a designated lead doctor for diabetes management.

All patients were interviewed 10 months later to evaluate experienced continuity of care as part of a cohort study to determine whether continuity of care is associated with health outcomes. The results of the cohort study will be reported elsewhere.

Two nested substudies were completed at the 10-month follow-up interview. In the first substudy, the test-retest reliability of the questionnaire was evaluated by repeating interviews over the telephone in a convenience sample of 30 patients, with 26 giving complete total scores. The average interval between interviews was approximately 11 weeks. In a separate substudy, a self-administered version of the questionnaire was completed by a convenience sample of 56 patients, with 48 giving complete total scores, and these responses were compared with those obtained by interview. The interval between interviews and self-administered questionnaires ranged from 1 day to 1 month.

Analysis

Correlations between each item and the total score (item-score correlations) and Cronbach’s α were estimated. The factorial composition of the measure was evaluated with factor analysis using the principal factor method in Stata version 9.²³ Factor loadings were obtained after varimax rotation by default, but oblique rotation led to a similar interpretation. The number of factors was selected after inspecting the eigenvalues and a scree plot, and by using maximum likelihood estimation to compare the goodness-of-fit of models with different numbers of factors.^24,25 Test-retest reliability was evaluated by estimating the mean difference between scale scores from successive administrations.²⁶ To evaluate criterion validity, linear regression models were fitted to evaluate whether mean experienced continuity scores varied in different groups of patients or different organizational settings. To allow for clustering by practice, the family practice was fitted as a random effect with maximum likelihood estimation.²³ Intraclass correlation coefficients were estimated from a random effects model with maximum likelihood estimation.

We compared continuity of care according to type of diabetes care received. In the United Kingdom, diabetes care has traditionally been provided by the diabetes outpatient clinic at the local hospital (referred to as hospital-based care). For the last 20 years, family practices have increasingly taken on primary responsibility for the care of patients with type 2 diabetes (family-practice–based care). In some instances, care is coordinated between the family practice and the hospital clinic with the latter providing annual reviews and specialist advice (shared care). We also compared continuity scores according to whether the practice had a designated clinician for diabetes care. In the United Kingdom, practices may assign a doctor or nurse the responsibility for organizing, coordinating, and delivering care for the practice’s diabetic patients.

Reliability

Potential ECC-DM scores ranged from 0 to 100 with an observed mean of 62.1 (SD 16.0). Table 1⇑ gives the wording for each item. The items were easy to read, with a Flesch-Kincaid grade level of 3.5. The average inter-item correlation based on 19 items was 0.343, and Cronbach’s α was 0.908. In the substudy of test-retest reliability, the mean difference in total score between repeat interviews in 26 patients was −0.37 (95% CI, −4.17 to 3.44). In the substudy to test the self-completion questionnaire format, the mean difference in total score for self-completion format as compared with interview format in 48 patients with complete scores was 4.0 (95% CI, 1.2–6.9), indicating a slightly positive difference in score for self-completion as compared with interview format. Most, 45 of 56 (80%), respondents to the self-completion questionnaire said that the questionnaire was “very easy” or “easy” to understand.

Factorial Composition

Table 1⇑ shows rotated factor loadings from a factor analysis with 4 factors. Items associated with relational continuity load strongly on the first factor. Items LC4 and FC4, which include the concept of a usual doctor or nurse, also load strongly on this factor. The second factor is associated with items representing team and cross-boundary continuity, the third with longitudinal continuity, and the fourth with flexible continuity. Item LC2 did not load strongly on any factor.

Table 2⇓ gives the overall mean (and standard deviation) score for the continuity-of-care scale and its subdomains, as well as values for average inter-item correlations, Cronbach’s α, and correlations with total scores. The high inter-item correlation for relational continuity was explained by each item having a 0 score if there was no usual clinician. The subdomains were strongly associated with each other. The highest correlation coefficient (0.656) was between relational and flexible continuity.

Criterion Validity

Table 3⇓ shows the variation in mean continuity scores among 19 different family practices. The mean number of patients per practice was 10 (SD 4.2, range 3–17). The mean total experienced continuity score varied from 46 to 78 at different practices. The intraclass correlation coefficient for the total score was 0.14 (95% CI, 0.04–0.32; P = .001). This finding provides evidence that patients from the same practice tend to rate their experience of continuity of care more similarly than patients registered with different practices. The same was true for the subscale scores for longitudinal, flexible, and relational continuity, but not for team and cross-boundary continuity, which may depend on experiences outside the practice.

Table 4⇓ shows the mean continuity-of-care scores by type of care setting. Patients who received diabetes care only from hospital-based specialty clinics gave lower mean scores for the total experienced continuity-of-care score and for each of the subscales except team and cross-boundary continuity. Patients from practices with a designated doctor for diabetes care gave higher continuity of care scores than patients registered with practices with no designated doctor for diabetes care. Adjusting for the type of diabetes care received by registered patients and whether the practice had a designated doctor for diabetes care explained most of the observed variation in continuity-of-care scores between practices. The ICC for total score adjusted for the variables shown in Table 4⇓ was 0.04 (95% CI, 0.00–0.25; P = .169). The continuity-of-care score was not consistently associated with age, sex, or duration of diabetes, and the associations in Table 4⇓ were robust to additional adjustment for these case-mix variables (estimated difference for type of care, −12.9; 95% CI, −18.4 to −7.3; estimated difference for designated diabetes doctor, 7.4; 95% CI, 2.0–12.7).

Limitations of Study

The experienced continuity-of-care measure has generally satisfactory psychometric properties, but some issues require clarification. Although all of the items were supported by our qualitative data, in a short scale not all relevant concepts can be included. Some of our selections may be justified with reference to other measures, because items with similar content may be found in current patient satisfaction surveys.^21,22 Unlike these earlier measures, the ECC-DM is grounded in a conceptual model that identifies experienced continuity of care as a construct of patient satisfaction,²⁸ and marks a shift in thinking away from the earlier view of continuity of care as a process-of-care measure.²⁹ Eight of the items referred to the concept of a “usual doctor or nurse.” This concept was further defined as “the doctor or nurse who knows you and your diabetes best.” Whereas the dependence between these items might be considered an undesirable statistical property of the measure, the importance attached to the concept of a usual doctor or nurse both in the literature on continuity of care^30,31 and in our qualitative data make this feature an important part of the measure. Item LC2 concerning appointment letters did not load strongly on any of the domains and might be a candidate for omission, but this item was justified by systematic review evidence that active processes of recall facilitate better care.³² Because items LC4 and FC4, which concern the concept of the “usual doctor or nurse,” load the factor associated with relational continuity, it may not be entirely justified to use the subscales separately, and the overall scale should generally be preferred. The team and cross-boundary continuity items appeared to have satisfactory psychometric properties, but assessment of team and cross-boundary continuity did not vary between practices or between types of care. This finding may suggest that team and cross-boundary continuity does not vary according to these variables, but equally it is possible that the questions relating to team and cross-boundary continuity lack discrimination, because patients are not well able to judge these aspects of their care. These issues require further consideration in the future development of the measure.

A limitation of this study is the low response rate achieved in the interview survey. This low response rate is a consistent finding in inner-city areas in many countries. Because our aim was to develop a reliable and valid tool, however, representativeness was less important to our study than diversity of experience. There was evidence that patients with less-favorable experiences of continuity were less likely to respond to the survey, which might lead to our estimates being too conservative for effects of different organizational arrangements. It may be noted that in the main study, patients completed the questionnaire by interview, and the response rate was not an indicator of the ease of completion of the measure.

Comparison With Other Work

Dolovich et al recently described a different diabetes-specific questionnaire measure of continuity of care.³³ Their approach differed from ours because the measure did not refer to an underlying conceptual model of continuity of care.³⁴ Our results using the new measure show that diabetic patients’ experiences of continuity of care vary systematically among health care organizations. This observation requires further study, but it was clear that some organizational characteristics are associated with less-favorable experiences of continuity of care. Hospital-based diabetes services, where there are more complex systems of care and higher staff turnover, are generally associated with lower continuity of care. In primary care, the identification of a designated physician for diabetes care was associated with higher experienced continuity of care. This finding is consistent with systematic review evidence that case manager roles may enhance the delivery of care in chronic illness.⁹ As a construct of patient satisfaction, continuity of care may be valued in itself,²⁸ and interventions that promote continuity are to be encouraged. Further research is required to find out whether more-favorable experiences of continuity of care are associated with better treatment or improved treatment outcomes.

We have provided evidence for the reliability, construct validity, and criterion validity of a new measure of experienced continuity of care. The measure is brief, quick to complete, acceptable to patients, applicable for use in different care settings, and therefore suitable for use in the routine monitoring of quality of care. The measure may be used in ambulatory care in both specialist and family practice settings to provide information concerning factors that enhance or impede patients’ experiences of continuity of care and the relationship between continuity, processes of care, and health outcomes.

The measure incorporates experiences that are widely applicable, and it may be used without modification. We acknowledge that further development of the measure is desirable to address some of the questions raised in this study. Further work to evaluate the self-completion version is desirable. In addition, modifications may be required for different health systems, as when barriers to access may impede continuity of care for example. The measure also has potential for adaptation and use in other chronic illnesses.