Abstract
PURPOSE Efforts to better understand the impact of clinic member relationships on care quality in primary care clinics have been limited by the absence of a validated instrument to assess these relationships. The purpose of this study was to develop and validate a scale assessing relationships within primary care clinics.
METHODS The Work Relationships Scale (WRS) was developed and administered as part of a survey of learning and relationships among 17 Department of Veterans Affairs (VA) primary care clinics. A Rasch partial-credit model and principal components analysis were used to evaluate item performance, select the final items for inclusion, and establish unidimensionality for the WRS. The WRS was then validated against semistructured clinic member interviews and VA Survey of Healthcare Experiences of Patients (SHEP) data.
RESULTS Four hundred fifty-seven clinicians and staff completed the clinic survey, and 247 participated in semistructured interviews. WRS scores were significantly associated with clinic-level reporting for 2 SHEP variables: overall rating of personal doctor/nurse (r2 =0.43, P <.01) and overall rating of health care (r2= 0.25, P <.05). Interview data describing relationship characteristics were consistent with variability in WRS scores across low-scoring and high-scoring clinics.
CONCLUSIONS The WRS shows promising validity as a measure assessing the quality of relationships in primary care settings; moreover, primary care clinics with lower WRS scores received poorer patient quality ratings for both individual clinicians and overall health care. Relationships play an important role in shaping care delivery and should be assessed as part of efforts to improve patient care within primary care settings.
INTRODUCTION
During the past several decades, primary care providers in the United States have faced increasing challenges, including rising health care costs and growing rates of chronic disease among patients that require escalating time and expertise to manage.1,2 A variety of strategies have been proposed to aid primary care providers, including redistributing the management of patients’ preventive and chronic illness care across interdisciplinary teams of physicians, physician assistants, and nurses—for example, as in the patient-centered medical home.3,4 Because communication and teamwork are critical to the success of such teams, achieving high-quality relationships among clinicians and staff is an essential component of improving primary care delivery.5,6
A number of researchers have proposed theoretical models of how high-quality relationships within primary care settings may contribute to improved quality of care. In a review of related research, Lanham et al7 identified 7 characteristics of work relationships in settings with high-quality practice outcomes: (1) trust, or feeling comfortable making oneself vulnerable to others; (2) mindfulness, or demonstrating openness to new ideas and multiple perspectives; (3) heedful interrelating, or individuals being attentive to how their roles and actions affect and intersect with those around them; (4) respectful interaction, or engaging in honest, confident, and appreciative relations between coworkers; (5) diversity, or having sufficient diversity to support creative problem solving; (6) social- and task-related communication; and (7) rich, or face-to-face, communication.
Lanham et al also proposed a model of how these relationship characteristics may influence practice outcomes by affecting how clinicians and staff engage in reflection and learning. Safran and colleagues suggest that these characteristics form the basis for organizational cycles of reflection and action through which a “culture of continual learning” is created.8(pS13) More recently, Miller et al have argued for the importance of these relationship characteristics in determining a practice’s adaptive reserve, ie, its effective resilience during periods of stress or change.9 Despite the value of such work, it has remained difficult to quantify the influence of primary care practice relationships on quality of care without a validated measure of relationships that allows for more direct comparison across sites.
The goal of the current study, therefore, was to develop a scale to assess staff and clinician relationships within primary care settings, and to conduct a preliminary validation of the scale using quantitative and qualitative methods. Because the current study was conducted in Department of Veterans Affairs (VA) primary care clinics, it also provided an opportunity to examine whether characteristics of relationship quality initially defined in non-VA settings remain meaningful in the VA, which has invested heavily in the effort to provide high-quality primary care in recent years.10,11 Little research has examined relationships within VA primary care settings, although successful implementation of clinical care practices in the VA context is thought to be at least partially dependent on work unit dynamics.12
Building on previous work in this area, we developed and validated a Work Relationships Scale (WRS) using both quantitative and qualitative methods.
METHODS
Study Design
We drew upon Lanham et al’s 7 characteristics to develop a set of 19 items assessing the perceived quality of work relationships7; in some cases items were developed in collaboration with other researchers working across multiple studies and adapted for use in the VA.13,14 All items were tested for face validity in a pilot project of 5 primary care clinics and refined based on insights gained. The resulting WRS was administered as part of a mixed methods survey of learning and relationships conducted among 17 VA primary care clinics in South and Central Texas. Questionnaires were distributed to all clinicians and staff of each clinic.
Information on patient experiences within these clinics was drawn from the Survey of Healthcare Experiences of Patients (SHEP), an ongoing initiative of the VA’s Office of Quality and Performance.15 The SHEP collects regular input from a stratified random sample of VA patients who have received inpatient or outpatient care in the preceding month. We selected 4 SHEP variables from fiscal years 2010 to 2011 for the purpose of assessing overall clinic function and patient satisfaction during the study period. “Getting care quickly” assesses whether patients felt care was provided “as soon as you thought you needed” for urgent and regular care concerns in the previous 12 months; reported percentages reflect responses of either usually or always. “Overall rating of personal doctor/nurse” and “overall rating of health care” are scored from 0 to 10, where a rating of 0 indicates worst possible and 10 indicates best possible; reporting reflects the percentage of responses that endorse a rating of 9 or 10. “Clinician wait time” describes how long the patient waited before being seen; reporting reflects the percentage of responses indicating wait times of 20 minutes or less. Data for all variables were aggregated at the clinic level.
Study team members also observed clinic members interacting within each of the clinics for 3- to 5-day periods between January 2009 and September 2011. Key informants representing all clinic roles were selected based on these observations and asked to participate in semistructured individual interviews guided by open-ended questions regarding relationships within the clinic (Supplemental Appendix, available at www.annfammed.org/content/11/6/543/suppl/DC1).
All study procedures were approved by the appropriate institutional review boards.
Data Analysis
Descriptive statistics and classical item analysis statistics were computed using procedures in SAS 9.2 (SAS Institute). Rasch item analysis and principal components analyses were conducted. A partial-credit Rasch model was used to evaluate the original 19 items. The final subset of items to include in the scale were selected based on item-response convergence with respect to the underlying latent trait, the in-fit and out-fit statistics, and established ordinality of the responses. Unidimensionality was established by the Martin-Löf test and the results from a principal components analysis on the person-item residuals. SAS PROC Mixed was used to evaluate clinic differences in WRS scores. K-nearest neighbor analysis was used to impute missing data where 2 or fewer responses to the WRS were missing. Class statement was included to account for clustering of patient and respondent data within clinics. Spearman rank-order correlations between SHEP variables and mean WRS scores were used to assess the relationship between WRS relationship scores and patient perceptions of care quality.
As an additional means of assessing construct validity, we used mean clinic WRS scores to identify the 3 highest- and 3 lowest-scoring clinics and examined whether interview data revealed variation in clinic work relationships that was congruent with variability in scale findings.16 All interviews were transcribed and uploaded into the qualitative software program NVivo 8 (QSR International). We then conducted content analysis of all interview transcripts using a codebook developed by the research team and based upon Lanham et al’s 7 relationship characteristics. All members of the research team participated in coding, with at least 2 coders reviewing each transcript, and all transcripts were compared for agreement. For this analysis, we focused on clinic members’ descriptions of staff and clinician interactions and the presence or absence of Lanham et al’s relationship characteristics, noting how particular characteristics (eg, trust) clustered across responses from members of high- and low-scoring clinics. Case studies were prepared by the first author and then reviewed and refined by the rest of the research team.
RESULTS
Four hundred fifty-seven clinicians and staff from the 17 VA primary care clinics completed the clinic member questionnaires (overall response rate = 64.7%, range across clinics 29%–100%). Clinic member respondents were 70.2% male, with a mean age of 48.8 years (SD = 10.4), and represented all clinic roles: administrative officers (n = 13), front office staff (n = 72), medical directors (n = 14), physicians (n = 69), nurse practitioners and physician assistants (n = 12), and nursing staff (registered nurses and licensed vocational nurses; n = 169). Remaining individuals were from a range of specialized roles, such as dietician, psychologist, or social worker.
WRS Development: Item Evaluation
Using WRS responses to questionnaires collected as part of the clinic member survey, we fit a partial-credit Rasch model to item responses. Initial analysis of the degree of fit between scale items and the scale’s latent trait showed that 15 of the 19 original items had an in-fit and out-fit mean square ranging between 0.5–1.5, indicating they had good fit to the model and were productive for measurement (Supplemental Table, available at www.annfammed.org/content/11/6/543/suppl/DC1). The 4 items that fell outside this range were excluded. The original scale was coded on a 6-point scale ranging from 1 (strongly disagree) to 6 (strongly agree); however, inspection of the person-item map (Supplemental Figure 1, available at www.annfammed.org/content/11/6/543/suppl/DC1) showed disordered thresholds between the middle 2 categories (3 = slightly disagree and 4 = slightly agree) on a number of items. We therefore collapsed these categories to create a 5-point scale with a middle neutral response option. The likelihood ratio statistic for the Martin-Löf test supported the unidimensionality of the WRS. Principal component analysis on the person-item residuals showed little clear residual structure in the item responses. Item scores were summed to produce a total score ranging from 15 to 90. Higher scores suggest higher quality relationships. The 15-item WRS (Table 1) was found to have excellent internal reliability (Cronbach’s α = .95), and inter-item correlations ranging from 0.29 to 0.80. Figure 1 displays the person-item map for the 15-item WRS.
Clinic Variation in WRS Scores
Clinic WRS scores are displayed in Table 2. Significant differences in WRS scores emerged between clinics (F value = 7.14, P <.0001). There was considerable variation in the number of patients enrolled in each clinic (mean = 7,630, range = 2,634–21,621) and the number of staff in each clinic (mean = 43.8, range = 17–92), but clinic WRS scores were not associated with either patient enrollment (r2 = −0.39, P = .12) or number of staff (r2 = −0.31, P =.22).
WRS Scores and Patient-Assessed Quality
We also assessed patient satisfaction with clinic-level quality of care, using the SHEP variables “getting care quickly” (mean = 70.4, SD = 7.8), “overall rating of personal doctor/nurse” (mean = 50.1, SD = 7.8), “overall rating of health care” (mean = 63.2, SD = 8.4), and “clinician wait time” (mean = 69.8, SD = 11.8). Two variables were significantly associated with mean WRS score: “overall rating of personal doctor/nurse” (r2= 0.53, P <.05) and “overall rating of health care” (r2= 0.50, P <.05; Table 3). Relationships between mean WRS score and the 2 remaining variables—”getting care quickly” (r2= 0.19, P= .50) and “clinician wait time” (r2= 0.08, P= .78)—were not significant.
Relationships in High- and Low-Scoring Clinics
Two hundred forty-seven clinic members representing all 17 clinics also participated in semistructured interviews. For this analysis, we examined all interviews from the 6 clinics that scored highest and lowest on the WRS. Five primary relationship characteristics emerged as consistently present in clinics with higher WRS scores: (1) rich or face-to-face communication, particularly during problem solving and conflict resolution; (2) heedful interrelating; (3) trust; (4) respectful interaction; and (5) mindfulness, particularly across clinic roles. In contrast, low-scoring clinics showed a marked lack of these characteristics. Table 4 provides quotes illustrating the presence or absence of these characteristics in high- and low-scoring clinics. Of the 2 remaining relationship characteristics, we found that diversity overlapped in this context with mindfulness and concluded that diversity did not improve our ability to characterize clinic-level relationship quality. We also found that clinic members across all clinics described using both social- and task-related communication. As comparable use of both types of communication was described across high- and low-scoring clinics, we determined that these communication styles may not function well as markers of relationship quality.
Considerable differences emerged in patterns of communication and relating between low- and high-scoring clinics. Low-scoring clinics were more likely to make use of e-mail and other lean communication methods to transmit patient and clinic-related information. Clinicians and staff in these clinics often described good working relationships with those they work with most closely, such as those within their discipline (eg, other nurses) or those on their team (as in a physician-nurse pairing), while expressing reservations about relationships with individuals in more distant clinic roles or with upper management, which at times appeared to result in conflict and inefficiency. Clinicians and staff were less likely to feel comfortable raising potential concerns and sometimes felt as though workloads were not fairly distributed. Clinic members in low-scoring clinics often described feeling as though their concerns, when raised, went unheard and unaddressed.
High-scoring clinics, by contrast, tended to augment lean communication strategies with face-to-face communication to convey information, resolve conflict, and engage in problem solving. Those in high-scoring clinics described more heedful interrelating, with a strong sense of teamwork and shared purpose, as well as greater mutual helpfulness and task coordination among diverse clinic members. Trust and respectful interaction were more frequently observed in these clinics, where staff members described feeling comfortable asking questions and expressing opinions and were notably appreciative of the skills and hard work of their colleagues. Members of high-scoring clinics also described an environment in which concerns were heard and acknowledged, reflecting greater mindfulness.
DISCUSSION
We describe the development and initial validation of a scale to assess the quality of staff and clinician relationships within primary care clinics, moving toward the goal of supporting future research on the role of clinic member relationships in shaping quality of care. The triangulation of quantitative and qualitative data in this study provides insight into those characteristics associated with high- and low-quality relationships in primary care clinics and also provides the first description of such relationships in VA settings.
The 15-item Work Relationships Scale (WRS) was developed based on qualitative research examining relationships in high-functioning non-VA primary care clinics.8,17 Item content was based on 7 characteristics of work relationships identified in prior literature.7 Rasch analysis of the person-item responses and the principal components analysis on the person-item residuals show the WRS functions as a unidimensional measure of the overall quality of working relationships among team members in VA primary care clinics. This result, in turn, suggests that the characteristics of work relationships are tightly interrelated in these clinics. The WRS has excellent internal reliability and identified significant variation in relationship quality across the 17 primary care clinics examined in this study.
To assess whether the WRS was related to clinic-level patient outcomes, as would be predicted based on existing conceptual models, we also examined the association between mean WRS and patient experiences of care for each clinic. “Overall rating of health care,” which reflects patients’ perceptions of their care as a whole, and “overall rating of personal doctor/nurse,” which reflects patients’ perception of individual clinicians, were both significantly associated with WRS scores. In addition to providing evidence to support the convergent validity of the WRS, this finding also suggests that relationship quality within clinics may influence the patient’s experience of health care. Patients’ perceptions of health care are complex, and patients may not always be in a position to evaluate the technical competence of their care.18,19 Nonetheless, patients’ perceptions of health care are a valuable indicator of the quality of care delivery, offering insight into patient-clinician communication and the ability of clinicians to meet patients’ health care needs.20,21 These findings are consistent with conceptual models hypothesizing that relationship characteristics may have a downstream impact on practice outcomes7–9 and more broadly with the recognition that organizational factors, such as climate and culture, can affect care delivery.8,22 In describing the micro-relations within work units, relationship quality may make a unique contribution to understandings of organizational context, with relevance for quality improvement and the implementation of evidence-based practices.12 Further research in this area will be required.
Because both “getting care quickly” and “clinician wait time” require coordination of care across clinic clinicians and staff, and existing conceptual models predict that coordination of care should benefit from relationship quality, we also anticipated that mean clinic scores for each of these variables would be associated with clinic WRS scores. This did not prove to be the case. It may be that the ability to accommodate patient appointment needs and see patients in a timely fashion are affected by multiple factors—eg, the ratio of clinic staff to clinicians—that confound any impact of relationship quality in the current analysis.
Variability in WRS scores occurred in a predictable way alongside the presence (or absence) of relationship characteristics in the coded interview data. For example, clinicians and staff of high-scoring clinics were more likely than those of low-scoring clinics to describe viewing the clinic as a team and sharing tasks on a flexible basis. This heedfulness appeared to build morale and a sense of shared priorities among team members.
This study had a number of limitations, including a moderate response rate and relatively small sample of clinics from a single state. Nonetheless, both qualitative and quantitative data support the conclusion that the WRS has good construct and scale validity. Future research should examine the differential and predictive validity of the WRS and establish its ability to assess relationship quality in a variety of VA and non-VA clinical care settings.
This study is one of the first to show that relationships within a care organization affect patient satisfaction. Clinic member relationships appear to have a significant impact on patient perceptions of care and should be assessed as part of efforts to improve delivery.
Acknowledgments
Drs Benjamin Crabtree, Kurt Stange, Paul Nutting, William Miller, Reuben McDaniel, and Michelle Jordan contributed valuable work in developing the construct of work relationships in the primary care setting, and in developing some of the items from which the Work Relationships Scale items were adapted.
Footnotes
-
Conflicts of interest: authors report none.
-
Funding support: This work was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (HSR&D) project no. IIR 06-063. Dr. Finley is an investigator with the Implementation Research Institute, George Warren Brown School of Social Work, Washington University, St. Louis; through an award from the National Institute of Mental Health (R25 MH080916-01A2) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).
-
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
- Received for publication April 2, 2012.
- Revision received January 30, 2013.
- Accepted for publication February 18, 2013.
- © 2013 Annals of Family Medicine, Inc.