Background
Primary health care is the pillar of the health care system, with interprofessional collaborative teams being increasingly considered a key tenet and an important means for providing quality primary health care services (Starfield et al., Reference Starfield, Shi and Macinko2005; Samuelson et al., Reference Samuelson, Tedeschi, Aarendonk, de la Cuesta and Groenewegen2012; Pullon et al., Reference Pullon, Morgan, Macdonald, McKinlay and Gray2016. While interprofessional collaboration is recognized as an essential aspect of care for patients with complex and chronic conditions (Reeves et al., Reference Reeves, Goldman, Gilbert, Tepper, Silver, Suter and Zwarenstein2011; Pullon et al., Reference Pullon, Morgan, Macdonald, McKinlay and Gray2016), the evidence remains in its infancy. For these reasons, policy makers in Canada and elsewhere have continued to call for greater integration of interprofessional team-based primary health care and the development of indicators that specifically recognize the value of this added lens (Hutchison et al., Reference Hutchison, Levesque, Strumpf and Coyle2011; Ministry of Health and Long-Term Care, 2015).
Although Reeves et al. (Reference Reeves, Goldman, Gilbert, Tepper, Silver, Suter and Zwarenstein2011) demonstrate the variations that exist in conceptualizing interprofessional collaboration in primary health care, it can be understood as the integration and cooperation of different health care professionals that complement one another’s competencies, practices, and skills, thus resulting in optimal use of health care resources (Supper et al., Reference Supper, Catala, Lustman, Chemla, Bourgueil and Letrilliart2014. In primary health care, this means that family physicians and/or nurse practitioners come together with other health care providers such as social workers, occupational therapists, dietitians, pharmacists, and others, in order to provide a broad range of services and programs (Ambrose-Miller and Ashcroft, Reference Ambrose-Miller and Ashcroft2016). Approximately 40% of Canadians have access to interprofessional collaborative teams and individuals with multiple chronic conditions are most likely to receive team-based primary health care (Zygmunt and Berge, Reference Zygmunt and Berge2014). Internationally there is also a strong move to team-based primary health care, often referred to as medical homes, to enhance integration of services and emphasize health promotion and chronic disease management (Hutchison et al., Reference Hutchison, Levesque, Strumpf and Coyle2011; Naccarella et al., Reference Naccarella, Greenstock and Brooks2013). The evidence to date suggests that interprofessional collaboration contributes to the structures and processes of primary health care, including patient satisfaction and access to care (Donabedian, Reference Donabedian2005; Hogg et al., Reference Hogg, Rowan, Russell, Geneau and Muldoon2008), however, limited knowledge exists about the impact that interprofessional collaborative teams have on patient health outcomes (Paradis and Whitehead, Reference Paradis and Whitehead2018).
Despite the surge of team-based models across Canada, there is relatively little evidence as to whether or not the intended outcomes of interprofessional collaborative teams have been achieved (Spenceley et al., Reference Spenceley, Andres, Pains, Wedel, Gelber and Halma2013; Ashcroft, Reference Ashcroft2014; Glazier et al., Reference Glazier, Hutchison and Kopp2015; Zygmunt and Berge, Reference Zygmunt and Berge2014). Performance measurement is an important part of understanding and improving primary health care, yet the challenge remains as to how best to measure the value of interprofessional collaborative teams. Little knowledge exists about indicators that can help demonstrate the impact on quality of care outcomes for patients receiving interprofessional collaborative primary health care (Thannhauser et al., Reference Thannhauser, Russell-Mayhew and Scott2010). Furthermore, there is some evidence that performance measures being used in primary health care settings may even be inappropriate for interprofessional teams, resulting in unintended consequences (Ashcroft, Reference Ashcroft2014). One of the reasons may be the complexity of practice contexts, variations of the types of providers that might be involved in collaborative care practices, and the broad range of programs and care associated with primary health care (Levitt et al., Reference Levitt, Nair, Dolovich, Price and Hilts2014). Instead, indicators largely focus on organizational structures, collaborative processes, and patient experiences (Levitt et al., Reference Levitt, Nair, Dolovich, Price and Hilts2014; Samuelson et al., Reference Samuelson, Tedeschi, Aarendonk, de la Cuesta and Groenewegen2012). Ontario, Canada, is one province that has substantially invested in integrating interprofessional collaboration in primary health care; however, performance indicators used to determine impact of these team practices are largely based on physician-focused data (Jaakkimainen et al., Reference Jaakkimainen, Barnsley, Klein-Geltink, Kopp and Glazier2011) with limited to no inclusion of data representative of other interprofessional health providers involved in patient care. Not only is there a gap in knowledge about the impact of these newer interprofessional teams, applying indicators developed for a solo-provider primary care context may be harmful to successful interprofessional collaboration (Ashcroft, Reference Ashcroft2014).
The purpose of our study was to (1) describe current practices that support interprofessional primary health care and (2) identify current performance indicators measuring the impact of collaboration in interprofessional primary health care teams from an interprofessional health provider perspective.
Methods
This study was part of a larger developmental evaluation being completed with interprofessional primary health care teams in the province of Ontario, Canada. Developmental evaluations are an approach that is ideal for contexts that are dynamic and can support the ongoing evolution of programs (Patton, Reference Patton2010). The evaluation has included multiple strategies to engage stakeholders for the purposes of development and improvement. For this study, we are reporting on a stakeholder consultation meeting, which is an exploratory descriptive component of the larger developmental evaluation (Miles et al., Reference Miles, Huberman and Saldanña2014). ‘Stakeholders’ are defined as those with a vested interest in an activity and its outcome (Leviton and Melichar, Reference Leviton and Melichar2016). The stakeholder consultation meetings that we are reporting on in this study took place immediately before the commencement of the 2016 annual meeting of a provincial association of primary health care providers in Ontario.
Facilitated stakeholder meetings are an effective method to engage health care providers on a range of topics related to the structures and processes of health care (Leviton and Melichar, Reference Leviton and Melichar2016; Mercer et al., Reference Mercer, Smith, Wyke, O’Dowd and Watt2009; Protheroe et al., Reference Protheroe, Wallace, Rowlands and DeVoe2009; Hand et al., Reference Hand, Letts and von Zweck2011). Participants invited to attend the stakeholder consultation meetings were interprofessional health providers employed in primary health care teams in Ontario, Canada, and who were members of the professional association holding the annual meeting. Fourteen professional groups were represented including, but not limited to, nurses, social workers, pharmacists, physical therapists, occupational therapists, quality improvement specialists, and administrators. Ethics approval was received from the local University Office of Research Ethics (Protocol #31773).
Data collection
Data collection was completed in small groups composed of participants from a similar professional background. Participants were seated with their designated professional group and were provided with five structured questions that aimed to elicit information on outcomes and indicators used to measure the value of interprofessional care. Nominated leads representing each of the 14 interprofessional health providers’ professional groups helped collaboratively develop the five questions in advance of the stakeholder meeting. The development of the questions was guided by the literature on interprofessional collaboration and team-based primary health care (Gocan et al., Reference Gocan, Laplante and Woodend2014), yet were also developed in a way that was relevant for the stakeholders and their context. Through teleconference meetings, the nominated leads and the stakeholder meeting facilitators achieved consensus on the questions that were used to explore stakeholders’ perspective on the current practices that support interprofessional primary health care, and performance indicators that can measure the impact of collaboration in interprofessional primary health care teams.
The stakeholder meetings were led by a team of eight experienced facilitators and researchers in the areas of interprofessional care that included the five co-authors and three additional research assistants. Due to the number of participants, six stakeholder meetings occurred simultaneously in separate rooms. Participants completed the workshops independently and came together for a final summary discussion.
Each room was set up so that participants were sitting at round tables with approximately 8–10 individuals per table. Each stakeholder meeting facilitator had a designated script that they read out-loud to begin the session, the set of five questions to guide small group discussion, clearly defined objectives to ensure consistency in the facilitation across the different professional groups, and worksheets to guide the discussion and documentation. Each stakeholder meeting was three hours in duration and ran from 09:00am to 12:00pm. The first half of the workshop sought to identify outcomes, assessment, and/or indicators currently being used in their interprofessional teams. Questions were introduced sequentially by the facilitator, followed by time for the participants to discuss each question within the smaller tables (see Appendix A). Participants documented their discussion on worksheets and larger flip sheet paper provided to each table by the facilitators. Following each question, there was a brief opportunity for groups to present the results of their discussion back to the larger group. The facilitators kept notes of the large group discussion. All artifacts were maintained for analysis.
The terms outcomes, assessment, and indicators were used in order to be as inclusive as possible to all ways in which teams describe approaches to measuring and demonstrating the value of team-based primary health care. All professions participated in this component of the workshop. The second half of the workshop focused on future possibilities of outcomes and indicators that participants might recommend as an option to demonstrate the value of interprofessional teams on patient outcomes. Two professional groups (nursing and pharmacy) did not participate in the second half due to a pre-scheduled education session, thus, opting out of participating in the final discussion about recommendations.
Data analysis
Given that a developmental evaluation by design allows for methodological flexibility (Patton, Reference Patton2010), the primary investigators employed both an initial conventional content analysis approach to identify key themes from the data, as well as engaged in a subsequent rank order of the frequency of codes to help analyze the results (Hsieh and Shannon, Reference Hsieh and Shannon2005). The combination of both identifying key themes, followed by a frequency count, intends to help elucidate both the descriptive thematic categories identified in data, and strives to demonstrate the importance of these categories across diverse professional perspectives. This approach has been effectively employed by other researchers to analyze data from stakeholder meetings (Hand et al., Reference Hand, Letts and von Zweck2011; Mercer et al., Reference Mercer, Smith, Wyke, O’Dowd and Watt2009; Protheroe et al., Reference Protheroe, Wallace, Rowlands and DeVoe2009).
Beginning with the first question, two of the authors (A.M., N.B.) read the responses from all the professions and developed a set of preliminary codes. The authors (A.M., N.B.) then completed a second reading and independently applied the preliminary codes to the data. They (A.M., N.B.) reviewed the applied codes for agreement, and a third author (C.D.) was consulted when consensus was unable to be attained. As data analysis progressed additional codes emerged, and the initial codes were revised, and some codes were collapsed. For example, a code-entitled communication was created to include both formal (meetings) and informal communication (hallway consults). This process was systematically applied to each of the five questions for all professions. A final review of all the generated codes was completed to ensure clarity and consistency. The codes were then organized into tables to capture the frequency of codes across professions for all five questions.
Results
A total of 283 participants from 14 professions participated including nursing, administrators, social work, occupational therapy, physical therapy, data management specialists, dietitians, pharmacists, chiropodists, health promoters, nurse practitioners, physician assistants, respiratory therapists, and chiropractors. Twelve unique characteristics were identified that were seen to support interprofessional collaboration in the primary care teams. The top three characteristics (total number of response = 628) were communication (n = 146), co-treatment (n = 112), and patient-based conferences (n = 81) (for details refer Table 1). The top three outcomes, assessments, and/or indicators that were identified as being used to demonstrate the value of interprofessional collaboration (total n = 241) were patient experience (n = 71), patient health status (n = 35), and intra-agency team referrals (n = 30). Quality improvement was identified as an outcome, assessment, and/or indicator, by a total of 57 participants, but was removed from final analysis as it was felt that quality improvement represents a process in which teams can demonstrate value, rather than a measure itself (for complete details, refer Table 2). The top three performance indicators demonstrating the value of individual providers’ professional contribution within the team (as opposed to collaborative interprofessional contribution) (total n = 189) were workload measurement (n = 43), patient experience (n = 40) and patient health status (n = 34) (for details, refer Table 3). Quality improvement was identified by a total of 76 participants but was again removed from final tally in this category as it was felt to represent a process versus an outcome indicator.
Table 1. Characteristics that support collaboration in primary care teams
Table 2. Measures, outcomes and indicators reported currently being used to demonstrate the value of interprofessional collaboration in primary care teams
Table 3. Measures, outcomes, and indicators reported currently being used to demonstrate the value of Individual Professional contribution within the team
When participants were asked to identify potential performance-specific measures at the level of the individual patient, the team and broader population in order to demonstrate the value of their professions contribution they identified a total of 14 indicators. The top three responses are presented in Figure 1. Teams were also asked to identify potential performance-specific measures at the level of the individual patient, the team and the population level to demonstrate the value of collaboration within interprofessional primary health care teams. A total of 14 unique outcomes, assessments, and/or indictors were identified, and the top three responses are presented in Figure 2.
Figure 1. Profession specific future measures of interprofessional collaboration at the population, team and patient encounter level
Figure 2. General measures of future interprofessional collaboration at the population, team, and patient encounter level
Discussion
This study provides a unique contribution to the literature on performance measurement of interprofessional primary health care teams by providing the provider perspective on both the existing indicators being used, as well as recommendations for alternative indicators that might better capture collaborative contributions and impact on patient outcomes. The development of performance measurement in primary health care is traditionally completed using top-down approaches such as expert panels (Canadian Institute of Health Information, 2016), literature reviews (Langton et al., Reference Langton, Wong, Johnston, Abelson, Ammi, Burge, Campbell, Haggerty, Hogg, Wodchis and Mcgrail2016), and arm’s-length organizations (Health Quality Ontario, 2016). Top–down approaches to performance measurement are particularly challenging for interprofessional primary health care team members who feel distanced from indicators that are largely biomedical in nature and do not cover the comprehensive attributes and services by non-medical team members (Wong et al., Reference Wong, Browne, Varcoe, Lavoie, Smye, Godwin, Littlejohn and Tu2011; Ashcroft, Reference Ashcroft2014; Johnston et al., Reference Johnston, Green, Thille, Savage, Roberts, Russell and Hogg2011). There is also evidence to demonstrate that when primary health care teams identify their own performance measures and are able to receive direct feedback on these measures, they are likely to integrate this feedback to adjust practice behaviors (Donnelly et al., Reference Donnelly, Shulha, Klinger and Letts2016). Examples such as this reinforce the need to engage interprofessional health providers in determining meaningful and actionable indicators to demonstrate the value of team-based primary health care. Work is being conducted by the Association of Family Health Teams of Ontario, an organization that represents interprofessional primary health care teams in the province of Ontario, Canada, to engage interprofessional primary health care providers in the identification of performance measures for primary health care teams (Association of Family Health Teams of Ontario, 2017).
Characteristics of collaboration
In our study, participants identified the characteristics of collaboration they experienced within their interprofessional primary health care teams. These elements were consistent with what has been previously reported in both team-based primary health care (Donnelly et al., Reference Donnelly, Brenchley, Crawford and Letts2013; Gocan et al., Reference Gocan, Laplante and Woodend2014; Morgan et al., Reference Morgan, Pullon and McKinlay2015; Saint-Pierre et al., Reference Saint-Pierre, Herskovic and Sepúlveda2018) and the broader literature on collaborative practice (Reeves et al., Reference Reeves, Pelone, Harrison, Goldman and Zwarenstein2017). The findings of this study add to this growing body of literature on collaborative practice that has clearly identified the importance of team processes (e.g. team meetings), organizational supports (e.g. electronic medical records), and environmental structures (e.g. co-location) in supporting interprofessional primary health care teams.
Performance indicators
Although both the literature and the results identify processes known to support teams, the actual extent to which these processes are enacted is less clear. There is also relatively little research that has examined the impact of the elements of collaboration on patient outcomes (Brandt et al., Reference Brandt, Lutfiyya, King and Chioreso2014; Morgan et al., Reference Morgan, Pullon and McKinlay2015; Reeves et al., Reference Reeves, Pelone, Harrison, Goldman and Zwarenstein2017), despite a growing focus on the importance of a team-based approach to managing complex and chronic conditions. Importantly, of the top 10 assessments, outcomes, and/or indicators identified in this study, seven would be considered process indicators (e.g. communication, co-visits, referrals) supporting collaboration. Both this professional readiness and the literature strongly point to the need to include a characteristic(s) of collaboration (e.g. number of team meetings, percentage of the team with access to a common electronic medical record, number of team members who are co-located) as a performance indicator(s) for team-based primary health care. This addition would not only provide a raw team function score, it would also offer insights into the relationship between collaborative processes and patient outcomes.
In Canada, the Canadian Institute of Health Information (CIHI) has developed the Pan-Canadian Primary Health Care Indicators, with a total of 51 indicators, of which 27 indicators are for primary health care policy makers and 24 indicators have been identified for use by practitioners (Canadian Institute for Health Information, 2016). Table 4 provides an overview of the 24 indicators juxtaposed with the outcome/measures/indicators identified in this study. Of note is the heavy focus on coordination outcomes found in this study. One of the CIHI domains, coordination, does in fact address team collaboration; however, there have been no published results related to this indicator, despite data for other key indicators. This lack of data highlights the challenges of collecting data on team processes. Given that ‘coordination’ is one of the few indicators to which non-physician team members directly contribute, both researchers and policy makers need to attend to how a process indicator such as team coordination might be measured consistently.
Table 4. Canadian Institute of Health Information (CIHI): Pan Canadian Primary Health Care priority indicators for providers compared to study results
CIHI = the Canadian Institute of Health Information.
Although collaborative processes are important to consider from a performance indicator perspective, fundamentally team-based primary health care is a model of primary health care designed to expand the traditional biomedical approach by bringing in multiple perspectives to assist in addressing the broader determinants of health. However, the current indicators and mechanisms of assessing the quality of care in teams remain narrow in scope and focus, primarily on individual behaviors closely aligned with a biomedical model of care (e.g. medication review, diabetes complications) (Health Quality Ontario, 2015) and have yet to catch up with this shift to comprehensive, integrated primary health care services.
It must be recognized that team-based models are only a small subset of primary health care services (Glazier et al., Reference Glazier, Hutchison and Kopp2015), but determining appropriate indicators will support quality care and help ensure those individuals who would most benefit from this model of care receive the appropriate services. The World Health Organization’s International Classification of Functioning (WHO-ICF) (World Health Organization, 2013) might offer a framework to support future work in measuring and evaluating interprofessional primary health care. The WHO-ICF is an internationally recognized perspective that recognizes the role of personal, social and contextual factors in optimizing function and health (Dufour and Lucy, Reference Dufour and Lucy2010). It is an ideal framework in which to situate interprofessional collaboration and would assist in broadening the approach to measurement in primary health care in Canada. The WHO-ICF could easily be applied to primary health care practices thereby providing a universal, philosophical foundation, and approach to interprofessional primary health care across various practice models and settings. The WHO-ICF (World Health Organization, 2013) also aligns well with popular approaches to health promotion and chronic disease management such as the expanded chronic care model (Barr et al., Reference Barr, Robinson, Marin-Link, Underhill, Dotts, Ravensdale and Salivaras2003).
Quality improvement was identified as one of the top processes to support interprofessional primary health care teams, and the literature has shown that quality improvement processes such as audit and feedback (Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012) lead to enhanced quality health care. Within team-based primary health care, a study by Donnelly et al. (Reference Donnelly, Shulha, Klinger and Letts2016) used a participatory evaluation approach that involved primary health care team members not only identifying their own program outcomes but receiving regular feedback on their performance during team process meetings. Team members specifically highlighted the use of such an evaluation approach to build cohesion among the team.
Interestingly quality improvement was also identified as an assessment, outcome, and/or indicator to measure the value of care. The research team removed these responses from the analysis as quality improvement was considered to be the actual process in which teams are measured not the indicator itself. However, it could, in fact, be argued that the nature, depth or extent of engagement in quality improvement may in fact be a performance indicator for quality in team-based primary health care. Research from the field or program evaluation and their work examining ‘process use’ (Patton, Reference Patton1998) could inform an exploration of how quality improvement itself could be formulated to be an indicator for primary health care teams. However, these results also point to the fact that determining what is an indicator for primary health care teams is very difficult. How teams operationalize these concepts are challenging, with providers coming from a diverse set of professional backgrounds and providing care to the broadest range of clients and conditions. In this study, participants responded with almost three times greater frequency when asked about characteristics of collaboration as compared to questions related to identifying outcomes, indicators, or assessments of team-based primary health care.
Limitations
The authors acknowledge limitations to the study that include the use of a convenience sample of participants’ attending the pre-conference stakeholder meeting and aggregate data that does not allow for granular level of data analysis by profession. Different facilitators led the stakeholder meetings and therefore there may have been differences in how the questions were discussed. Next steps of this work are to develop mechanisms to identify data at the patient encounter level for providers of primary health care teams to further inform and develop indicators.
Conclusion
With the growing emphasis on primary health care teams, it is critical that measures that capture the value of the team include indicators that teams feel they can directly act or contribute to. The study is a critical step in this process as it offers data from the interprofessional team members on both current and potential performance measurement. Team members specifically identified the need to include a team process indicator as well an indicator that focuses on global health status as opposed to the current biomedical indicators. While there are many frameworks to measure primary health care quality, these have relied on delivery models where primary health care is largely provided by solo practitioners. With an aging population and growing emphasis on chronic disease management, it will be imperative for comprehensive, interprofessional primary health care teams to turn to frameworks that consider health within a broader lens, where quality of life, function, and participation become important outcomes.
Author ORCIDs
Catherine Donnelly, 0000-0002-0809-3059
Acknowledgments
We would like to acknowledge the interprofessional health providers who participated in the stakeholder meetings and Association of Family Health Teams of Ontario (health care association).
Financial Support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Conflicts of Interest
None.
Ethical Standards
The authors assert that all procedures contributing to this work comply with the ethical standards of Canada’s Tricouncil Ethical Conduct for Research Involving Humans and with the Helsinki Declaration of 1975, as revised in 2008.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1463423619000409.
Open access
