Measuring Primary Care Across 35 OECD Countries

INTRODUCTION

Primary care is widely viewed as the cornerstone of a high-functioning health care system.^1-4 Considerable variability in health care system design and in how primary care is provided presents an opportunity to learn from different approaches,⁵ in order to better understand and improve what matters about primary care.⁶ Most system-level measures of primary care, however, are crude in measuring what actually happens regarding the multiple functions of primary care. With a few limited exceptions, including the Primary Care Assessment Tool,⁷ World Health Organization’s (WHO) Primary Care Evaluation Tool (PCET),^8,9 and measures developed for the Quality and Costs of Primary Care in Europe (QUALICOPC) study,¹⁰ most patient-report measures that are sensitive to the actual delivery of personalized care have not been translated or validated for use in multiple countries.^11,12

As published previously in this journal,¹³ the Person-Centered Primary Care Measure (PCPCM) is an 11-item, patient-reported measure that assesses the broad scope and integrative, comprehensive nature of primary care including: accessibility, advocacy, community context, comprehensiveness, continuity, coordination, family context, goal-oriented care, health promotion, integration, and relationship. Developed with extensive prior research, the PCPCM is based on what hundreds of patients, clinicians, and to a lesser extent payers say matters in primary care.¹⁴ The breadth of characteristics included in the measure reflect the complex role of a personal physician and a primary care practice. The PCPCM has been validated in 3 large Internet samples and 11 smaller clinical samples.¹³ We translated the PCPCM from American English into the 27 other primary languages used in Organisation for Economic Co-operation and Development (OECD) countries.¹⁵

The purpose of this study is to examine the reliability and validity of the Person-Centered Primary Care Measure in 28 languages and 35 countries, and to explore differences in primary care across 35 OECD Countries. We aim to make the PCPCM available for use across diverse languages and countries, and to stimulate preliminary hypotheses about inter-country differences that can be examined in future research to guide policy and practice based on the natural experiment of different countries approaches to health care.

METHODS

We engaged a company with considerable global and corporate experience (Morningside Translations LLC) to translate the PCPCM from English (USA) into 27 languages used as primary languages in OECD countries. (The surveys fielded in Spain were in [European] Spanish, and the ones in Chile and Mexico were in Latin American Spanish. The UK surveys were in British English.) Items and instructions were translated and then back-translated, with attention to cultural as well as linguistic distinctions, and then reviewed by the study team. In a few instances, we engaged additional input from members of relevant cultural and linguistic groups to resolve any questions about item appropriateness. The translated PCPCMs are available for public use, along with implementation and scoring guidance, on the website of the Larry A. Green Center for the Advancement of Primary Health Care for the Public Good.¹⁵

We contracted with SurveyMonkey (SurveyMonkey Inc) to reach a sample of 360 survey respondents representing the adult population demographics in each of the 36 OECD countries in 2019.

The survey captured population demographics (male, female, trans, other), age (aged <18, 18-29, 30-44, 45-60, >60 years), and whether respondents considered themselves to be a member of a minority ethnic or racial group, but we were not able to specifically sample on these characteristics.

Success of the recruitment strategy was determined by the panel size of SurveyMonkey’s respondents within the given country. Because there were no screener questions to determine whether a respondent fit the criteria to complete the survey, SurveyMonkey sent the survey to as many respondents as necessary until the minimum number of responses were collected (in this case, 360). Those that received the survey in their country were given the option of completing the survey or skipping it. Once enough respondents completed the survey, SurveyMonkey stopped sending the survey to more potential respondents.

For Iceland, SurveyMonkey was unable to provide sufficient sample size, and that country was excluded. For Luxembourg, SurveyMonkey was able to obtain a sample size sufficient for validation (n = 150), but not the 360 requested that would allow for future analyses. A sample size of 360 allows sufficient power to detect small effect sizes in statistical analyses,¹⁶ but for this ecological study, the statistics involved only descriptive and psychometric analyses.

In these samples, we fielded an online survey translated to the predominant language in each OECD country. For Belgium, where residents are nearly equally split in their primary use of 2 languages, the survey was preceded by an introductory question asking whether respondents would prefer to take the survey in Dutch or French (49.9% chose Dutch and 50.1% chose French). The survey consisted of the PCPCM and questions about demographics, the Patient-Enablement Instrument (PEI),^17-20 number of years the person had been with their primary care physician and practice, and whether the patient thought the doctor knowing the results would improve their care.

In order to have confidence in interpreting each country’s PCPCM score, especially given multiple language translations and cultures, it was necessary to reestablish the psychometric properties of the PCPCM for each country. This analysis consisted of providing country-specific evidence of the measure’s reliability and concurrent validity. Evidence of the measure’s internal consistency reliability was provided by examining the item-total correlations and by computation of Cronbach’s α reliability coefficient. Evidence in support of the measure’s concurrent validity was provided by examining the correlation of the PCPCM with the PEI and by examining the associations of PCPCM scale scores with important single item outcomes, such as years with the current physician and practice and by asking if knowing the results would help the physician understand the respondent’s feeling about their health care. Taken together, these analyses are designed to provide evidence that the PCPCM item set was working effectively in each country, as documented by their psychometric properties, and thereby increasing confidence in the interpretation of the country-specific PCPCM scores.

We also report the total and item-specific scores for each country and include the P value for Bonferroni adjustment for multiple hypothesis testing.

This study protocol was reviewed by Virginia Commonwealth University’s Institutional Review Board.

RESULTS

The 35 mean PCPCM scale scores exhibited a wide range of scores from Sweden (lowest observed score: 2.28) to Turkey (highest observed score: 3.08). The overall mean of the 35 countries was 2.74 with a standard deviation of 0.19. Given this range and the small standard deviation, it is clear that there are many inter-country differences in PCPCM mean scores (Table 1). Since each item represents a separate and important domain of primary care, comparative item analyses among countries can be informative (Table 2). Moreover, examining the item means in relation to each country’s total score helps to understand which domains contributed the most to the total score.

Examining the instrument’s psychometric properties, the 11 item-total correlations for each country exhibited a solid range of correlations ranging from 0.47 to 0.81 with the vast majority of countries ranging from the low 0.50s to the high 0.70s. Not one item in any of the 35 countries exhibited an inconsistent association with the total score. Consistent with these ranges are the country-specific Cronbach α reliabilities. Alpha reliabilities ranged from 0.88 to 0.95. These high reliability coefficients indicate a high degree of internally consistent responses by the respondents. Taken together, these reliability analyses indicate the 11-item scale was performing as designed despite language translations and cultural differences.

Evidence of concurrent validity is provided by the solid correlations of PCPCM scores and scores on the PEI (Table 1). These Pearson correlations ranged from 0.57 to 0.74 and provide initial evidence of concurrent validity.

Further evidence of concurrent validity is provided by the association of PCPCM scores with single-item outcomes. Substantial rank-ordered associations were observed between mean PCPCM scores and responses to the item: “How many years have you known this doctor?” Twenty-seven countries (77%) had a significant association with this item outcome. The results consistently showed, the more years with the doctor, the higher the PCPCM score.

Similarly, strong associations were observed with the item: “How many years have you been a patient at this practice?” Nineteen countries (54%) had a significant association with this item outcome. In general, the more years with the practice, the higher the PCPCM score.

Next, consistent and strong associations were observed between PCPCM scores and the item: “If your doctor or practice received the answers to these questions, would it help them to understand how you feel about your care?” In 33 of the 35 countries (94%), the respondents that said yes to this question had substantially higher PCPCM scores than those that responded with a no.

Finally, several associations were observed with the variables of age and sex. Eight countries (23%) demonstrated an association between patient age and PCPCM scores. In the majority of instances, these associations were not rank ordered. Often both younger and older patients exhibited higher scores while in other associations, middle-age patients scored higher. Sex differences were observed in 8 countries (23%) with the majority of associations indicating males scoring higher than females.

A final analysis examined PCPCM scores associated with the item: “Was it hard to complete this form?” Only 2 countries (6%) showed an association of PCPCM scores with this variable, Japan and Chile, but in opposite directions. In Japan respondents who answered no to this question had higher PCPCM scores than those that said yes. In Chile, the opposite was true, those that responded yes had higher scores than those reporting no.

DISCUSSION

Taken together, the strong and generally consistent psychometric results support the use of the PCPCM in each respective country. The reliability results were solid across all countries and the concurrent validity findings were likewise solid given the associations with the PEI and the single item outcomes.

This study builds on prior research using somewhat different measures to compare countries and to assess the role of primary care in achieving effective and universal coverage,²¹ and provides a basis for further comparative research.^10,22-24 The grounding of the PCPCM in careful analysis of what patients, clinicians, and employers say is important in primary care¹³ and provides a particularly strong basis for further research. The consistency with which the broad constructs assessed by the PCPCM are understood and appreciated by patients/the public across 35 countries with different cultures and health systems provides evidence of the universal internal consistency and conceptual coherence of the breadth of primary care attributes.

The variability of the total score and individual items across countries is interesting. We encourage readers with local knowledge of each country to use this journal’s e-letter feature to share hypotheses that might explain these differences.

At the extremes—the high scores for Turkey may reflect reforms of the past decade that have attempted to increase the availability of primary care, and previous high levels of patient satisfaction with their treatment by family doctors.²⁵ The low scores for Sweden may surprise some who have heard of the strong Swedish welfare state and relatively homogeneous culture. However, despite many health care system and primary care strengths and reforms,²⁶ according to one analyst:

“In Sweden, and increasingly even in traditional strongholds of general practice, team-based primary care is thought to respond to increasing demands, filtering out non- and minor disease through triage, practicing task distribution, and moving the GP to a secondary level working with the ‘really sick’, in all a decline in direct contact between patient and GP. Conclusions: When this happens, clinical medicine as a whole becomes drained of the practice of its human dimension.”²⁷

We wonder if the relatively high scores for the United States (ranked #3), given inter-country rankings using different measures that rate US health care and primary care more poorly,²⁸ reflects the fact that the PCPCM was developed based largely from data on what US patients, clinicians, and payers say is important, but we invite readers to posit other hypotheses.

The study is limited by a sampling scheme that, while similar across countries, may not have yielded a representative sample of patients in each country. It also is possible that, despite efforts at linguistically and culturally sensitive translation, some differences between countries could be due to contextual factors affecting interpretation of specific PCPCM items.

Nevertheless, the solid reliability and validity of the PCPCM across different languages and countries, and the variability of responses across countries call for further ecological and individual research using this measure to learn from the natural experiment of different approaches to health care across different countries.^29-31