Dutch Translation and Psychometric Evaluation of the Person-Centered Primary Care Measure

Translation

We first translated the English version of the PCPCM into Dutch, and then the Dutch translation back to English. The text of the back translation was shared with one of the original PCPCM developers (R.S.E.) to see if it was in line with the original formulation. Then we further adapted the Dutch version to ensure that people with low literacy would be able to fully participate in this quality assessment activity. This too was a 2-step process. First, the language of the PCPCM was adapted to meet the requirements of level 1 or level 2 literacy as defined by the International Adult Literacy Survey¹⁷ through the Dutch center with expertise on health disparities. Second, Pharos (https://www.pharos.nl) engaged a group of persons with low literacy to comment on the questionnaire and help to adapt the phrasing until they were able to read and understand the items without assistance.

Data Collection

A member of our research team visited each practice 3 to 4 days and asked all consecutive patients in the waiting room to participate. When they consented, they filled in the informed consent form and the questionnaire. They were given the choice of filling in the questionnaire on site or taking it home and returning it by mail. The researcher offered assistance in filling out the questionnaires. Excluded were patients aged under 18 years, those not registered in the practice, and those present for a first visit with their general practitioner (GP). We aimed for inclusion of at least 100 participants based on COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) guidelines.¹⁸

Questionnaire

The questionnaire consisted of a section for demographics, the translated PCPCM, 6 questions from the QUOTE, and a health status assessment using a 5-point scale (1 for poor health, 5 for excellent health). The PCPCM has 11 questions (1 for each of 11 aspects of good quality primary care) answered with scores from 1 through 4 (not at all, somewhat, mostly, definitely).¹⁴ The total PCPCM score is calculated by dividing the total score by the number of answered questions resulting in scores from 1 through 4; higher scores indicate better quality care. We only calculated the total score when at least 8 items had been answered. The QUOTE assesses quality of primary care from the patient’s perspective.¹⁹ From this questionnaire we used the 6 questions about person-centeredness for the assessment of convergent validity. Each question has a score range from 1 through 4 (never, sometimes, often, always) and the total score is calculated by dividing the sum of all item scores by 6.

Psychometric Evaluation

We used descriptive statistics for demographic characteristics and ANOVA and independent t-tests for the relation between demographic characteristics and the total PCPCM score. Before the analyses, we imputed data when missing data did not exceed more than 3 items (at least 8 items answered).¹⁴ We did not examine content validity because this had been done extensively during the development of the original survey. In agreement with the COSMIN study design checklist, we tested structural validity (the degree to which the scales or subscales are congruent with the conceptual model), convergent validity, and internal consistency. We tested structural validity by performing a confirmatory factor analysis. We considered factor loadings above 0.4 as a good association between the item and the construct.¹⁴ Internal consistency was calculated using Cronbach’s α. We planned calculation of item-total correlations for every factor, assuming the 1-factor structure of the original survey. Cronbach α values >0.8 are considered satisfactory. The Pearson correlation test was used to investigate convergent validity of person-centered care as assessed with the PCPCM compared with person-centered care as assessed with the 6 QUOTE items. We considered a correlation of >0.50 as adequate.²⁰

For all analyses we used SPSS version 25 (IBM Inc). We considered a P <.05 as statistically significant.

Study Population Characteristics

Twenty-six GPs from 4 practices (rural and urban) participated in the study. In 12 days, we enrolled 205 patients in the waiting rooms of the practices. The mean age of participants was 50.5 years (SD 18.0) with approximately one-third of all participants aged 60 years or older. Participants were mostly female (59%). The mean self-reported health status was 2.78 (SD 0.97) with 27% reporting their health as either fair or poor. Younger people reported higher health statuses than older people. Most respondents (87%) identified a GP as their primary care provider (Table 1).

Translation

An acceptable version in Dutch was obtained after 1 round of forward/backward translation for items 1, 5, 10, and 11; after 2 rounds for items 2, 3, 4, 8, and 9; after 3 rounds for item 7; and after 4 rounds for item 6. For example, the original formulation of item 7 is “My doctor or practice stands up for me.” This was backtranslated from Dutch as “Does your doctor do her best for you?” Through consultation with the original developer of the PCPCM this translation did not sufficiently cover the idea of doctor advocacy for the patient. Therefore, we changed this into “Does your GP stand up for you?” and added an example following the advice of language experts for people with low literacy which was “For example, did your GP help you at a time when you experienced difficulties with treatments in the hospital?”

PCPCM Scores

After imputation of missing data, we were able to calculate the total PCPCM score for 202 of the 205 patients. The mean PCPCM total score was 3.18 (SD 0.60). Item scores ranged from 2.55 (item 6) to 3.57 (item 3). PCPCM total scores are correlated with age (Pearson’s r = 0.34, P <.001) with older participants scoring slightly better. There was a weak correlation between PCPCM total scores and duration of registration to a practice (Pearson’s r = 0.17, P = .026). The items with the greatest number of missing values were 7 (help of GP in acquiring treatment); 8 (family context); 9 (personal context); and 11 (GP helps stay healthy). Data shown in Table 2.

Structural Validity

The confirmatory factor analysis of the PCPCM survey could not confirm the 1-factor solution. A 3-factor and a 4-factor model showed the best fit (Table 3). For clinical reasons the 3-factor solution seemed the best. The 3 factors were comprehensiveness of care, personal relation, and contextual care. Comprehensiveness of care loaded on items 1 to 4, personal relation on items 5 to 6, and contextual care on items 7 to 11.

Internal Consistency

A 3-factor model showed the best fit. Cronbach’s α was 0.82 for the comprehensiveness of care subscale, 0.73 for the personal relation subscale, and 0.86 for the contextual care subscale.

Convergent Validity

The mean QUOTE total score was 3.72 (SD 0.60). The total PCPCM and QUOTE scores are correlated with a Spearman’s ρ of 0.65 (P <.001). Figure 1 shows the correlation between the total scores of both questionnaires in a simple scatter plot. Items from the contextual care subscale were highly correlated with the QUOTE total score (Pearson’s r = 0.61, P <.001). Items from the comprehensiveness of care subscale were also highly correlated with the QUOTE total score (Pearson’s r = 0.64, P <.001). Items from the personal care subscale were moderately correlated with the QUOTE total score (Pearson’s r = 0.31, P <.001).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Scatter plot of PCPCM and QUOTE total scores.

R² linear = 0.476

PCPCM = Person-Centered Primary Care Measure; QUOTE = Quality of Care Through the Patient’s Eyes.

Main Findings

The mean score of the PCPCM in our population was 3.18. We could not confirm the 1-factor structure of the translated version of the PCPCM. Instead, we found a 3-factor solution was the best fit. We interpreted these factors as comprehensiveness of care, personal relation, and contextual care. Content validity was not tested in this study; however, through robust testing and adaptation of the Dutch translation of the PCPCM with people having low literacy, we believe we have maintained the high content validity of the original version.

Comparison With the Literature

The mean PCPCM score in our study was higher than in the study across 35 Organization for Economic Cooperation and Development countries (3.18 vs 2.75).¹⁵ While we were aware of the existence of a Dutch translation when we started the study, we contacted the first author of the original PCPCM study and decided to redo the translation. We also adapted the new translation for people with low literacy. It is possible that the new version resulted in the higher PCPCM score because the survey was now comprehensible for more people.

When comparing our study with the original study it is notable that factor analysis in the original study¹⁴ revealed 1 factor. Our confirmatory factor analysis resulted in 3 factors. These factors make sense when referring to the 11 dimensions from the original questionnaire; all 3 are clearly part of the person-centeredness construct. The internal consistency of the 3-factor model was high and in line with scores of an earlier study.¹⁵ The difference between our study (3 factors) and the original, the Chinese,²¹ and Japanese²² versions (1 factor) is possibly the result of some elements of the Dutch health care system.¹⁶ In the Dutch health care system everyone is registered with a general practitioner—thus ensuring continuity of care—and access to secondary care is only obtained with a referral by the general practitioner (gatekeeper function).

Strengths and Limitations

A major strength of this study is the quality of the translation to ensure sufficient content validity. We translated the questionnaire in collaboration with 1 of the original developers and with organizations representing people with low literacy to ensure sufficient comprehensibility for more people. A second strength is that we followed the COSMIN guidelines and taxonomy transparently. A further strong point is that we tested the questionnaire with patients in waiting rooms of general practitioners, the target population for the PCPCM.

An important limitation is the possibility of social desirability bias. It is possible that patients in the waiting room responded assuming that their physician would be aware of their answers. This limitation can be addressed in future research by measuring this type of bias and controlling its effects in the analysis.²³ A further limitation is the high number of missing values for PCPCM items 7, 8, 9, and 11. These questions are all within the contextual care factor. In particular, item 7 assessed the role of the GP in acquiring specialized (hospital) care. We assume that, due to the gatekeeping function of primary care physicians and the infrequent occurrence of a necessary visit to the hospital, fewer participants were able to answer this question. Items 8 and 9 inquired about the interest of a GP in the family and community of a participant when offering care. We assumed that answering this question required the patient either to have complex medical problems needing involvement of family and community, or to have dealt with these problems in the past. For younger patients or for those without a long-term relationship with their GP, these items might be difficult to answer. The same arguments are valid for question 11. This item is about whether your general practitioner helps you stay healthy. Another limitation is that our findings are not comparable with the results of the psychometric validation of the original, Japanese, and Chinese versions of the PCPCM.^21,22 We consider the specific properties of the Dutch health care system (strong and accessible primary care, secondary care not directly accessible, all patients have health insurance) responsible for this difference. For the same reasons, the Dutch health care system differs from the US health care system. The applicability of the survey in the Dutch context—certainly for question 7—has to be reconsidered in future studies. Also, we were not able to study test-retest reliability. This should be addressed in future studies. Further, as we did not specifically test the comprehensibility of the Dutch PCPCM among people with low literacy, we cannot be sure how effective the adaptations in translation were without further qualitative research (such as cognitive interviewing). Finally, we did not specifically assess the literacy level of participants in responding to the PCPCM. While we would expect that literacy levels would have no effect on PCPCM scores with these adaptations, we cannot confirm this with this study.

Further Implications

The Dutch version of the PCPCM should be evaluated further in a larger sample and include test-retest reliability, measurement error, responsiveness, and the relation between literacy level and total PCPCM scores.