Reliability and Validity of a Comprehensiveness of Care Measure in Primary Care, A Case Study of the PRIME Registry

Context: Comprehensiveness of care represents an important process measure within the contexts of primary care for core services. These services represent the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs.

Objective: To evaluate comprehensiveness of care measure reliability for clinicians and advanced practice practitioners (e.g. nurse practitioners and physician assistants), as well as validity of the measure and its association with poorly controlled diabetes (e.g. Hemoglobin A1C > 9.0 as a case study).

Study Design and Analysis: A retrospective cohort of providers and its patient panel for two performance years, 2019 and 2022. Intraclass correlation coefficients as measured through provider-level random effects from a hierarchical regression was performed for reliability evaluation and validation analysis used a hierarchical Poisson regression with practice-level random effects and repeated measures for providers measured across multiple performance years (2019 and 2022).

Setting or Dataset: The American Family Cohort database, derived from the PRIME registry.

Population Studied: We identified all providers that had at least 300 patients available in both 2019 and 2022 performance years.

Intervention/Instrument: Comprehensiveness of care for validity testing.

Outcome Measures: Primary outcome was comprehensiveness of care and secondary outcome was poorly controlled hemoglobin A1C in validity testing analysis.

Results: We identified 1,276 providers with a reliability estimate of 0.784, showing very high reliability. Family medicine and internal medicine specialties had higher reliability than their advanced practice practitioner counterparts. The lowest comprehensiveness of care decile (<10th percentile of all providers) had the largest predicted proportion of its diabetic patient panel with poor control (2019: 52%; 2022: 67%). Conversely, providers in the top decile (>90th percentile) had among the lowest predicted proportions of its diabetics with poor control (2019: 26%; 2022: 31%). There exists evidence that comprehensiveness of care is a valid measure with poor control with diabetes.

Conclusion: Comprehensiveness of care is a marker of quality and represents a reliable measure for provider types across specialty groups. Comprehensiveness can inform opportunities for primary care providers to judiciously perform various services.