Abstract
PURPOSE Electronic Health Record (EHR) databases in community health centers (CHCs) present new opportunities for quality improvement, comparative effectiveness, and health policy research. We aimed (1) to create individual-level linkages between EHR data from a network of CHCs and Medicaid claims from 2005 through 2007; (2) to examine congruence between these data sources; and (3) to identify sociodemographic characteristics associated with documentation of services in one data set vs the other.
METHODS We studied receipt of preventive services among established diabetic patients in 50 Oregon CHCs who had ever been enrolled in Medicaid (N = 2,103). We determined which services were documented in EHR data vs in Medicaid claims data, and we described the sociodemographic characteristics associated with these documentation patterns.
RESULTS In 2007, the following services were documented in Medicaid claims but not the EHR: 11.6% of total cholesterol screenings received, 7.0% of total influenza vaccinations, 10.5% of nephropathy screenings, and 8.8% of tests for glycated hemoglobin (HbA1c). In contrast, the following services were documented in the EHR but not in Medicaid claims: 49.3% of cholesterol screenings, 50.4% of influenza vaccinations, 50.1% of nephropathy screenings, and 48.4% of HbA1c tests. Patients who were older, male, Spanish-speaking, above the federal poverty level, or who had discontinuous insurance were more likely to have services documented in the EHR but not in the Medicaid claims data.
CONCLUSIONS Networked EHRs provide new opportunities for obtaining more comprehensive data regarding health services received, especially among populations who are discontinuously insured. Relying solely on Medicaid claims data is likely to substantially underestimate the quality of care.
- Electronic health records
- access to health care
- community health centers
- primary health care
- preventive health services
- diabetes mellitus
- medically uninsured
- Medicaid
INTRODUCTION
An important part of most health care quality assessment efforts is measuring rates of receipt of recommended services. Health insurance claims databases have often been used for reporting receipt of these quality measures, assessing the performance of health care providers, and measuring geographic variation in care delivery and expenditures.1–4 Such administrative claims data, however, have long been known to contain incomplete information on care received.5–8 Claims data are incomplete because not all services received are billed, patients change insurance payers, and a high percentage of US patients do not have stable insurance coverage and thus incur no claims when they receive care.
Community health centers (CHCs) provide services to such uninsured populations9–15; thus, their medical records include information about receipt of care that would not be captured in claims data. Until recently, however, studies of services received in CHCs have been limited to labor-intensive audits of paper medical records, or the collection of primary data from patient surveys.16–27 Electronic health record (EHR) databases have the potential to help researchers and policy makers overcome many of the previous limitations to measuring receipt of care in CHCs. Thus, these emergent data resources present new opportunities for measuring care quality in vulnerable populations, using automated EHR data available in real time.
To test the hypothesis that, in a CHC population, EHR data will portray outpatient care received more accurately than will Medicaid claims, we studied receipt of preventive care services among established diabetic patients from a linked CHC network. We created individual-level linkages between EHR data from 50 Oregon CHCs linked through the OCHIN Health Center-Controlled Network, and Medicaid insurance claims. We aimed (1) to examine congruence between these 2 data sources, and (2) to identify sociodemographic characteristics associated with documentation of services in EHR vs Medicaid claims data.
METHODS
Setting: A Network of Community Health Centers in Oregon with Linked EHR Data
In 2001, a group of CHCs in Oregon formed a collaboration, originally called the Oregon Community Health Information Network and shortened to OCHIN as other states joined, to facilitate the adoption of health information technology in CHCs. These CHCs collectively purchased a centrally hosted Epic EHR system, and then instituted an enterprise-wide master patient index. OCHIN built and maintains this fully integrated electronic health information exchange system, in which each patient has a single medical record available to clinicians across the network. OCHIN’s single database spans clinical, access, and revenue functions. As of December 1, 2010, OCHIN was supporting EHRs at more than 200 CHC sites serving more than 860,000 unique patients with more than 7.9 million visits since 2002.
Study Population and Data Linkages
We selected adult patients who had at least 2 visits between January 1, 2004, and December 31, 2005, associated with a diabetes mellitus ICD-9 (International Classification of Diseases, Ninth Revision) diagnostic code and at least 1 visit for any reason in both 2006 and 2007 (diabetes ICD-9 codes: 250.x, 253.5, 357.2, 362.01-2, 366.41, 790.21-2, 790.29, 791.5-6, and V65.46). These criteria ensured a minimum level of continuity of care between 2004 and 2007 and reduced the likelihood of including patients who left the clinic network. OCHIN EHR data are limited before 2005, so 2004 data were used only to identify the study cohort; outcomes were assessed in 2005–2007 exclusively.
From 4,240 adult OCHIN patients with diabetes who met the initial inclusion criteria, we selected those with an Oregon Medicaid identification (ID) number (N = 2,103). Medicaid is the primary insurer of this population. When a visit was not covered by Medicaid, more than 90% of this population reported having no other insurance. Using Medic-aid ID numbers available in both data sets, we made individual-level linkages between OCHIN’s EHR data and Oregon’s Medicaid claims data. We obtained data-sharing agreements between all agencies and researchers involved in the study. The study protocol was approved by the institutional review boards of the Kaiser Permanente Center for Health Research and Oregon Health and Science University.
Variables
We focused on receipt of 4 evidence-based diabetes preventive care services: assessment of glycated hemoglobin (HbA1c), lipid screening (low-density lipoprotein cholesterol [LDL]), influenza vaccination, and nephropathy screening (urine microalbumin). It is recommended that diabetic patients receive these services at least annually.28–33 We assessed 2 outcomes for each of these 4 measures: (1) whether a patient received the service in any year during the 2005–2007 period, and (2) whether a patient received the service in all 3 years, 2005 through 2007. Services were identified using procedure codes and CPT codes commonly associated with each (Supplemental Appendix, available at http://www.annfammed.org/cgi/content/full/9/4/351/DC1). Although not an exhaustive list of codes, this list included all relevant codes that were used in the OCHIN database as confirmed by OCHIN data architects.
Analysis
We first described the sociodemographic characteristics of the study population. For the remainder of the analyses, we assessed whether services were documented in 1 or both data sources, as illustrated in Figure 1⇓, which included the following 6 possible combinations of data:
-
Medicaid insurance claims data (Figure 1, A+ab⇑)
-
OCHIN clinic EHR data (Figure 1, B+ab⇑)
-
A combined Medicaid claims-EHR data set (Figure 1, A+B+ab⇑)
-
Only in Medicaid claims and not in EHR data (Figure 1, A⇑)
-
Only in EHR data and not in Medicaid claims data (Figure 1B⇑)
-
Both Medicaid claims and EHR data (Figure 1, ab⇑)
We compared the percentages of patients who had received the services of interest using only the claims data (Figure 1, A+ab⇑), only the EHR data (Figure 1, B+ab⇑), and the combined claims/EHR data set (Figure 1 (A+B+ab)⇑. Then, among all patients who received at least 1 service over a specified time period (obtained from the combined data sets), we determined the percentage with service documentation only in Medicaid claims data (Figure 1, A⇑), only in EHR data (Figure 1, B⇑), or in both data sources (Figure 1, ab⇑). Finally, to better characterize the population subgroups most likely to be missed in claims data (Figure 1, A+ab⇑), we conducted a series of multivariate logistic regression models to identify sociodemo-graphics significantly associated with having documentation in only the EHR data (Figure 1, B⇑) and not in the claims data. All analyses were conducted using SAS 9.2 statistical software.34
RESULTS
The majority of patients were aged 19 to 64 years (78.5%); only 21.5% were 65 years or older (Table 1⇓). The population was 17.0% Hispanic and 9.5% black, and almost all were from households below 200% of the federal poverty level (FPL).
Documentation in Medicaid Claims, EHR, and a Combined Medicaid Claims-EHR Database
As displayed in Table 2⇓, the optimal service reporting was obtained when combining the EHR and claims data [Figure 1 (A+B+ab)⇑]. When comparing service receipt rates in claims vs EHR data, the rates in EHR data (Figure 1, B+ab⇑) were consistently closer to the combined total than were those from the claims data (Figure 1, A+ab⇑).
Documentation in One Data Set vs Both Data Sets
In almost all cases fewer than one-half of the patients who received services had documentation in both EHR and Medicaid claims data (Figure 1, ab⇑) (Table 3⇓). The percentage of patients with documented receipt of service(s) in EHR only (Figure 1, B⇑) was consistently higher than in claims only (Figure 1, A⇑). For example, in 2007, 7.0% of patients had an influenza vaccination documented in claims only vs 50.4% in EHR only; the remaining 42.6% were documented in both data sets. For patients who received at least 1 microalbumin test to screen for nephropathy in 2007, 10.5% were documented in claims data but not in EHR, compared with 50.1% in EHR data but not in claims data, with the remaining 39.4% in both data sets.
Table 4⇓ further shows that using Medicaid claims data (Figure 1, A+ab⇑) to measure preventive services delivered in the OCHIN clinic network would result in underreporting, with greater disparities in certain sociodemographic subgroups. Patients older than 64 years, men, Spanish-speaking patients, and those without continuous Medicaid coverage throughout the study period had higher odds of service documentation in the EHR data only (Figure 1, B⇑). Patients with incomes below the poverty level had lower odds of having data in the EHR only. There were few statistically significant differences associated with race/ethnicity.
DISCUSSION
As expected, the combined EHR and Medicaid claims data provided the most complete picture of diabetes services received, and the EHR data provided a more complete picture than did data from Medicaid claims. This finding suggests that assessing performance in CHCs based on Medicaid claims alone would be inaccurate, especially in certain subpopulations (Table 4⇑). These disparities may occur because some subgroups have a more difficult time maintaining Medicaid coverage. In some cases, these patients might be more likely to have more than 1 form of insurance coverage.
Data were missing from the Medicaid claims data for a large percentage of services received in an OCHIN-affiliated CHC. Although all patients in this study population had a Medicaid ID number, many had an insurance coverage gap during the study period; services received during a coverage gap were missed in claims data. Even some patients with continuous Medicaid coverage had services not documented in claims data. It is possible that these services were billed to a different insurer, especially for older patients with dual Medicare and Medicaid coverage. The most likely explanation for services documented in Medicaid claims but not in EHR data is that the service was received at a health care facility outside the OCHIN network and thus not documented in the OCHIN EHR, but billed to Medicaid.
Performance measures should optimally be measured using the most complete combination of data possible. When only one data set can be used, it should be EHR data. Although a relatively small percentage of services were missing in the EHR data, it was reassuring that for 3 of 4 outcomes measured (Table 2⇑), the EHR data moved closer to the combined data set rates with time (from 2005 to 2007), suggesting that EHR data are becoming more complete as systems mature. This finding also suggests a fairly reliable population denominator in the OCHIN clinic network’s EHR data set.
Research and Policy Implications
Previous work has shown the relevance of CHC data to policy discussions about how to improve health care delivery and outcomes for underserved patients.9,10,14,16,18,23,35,36 Our findings show how EHR data from CHC networks provide a new resource for policy makers to better understand health services delivery in CHC populations.10,15,16,37–40 Policy makers and payers need to look beyond Medicaid claims data to measure population health and CHC quality performance. EHR databases—such as the one being built and maintained by OCHIN—will be key to the study of how health reform policies implemented under the Patient Protection and Affordable Care Act of 2010 (PPACA) affect the uninsured and underinsured.41–49 Although some states are putting considerable resources toward building all-payer claims databases to better address the PPA-CA’s evaluation requirements,50 such databases will not include uninsured and transiently insured populations. Information from EHR data will be more accurate than administrative claims data; thus, it is imperative to further develop and validate important EHR data resources and to evaluate the extent to which EHR data can be used to supplement (or substitute for) claims data. Our results also show the potential usefulness of combining EHR and claims data sources for use in comparative effectiveness and translational research.
Practice Implications
In addition to the research and policy implications, our results have important implications for clinical practice. This study shows that a well-built EHR with comprehensive data can be more complete than claims data for measuring how a practice is performing. Clinicians can partner with researchers to study data from their own EHRs in the evaluation of quality improvement interventions and to measure the impact of these efforts on care delivery and outcomes.
Limitations
We only compared the OCHIN EHR data with Oregon Medicaid claims data, as Medicaid is the primary insurer of this population of patients. The analyses could be strengthened by linking EHR data with claims from additional payers. It was beyond the scope of these analyses to determine where patients had received care outside the OCHIN network. For example, this population’s low rate of influenza vaccination is likely because many patients sought vaccinations outside the CHC system. Further research could identify external sites commonly used by this population and pursue methods to capture that data in the future.
Our results may give conservative estimates of the diabetic patient population in these clinics. We defined patients as diabetic if they had 2 or more visits associated with a diabetes diagnostic code over a 2-year time period, so we could avoid incorrectly considering patients as diabetic based on a single rule-out diagnostic code. This method is commonly used for the Healthcare Effectiveness Data and Information Set (HEDIS) measures. We further limited our population to ensure a continuity relationship. Thus, we likely missed some diabetic patients who had fewer than the minimum number of visits during the study period. Although our rates are comparable to available estimates from nationally representative data,16 directly comparing rates of receipt of care measures with those of other populations was not feasible, as previous assessments varied in how receipt of care was measured and how populations were defined.14–17,23,38,51 This analysis was done in one state, but it could be replicated in other states with similar CHC networks.
Despite the above limitations and the need for further study, we believe that OCHIN’s data set is unique and substantially improves on what has been previously available. This study would not have been possible using older methods of paper chart review, data-sifting, and less-comprehensive data sets.
Networked EHRs from CHCs provide new opportunities for obtaining more comprehensive data on health services received, especially in discontinuously insured populations. These EHR databases can be linked and compared with claims databases, such as Medicaid, and may prove to contain more robust data for the measurement of primary and preventive care services utilization in vulnerable populations. Relying solely on claims data is likely to substantially underestimate the quality of care received. Further, our study shows how primary care practices can collaborate within information technology networks and effectively partner with researchers to study their own care delivery, to conduct comparative effectiveness research, to affect the translation of evidence into practice, and to inform policies.
Acknowledgments
We are grateful to OCHIN and to the Oregon Division of Medical Assistance Programs. We also wish to acknowledge the contributions of reviewers and editors who made this manuscript significantly better.
Footnotes
-
Conflicts of interest: authors report none.
-
Author contributions: All authors take responsibility for the entire content of the manuscript. Drs DeVoe and Gold had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs DeVoe and Gold designed the analyses and wrote the manuscript. Ms McIntire and Mr Puro conducted data linkages and created the analysis data set. Dr Gold conducted all analyses. Ms Chauvie and Dr Gallia provided substantial consultation in study design and results interpretation.
-
Funding support: This study was supported by grant number UB2HA20235 from the Health Resources and Services Administration (HRSA), grant number 1RC4LM010852 from the National Library of Medicine, National Institutes of Health (NIH), and a small pilot grant from the Oregon Clinical and Translational Research Institute (OCTRI), grant number UL1 RR024140 01 from the National Center for Research Resources (NCRR), a component of the NIH, and NIH Roadmap for Medical Research (Gold and McIntire), and received support from the OHSU Department of Family Medicine Research Division (DeVoe). Dr DeVoe’s time on this project was also supported by grant number 1K08HS16181 from the Agency for Healthcare Research and Quality (AHRQ). These funding agencies had no involvement in the design and conduct of the study; analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. Ms Chauvie and Mr Puro’s time was donated by OCHIN. Dr Gallia’s time was donated by the Oregon Division of Medical Assistance Programs.
- Received for publication November 10, 2010.
- Revision received March 22, 2011.
- Accepted for publication April 11, 2011.
- © 2011 Annals of Family Medicine, Inc.