Abstract
PURPOSE The purpose of this study was to assess patient and practice outcomes after introducing the Asthma APGAR (Activities, Persistent, triGGers, Asthma medications, Response to therapy) tools into primary care practices.
METHODS We used a pragmatic cluster-randomized controlled design in 18 US family medicine and pediatric practices to compare outcomes in patients with persistent asthma aged 5 to 45 years after introduction of the Asthma APGAR tools vs usual care. Patient outcomes included asthma control, quality of life, and emergency department (ED), urgent care, and inpatient hospital visits. The practice outcome was adherence to asthma guidelines.
RESULTS We enrolled 1,066 patients: 245 children, 174 adolescents, and 647 adults. Sixty-five percent (692 patients) completed both baseline and 12-month questionnaires, allowing analysis for patient-reported outcomes. Electronic health record data were available for 1,063 patients (99.7%) for practice outcomes. The proportion of patients reporting an asthma-related ED, urgent care, or hospital visit in the final 6 months of the study was lower in the APGAR practices vs usual care practices (10.6% vs 20.9%, P = .004). The percentage of patients with “in control” asthma increased more between baseline and 1 year in the APGAR group vs usual care group (13.5% vs 3.4%, P =.0001 vs P =.86) with a trend toward better control scores and asthma-related quality of life in the former at 1 year (P ≤.06 and P = .06, respectively). APGAR practices improved their adherence to 3 or more guideline elements compared with usual care practices (20.7% increase vs 1.9% decrease, P = .001).
CONCLUSIONS Introduction of the Asthma APGAR tools improves rates of asthma control; reduces asthma-related ED, urgent care, and hospital visits; and increases practices’ adherence to asthma management guidelines.
- asthma
- asthma control
- outcomes
- asthma management
- primary care
- pragmatic research
- practice-based research
- protocol
- implementation
- guideline
- asthma tool
- randomized clinical trial
INTRODUCTION
Asthma is a common chronic condition,1–3 and primary care is the site of most asthma diagnoses and management.2–5 Implementation of asthma guidelines has been challenging, and the burden of asthma remains high with little improvement from 1990 to 2010.6–13
Use of a metric for assessing asthma control has been suggested for improving asthma care, and several asthma control scores have been developed and validated against allergists’ assessments.14–25 But use of these tools remains limited and has not significantly improved asthma outcomes.23,26–36
The Asthma APGAR (Activities, Persistent, triGGers, Asthma medications, Response to therapy) (Figure 1) tools were developed not only to provide a score for asthma control, but also to guide further evaluation of inadequate control with a link to a care algorithm (Supplemental Figure 1, available at http://www.annfammed.org/content/16/2/100/suppl/DC1/). In a small pilot study, use of the Asthma APGAR tools was associated with enhanced asthma guidelines adherence, and focus groups of patients and clinical staff reported that the tools were easy to use, “made sense,” and “improved care.”37 The Asthma APGAR control score has been validated against the Asthma Control Test (ACT).38
Herein we present results of our large pragmatic cluster-randomized trial comparing introduction of the Asthma APGAR tools with usual care. We hypothesized that introducing these tools would improve patient outcomes and practices’ adherence to asthma guidelines. We also explored the barriers to and facilitators of implementation in the study practices.
Our study was completed in practice-based research networks (PBRNs), which are particularly suitable settings for translational research.39–43 The purpose of the study was to facilitate selection of tools to improve asthma care and outcomes in the primary care setting.27,28,44,45
METHODS
Study Design
We undertook a pragmatic, cluster-randomized controlled effectiveness trial with crossover to intervention for all practices at study midpoint (Figure 2) that was completed in 18 family medicine and pediatric PBRN members. The study methods have been published previously.46 Of the 22 practices enrolled, 18 practices (9 intervention and 9 usual care randomized by practice size, practice type, and being a residency training site) completed the study. Three practices were lost before enrolling any patients (2 were sold and 1 became an urgent care site), and a 4th practice enrolled only 10 patients before being destroyed in a tornado.
The trial’s coprimary endpoints were patient-reported asthma control using the ACT for adults and the child version (C-ACT) for children,17,18 and asthma-related quality of life using the Asthma Quality of Life Questionnaire (AQLQ) for adults or the pediatric version (PAQLQ) for children.25 We also assessed changes in the proportion of patients reporting asthma-related emergency department (ED) visits, urgent care visits, or inpatient hospital visits (hospitalizations). Data were collected by questionnaires mailed to the patients at baseline, 6 months, and 12 months. Because of the pragmatic nature of the study, no study visits were required. All asthma visits and care were at the discretion of the patient and their chosen health care professionals. The study was powered to detect a 3-point difference in ACT scores,47 assuming a 65% survey response rate at 12 months.
Secondary outcomes were changes in practice documentation of adherence to elements of the 2007 National Asthma Education and Prevention Program (NAEPP) US asthma guidelines14 assessed by research staff review of medical records. Exploratory outcomes were facilitators of and barriers to implementation of the Asthma APGAR tools reported by the practice staff during exit interviews.
Institutional review board approval was obtained for each practice, the Olmsted Medical Center, and the American Academy of Family Physicians. All patients and practice staff signed informed consent forms.
Intervention: Asthma APGAR Tools
The Asthma APGAR tools include a patient-completed asthma control assessment covering 5 domains: activity limitations (A/activity), daytime and nighttime symptom frequency (P/persistence of symptoms), asthma triggers (G/triGGers), adherence to asthma medications (A/adherence), and patient-perceived response to therapy (R/response to therapy) (Figure 1). The assessment is linked to a care algorithm that goes beyond simple “step up” therapy48 (Supplemental Figure 1, available at http://www.annfammed.org/content/16/2/100/suppl/DC1/). Scores of greater than 2 on the activities and persistence questions are consistent with lack of asthma control.38 During training, practices requested that a further set of questions be developed to facilitate expansion on patients’ responses such as adherence evaluation. No attempt was made to validate this “practical practice” tool (Supplemental Figure 2, http://www.annfammed.org/content/16/2/100/suppl/DC1).
Study Practices
The enrolled practices were members of the American Academy of Family Physicians National Research Network (NRN) (http://www.aafp.org/nrn) or the American Academy of Pediatrics Quality Improvement Innovation Network (QuIIN) (http://quiin.aap.org) and were selected to provide broad geographic coverage of the United States (Supplemental Figure 3, http://www.annfammed.org/content/16/2/100/suppl/DC1). Practices had not previously engaged in asthma quality improvement and were instructed not to begin other asthma projects during the study period. Each practice was given $1,300 annually for 4 years in recognition of practice participation. Randomization was by practice.
Study Patients
Patients were recruited either by invitation at an asthma visit or by letter and telephone contact after identification using billing data that recorded at least 2 visits for asthma (International Classification of Diseases, Ninth Revision codes 493.xx). The patient enrollment process and the data collected were the same regardless of the practice’s assigned group, and patients were unaware of their practice’s randomization status.
Children younger than 5 years were excluded to avoid the asthma diagnostic uncertainty often found in preschool children.49,50 Adults older than 45 years were also excluded because of concerns of chronic obstructive pulmonary disease (COPD) misdiagnosed as asthma and the Asthma COPD Overlap Syndrome (ACOS), for which treatment, disease progression, and diagnostic evaluations differ from those for asthma.51,52 Individuals with other chronic lung diseases were also excluded. Spirometry testing was not required to confirm the asthma diagnosis.53–55 All patients were reported to have persistent asthma requiring daily maintenance medication.
Study Procedures
Centralized training was directed to 2 “trainers” from each practice (a physician and a nursing staff member) who returned to the practice and trained the rest of their practice members. Trainers were provided slides, handouts, and telephone support to facilitate practice site training. They also completed human subjects training (http://www.citiprogram.org). Usual care sites received a short asthma overview and training regarding study procedures. Intervention sites received this training plus an additional 6 hours regarding the use and interpretation of the Asthma APGAR tools. The intervention was to be used in all patients with asthma, but data collection was limited to enrolled patients.
As designed in the protocol, at approximately 18 months into the study, all usual care sites were crossed over to the APGAR tools intervention. The crossover practices’ trainers attended a short second centralized training session that addressed only the intervention. The trainers returned to educate their practice members on the intervention.
Data Collection
Patient-reported outcomes were collected by questionnaires using validated instruments for asthma control and asthma-related quality of life: the ACT and the C-ACT17,18 for asthma control, and the AQLQ and PAQLQ for asthma-related quality of life.56 Data collected at baseline and 12 months were used in analyses to account for potential seasonal variations that might be present at other intervals. Questions regarding health care use were adapted from those in the National Health Interview Survey.57 Questionnaires were returned by mail to the central site. Nonrespondents received a mailed reminder in 10 days, a second copy of the questionnaire in 21 days, and a telephone call at 28 days.
Medical record data were used to assess the practice’s guideline adherence. Prestudy assessments have been published previously.6 Each enrolled patient’s medical records were obtained from his or her enrolling practice from 1 year before enrollment through study completion, and were copied and mailed to the central site for abstracting by trained nurse abstractors using a standardized form. The nurse abstractors evaluated documentation of asthma control assessment, factors affecting control (medication adherence, inhaler technique review, trigger assessment), patient self-management support (asthma action plan25,26), and the prescribing of both a quick relief and a maintenance medication.7,28 Nurse abstractors’ intrarater and interrater reliability were assessed repeatedly and maintained at 95% agreement or higher.58 Semistructured telephone exit interviews were completed for all sites to explore barriers to and facilitators of Asthma APGAR implementation. Interview questions are available on request.
Data Analysis
Because of the transition of usual care practices to the APGAR tools intervention, some patients’ practices changed from usual care to the intervention during their enrollment period. To account for this change, patients were divided into 3 groups: usual care patients (completed 12 months of care while their practice was in usual care arm), APGAR tools intervention patients (completed 12 months of care while their practice was in the intervention arm), and transition patients (visited their practice during both its usual care and intervention periods). Primary outcomes analyses were completed comparing only usual care and intervention patients on an intention-to-treat basis. Subanalyses are presented by age-group for some outcomes to illustrate differences that occurred across age-groups. Analyses for practice adherence to guidelines included all enrolled patients regardless of whether they made visits during the study period.
For all outcomes, we compared frequencies between intervention and usual care at 12 months after patient enrollment for patient outcomes and at the end of each practice’s period of enrollment in either the usual care or intervention study arm. For patient outcomes, we also assessed changes in frequency within study arms at baseline and 12 months using the χ2 and McNemar tests. Association of practice and patient characteristics with rates of asthma-related ED, urgent care, and hospital visits as well as the number of guideline elements assessed was tested using logistic regression analysis with a random effects term for practice sites to address any clustering effect. We used the S-Plus statistics package version 7.0.6 (Tibco Software) for all computations.
RESULTS
Overall, 1,066 patients were enrolled (Figure 2). Of those enrolled, 79.1% (843 patients) returned their baseline questionnaire and 68.0% (723 patients) returned their 12-month questionnaire; 65.0% (692 patients) returned both, allowing their data to be used for patient outcomes. On the basis of medical records data, respondents and nonrespondents differed only on response rates, which were lower in adult men compared with women (P = .0005).59 Medical records data were available for 1,063 (more than 99%) of the enrolled patients for assessment of practice outcomes.
Characteristics of the patients who returned their baseline questionnaire are shown in Table 1 by their practice’s randomization. Supplemental Table 1 (http://www.annfammed.org/content/16/2/100/suppl/DC1) displays demographics by randomization and age-group. In general, patients were white, had modest family income (51.7% less than $50,000 per year), had inadequate asthma control, were often overweight, and had considerable exposure to tobacco smoke. During the 6 months before enrollment, the percent of patients reporting an asthma-related ED, urgent care, or hospital visit did not differ by randomization status. The enrolled practices’ baseline adherence levels to asthma guidelines were very limited and have been described previously.6
Patient Outcomes
During the final 6-month period of enrollment, the proportion of patients reporting an asthma-related visit to the ED, urgent care, or hospital was a significant 51% lower in the APGAR tools intervention group compared with the usual care group (10.6% vs 20.9%, P = .004). Relative to the 6 months before study enrollment, the proportion declined in the intervention group (from 17.5% to 10.6%, P = .001) but increased in the usual care group (from 15.9% to 20.9%, P = .06) (Figure 3). Findings by age-group were similar (Supplemental Figure 4, http://www.annfammed.org/content/16/2/100/suppl/DC1). The difference remained significant in logistic regression models controlling for age-group; sex; race; income; ethnicity; the patient’s previous asthma-related ED, urgent care, or hospital visits; and random effects adjustment for practice, with an odds ratio of 2.1 (95% CI, 1.01 to 4.2, P = .04) (Supplemental Table 2, http://www.annfammed.org/content/16/2/100/suppl/DC1).
The proportion of patients with “in control” asthma at 12 months trended toward improvement in the APGAR tools intervention group as compared with the usual care group (54.3% vs 46.0%, P ≤.06). The proportion with controlled asthma increased significantly between baseline and 12 months in the former group (from 40.8% to 54.3%, P <.0001) but not in the latter group (from 42.6% to 46.0%, P = .86) (Figure 3).
Changes in asthma-related quality-of-life scores showed a trend toward higher scores at 12 months in the APGAR tools intervention group vs usual care group, with mean scores of 5.37 vs 5.14, respectively (P = .07), which became significant after adjusting for multiple factors including patients’ age, sex, race, and ethnicity, and a practice variable (P = .04). The proportion of patients with a gain of at least 0.5 points (minimally clinically significant change) between baseline and 12 months also showed a trend toward improvement in the APGAR tools intervention group vs usual care group (36.6% vs 27.0%, respectively, P = .07).
Practice Outcomes
The APGAR tools intervention practices had an improvement in adherence to nonmedication guideline elements by 12 months compared with the usual care practices (P <.001 for adherence to 3 or more elements of the guidelines) (Supplemental Table 3, http://www.annfammed.org/content/16/2/100/suppl/DC1). This difference between groups remained significant when controlling for multiple factors using random effects adjustment for practice difference; patients’ age, sex, race, and ethnicity; and baseline adherence to guideline elements, with an odds ratio of 3.89 (95% CI, 1.87 to 8.07, P <.001) (data not shown).
Medication prescribing improved in both groups, with increases in rates of prescription of daily asthma medication (Supplemental Table 4, http://www.annfammed.org/content/16/2/100/suppl/DC1). The intervention group had a smaller share of patients with inadequate asthma therapy (no daily controller or no short-acting β2 agonist) prescribed at 12 months compared with the usual care group (14.7% vs 8.2%, P = .03).
Exploratory Outcomes
Exit interviews with study coordinators and clinicians highlighted several barriers to and facilitators of implementing the Asthma APGAR tools (Table 2). Although practice change was reported as difficult, clinicians and practice staff who used the tools found them helpful. Both physicians and nursing staff reported that learning and determining how to implement the new system required additional time and practice, but they thought that it rapidly “streamlined” asthma visits. Implementing the Asthma APGAR tools became a priority and may have delayed working on other practice improvements for at least the first year of the study.
DISCUSSION
In a geographically and socioeconomically diverse group of family medicine and pediatric practices, introduction of the Asthma APGAR tools resulted in a significant decrease in asthma-related ED, urgent care, and hospital visits; a trend toward an increase in the proportion of patients whose asthma was controlled; and increased adherence to asthma guidelines. There was also a trend toward greater improvements in asthma-related quality-of-life scores among intervention patients. As expected, practice change was reported to be difficult, but the Asthma APGAR tools were perceived to be useful and efficient.
These tools are designed to be used in primary care. Use of the Asthma APGAR tools does not require the addition of nurse managers,45 community health workers,28 or practice facilitators.27 Introduction of the tools does require education of practice staff or changes in clinician behavior, which remain barriers to implementation. Introducing a new tool or care system may also have unintended consequences, such as replacing some existing activity with those for the new tool.60 The enrolled practices did comment that they were unable to begin other new quality improvement projects for any medical condition for at least the first 6 to 12 months of working to implement the Asthma APGAR tools, but did not believe they eliminated any preexisting practice activities.
The Asthma APGAR tools provide a broad array of information beyond what is included with most control assessments and beyond what is collected in most primary care asthma visits.6,23,31 Unlike other assessments for asthma control, these tools highlight multiple reasons for inadequate control, including unrecognized or unavoidable triggers, incomplete adherence to asthma medications, and patient perception of response to therapy that may be due to poor inhaler technique. This enhanced decision support appears to improve patient and practice outcomes, as shown in this study. The favorable changes in asthma control in our study were not accompanied by a statistically significant change in asthma-related quality-of-life scores. This discrepancy likely highlights the different aspect of a patient’s life assessed by the domains of asthma control and the broader assessment of living with asthma or asthma-related quality-of-life. Use of the Asthma APGAR system was designed to address asthma control and may not improve overall asthma-related quality of life.
Time pressures and unwillingness to try a new tool have also been barriers reported in many studies of practice improvement programs.34 The ability to incorporate the tool into the electronic health record (EHR) as a facilitator is encouraging. Unfortunately, our practices reported that embedding the tools was often expensive and slow, especially if the practice or health system used one of the larger, more established systems with long wait times and high prices charged for these changes. Cave and colleagues29 are currently studying the impact of embedding asthma care tools into an EHR and may provide additional data and support for more rapid incorporation of tools.
Lenney and colleagues61 highlighted the complex nature of asthma care, health care, and similar barriers of incorporating change into care processes. Perhaps combining our work with that of Cloutier et al,62 who were able to improve pediatricians’ readiness to change, would reduce this barrier.
Although studies of asthma quality improvement programs are common, few use randomized trial designs, include patients of all ages, or report patient outcomes. A randomized trial in Quebec primary care practices63 used a paper tool to guide physicians’ assessment of asthma and demonstrated a 39% decrease in ED and hospital asthma care. In Spain, Román-Rodríguez et al30 reported on a randomized controlled trial that “slightly” increased use of asthma and COPD patient assessment tools after physician education, but did not assess patient outcomes. Mold et al27 compared implementation of asthma care modifications using practice facilitators, a learning collaborative, and both combined in family medicine practices. Practice facilitator use increased practices’ documentation of asthma severity and asthma control, but patient outcomes were not assessed. Studying only children, Lozano and colleagues45 reported a 36% decrease in exacerbation rates by using nurse managers. Home visits by community health workers increased symptom-free days by 2 during the 2-week assessment period but did not change children’s urgent heath care use.28
Other reports are limited to children and use only preprogram and postprogram comparisons.36,64–73 Ragazzi et al36 were able to improve 4 of 8 care processes, but in exit interviews, staff reported those improvements were unlikely to be sustained without ongoing external support. Adding nurse managers, plus “intense interaction” with families and home visits, both Woods et al72 and Lob et al69 demonstrated a pre-to-post decrease in rates of asthma ED visits (68% and 70%, respectively), whereas Bunik et al64 and Vernacchio et al68 used similar methods but found no improved outcomes, with improvement only in documentation of care processes. Okelo and colleagues12 highlighted the lack of studies that assessed patient outcomes even when introducing intensive external practice support. In contrast, we found that the Asthma APGAR tools improved patient outcomes and requires little additional practice support.
Our study has both strengths and limitations. The practices we included represent a diverse segment of the US population based on geography, race, ethnicity, and socioeconomic status. Study sites were primarily smaller, non–inner city practices. The results may not be generalizable to large health care systems. Such systems, however, often have greater support for quality improvement initiatives, including the ability to embed the tools into the EHR.
We included only patients aged 5 to 45 years, which may limit the generalizability of the results to older adults with asthma. At the time of the study inception, concerns regarding misdiagnosis of COPD were well spelled out in the literature, and early discussions of asthma–COPD overlap further highlighted the potential differences in treatments and patient assessment methods in adults older than 45 years.52
Although the pragmatic nature of our trial enhances the generalizability of its results, it may also have reduced survey response rates because of limited interaction of patients with the research team, and may have achieved less practice change than could be achieved with greater practice support or intervention.27 We did not specifically evaluate a practice’s readiness to change. Doing so may be a very important precursor to successful widespread implementation. Learning how to enhance readiness to change and acceptance of change may be even more important.
Our primary outcomes of health care use, asthma control, and asthma-related quality of life are patient reported and therefore may be subject to recall bias. In addition, outcomes can be obtained only from patients who completed questionnaires; therefore, responder bias is possible. Differences in how practices document may also introduce bias, given that the EHR is used to measure some outcomes. Our cluster-randomized study design helped reduce the risk of these bias types when comparing intervention and usual care practices. In addition, the lack of significant differences between respondents and nonrespondents on characteristics at baseline reduces the concerns about our moderate response rates. The presence and frequency of asthma exacerbations were patient reported. Because of limitations in the enrolled clinics’ electronic and billing systems, we were unable to ensure capture of information on all use of systemic steroid bursts74 or confirm all ED, urgent care, and hospital visits, which often occurred outside of the practice site. Limiting our practices to those with prescription-fulfillment data or those belonging to large health systems allowing EHR confirmation of most ED and hospital visits would have limited the types of practices we could have enrolled, decreasing generalizability to our target rural and smaller practices.
In conclusion, introduction of the Asthma APGAR tools improved asthma control; reduced asthma-related ED, urgent care, and hospital visits; and improved practices’ asthma guideline adherence, suggesting that these tools are an effective practice addition for asthma management in the primary care setting. Future studies will need to assess the impact of introducing the Asthma APGAR tools into the primary care practices of large health care systems.
Footnotes
Conflicts of interest: Barbara P. Yawn served on asthma and COPD advisory boards for Boehringer Ingelheim, GlaxoSmithKline, Novartis, and Teva. Young Juhn received support for asthma research from Genentech. All other authors report no conflicts of interest.
Funding support: This work was supported by AHRQ grant R01HS 008745, Barbara P. Yawn, Principal Investigator. Barbara P. Yawn has also received research grants from NIH and PCORI for asthma and COPD research. Young Juhn received support for asthma research from an NIH-funded R01 grant (R01 HL126667), an R21 grant (R21AI116839-01), the T. Denny Sanford Pediatric Collaborative Research Fund, and a Scholarly Clinician Award from the Mayo Foundation. Matthew A. Rank received support for asthma research from the AHRQ (grant R03 HS022126).
Previous presentations: Preliminary results were presented as a late-breaking poster at the American Academy of Allergy, Asthma & Immunology (AAAAI) meeting; March 2017; Atlanta, Georgia.
Trial registration: NCT01446315 at clinicaltrials.gov.
Supplementary materials: Available at http://www.AnnFamMed.org/content/16/2/100/suppl/DC1/.
- Received for publication April 24, 2017.
- Revision received September 19, 2017.
- Accepted for publication September 26, 2017.
- © 2018 Annals of Family Medicine, Inc.