Research ArticleOriginal Research

Adoption, Reach, Implementation, and Maintenance of a Behavioral and Mental Health Assessment in Primary Care

Alex H. Krist, Siobhan M. Phillips, Roy T. Sabo, Bijal A. Balasubramanian, Suzanne Heurtin-Roberts, Marcia G. Ory, Sallie Beth Johnson, Sherri N. Sheinfeld-Gorin, Paul A. Estabrooks, Debra P. Ritzwoller and Russell E. Glasgow
The Annals of Family Medicine November 2014, 12 (6) 525-533; DOI: https://doi.org/10.1370/afm.1710

Abstract

PURPOSE Guidelines recommend screening patients for unhealthy behaviors and mental health concerns. Health risk assessments can systematically identify patient needs and trigger care. This study seeks to evaluate whether primary care practices can routinely implement such assessments into routine care.

METHODS As part of a cluster-randomized pragmatic trial, 9 diverse primary care practices implemented My Own Health Report (MOHR)—an electronic or paper-based health behavior and mental health assessment and feedback system paired with counseling and goal setting. We observed how practices integrated MOHR into their workflows, what additional practice staff time it required, and what percentage of patients completed a MOHR assessment (Reach).

RESULTS Most practices approached (60%) agreed to adopt MOHR. How they implemented MOHR depended on practice resources, informatics capacity, and patient characteristics. Three practices mailed patients invitations to complete MOHR on the Web, 1 called patients and completed MOHR over the telephone, 1 had patients complete MOHR on paper in the office, and 4 had staff help patients complete MOHR on the Web in the office. Overall, 3,591 patients were approached and 1,782 completed MOHR (Reach = 49.6%). Reach varied by implementation strategy with higher reach when MOHR was completed by staff than by patients (71.2% vs 30.2%, P <.001). No practices were able to sustain the complete MOHR assessment without adaptations after study completion. Fielding MOHR increased staff and clinician time an average of 28 minutes per visit.

CONCLUSIONS Primary care practices can implement health behavior and mental health assessments, but counseling patients effectively requires effort. Practices will need more support to implement and sustain assessments.

INTRODUCTION

A substantial burden of unhealthy behaviors leads to chronic diseases and mental health disorders among patients seen in primary care settings.1 Health risk assessments (HRAs) can help identify and address factors that place a person at enhanced risk for morbidity or mortality. Primary care is a promising setting to conduct HRAs because risk identification can be linked to assistance from clinicians who have a longstanding and trusting relationship with the patient.2 Unfortunately, many primary care practices are overwhelmed by competing demands, and typical office visits provide little time to address health risk information.3,4

As early as 1970, clinician manuals promoted sample HRA questionnaires, risk computations, and feedback strategies.5 While HRAs were not widely adopted by the medical profession, they proliferated in workplaces and community-based programs.6,7 In these settings, HRAs improved health indicators such as blood pressure, weight, physical activity, and general health status.8,9 A critical finding was that merely administering an HRA questionnaire does not produce behavior change.10–12 Comprehensive, well-resourced follow-up is essential to help individuals gain the skills they needed to change health habits.

Until recently, primary care settings conducted HRAs infrequently. One survey found that fewer than 20% of practices routinely administered HRAs, and these tended to be larger practices affiliated with health care systems.13 However, the Affordable Care Act established a Medicare Annual Wellness Visit that mandates the inclusion of an HRA and a personal prevention plan.14 The mandated HRA (1) may be completed before or as part of a visit; (2) must identify chronic diseases, injury risks, modifiable risk factors, and urgent health needs; and (3) may be furnished through an interactive telephonic or Web-based program.15

Despite these new policies, little is known about effective ways for practices to implement an HRA and the extent to which practices can routinely engage patients. This article reports on the feasibility of conducting behavior and mental health assessments in a diverse sample of real world primary care settings. Specifically, we report on practices’ willingness to adopt the My Own Health Report (MOHR) tool, how practices implemented MOHR, the percentages and characteristics of patients completing a MOHR assessment based on implementation strategy, and whether practices maintained use of MOHR after study completion.

METHODS

The MOHR study is a cluster-randomized, mixed methods implementation trial. The design is purposefully pragmatic, allowing local tailoring of implementation to ensure that findings reflect real-life practice. Nine practice pairs were randomized with allocation concealment to implement MOHR (early implementation) or to provide usual care with a delayed option to implement MOHR (delayed implementation). Intervention practices were encouraged and helped to adapt their implementation workflow to fit local needs. The MOHR content and research data collection activities were standardized. Study methods and pilot data have been reported previously.16,17

The study was approved by the Virginia Commonwealth University (VCU) (#HM12746), (#12-0017900), and 5 other participating institutional review boards.

Setting

The MOHR study was a collaboration of 8 nationally distributed academic partners that manage practice based research networks (PBRNs) or participate in the Cancer Prevention and Control Research Network (CPCRN).18,19 Academic partners and study practices were purposefully selected to represent the spectrum of primary care practice type, ownership, location, electronic health record infrastructure, and patient population. Virginia Commonwealth University (VCU) served as the national coordination center. VCU recruited and managed 2 matched pairs of primary care practices, and the other academic centers recruited and managed 1 each, for a total of 18 intervention and control practices. If a participating practice was part of a health system or federally qualified health center, it was paired with another practice from the same system or center. This report focuses on the experiences of the 9 intervention practices.

Most practices were small to medium size, with 1 to 6 clinicians and an annual practice patient panel of 1,500 to 10,000 adults (Table 1). One practice did not have an electronic health record (EHR) and 2 did not offer patients a personal health record (PHR). No practice systematically offered a health risk assessment, instead relying on clinicians to ask about health behaviors and psychosocial issues as part of care.

View this table:
Table 1

Characteristics of Early Intervention Practices Fielding the My Own Health Report (MOHR) Assessment

Intervention

Early intervention practices were asked to adopt, implement, and field MOHR routinely to a minimum of 300 patients between March and December 2013. Practices were encouraged to continue using MOHR after study completion. MOHR is available electronically at http://www.MyOwnHealthReport.org or on paper in both English and Spanish. While we attempted to fully integrate the electronic version of MOHR into practices’ EHRs and PHRs, this was not feasible due to a variety of constraints.

The MOHR assessment asked patients 17 health behavior and psychosocial risk screening questions and 6 demographic questions previously identified by a consensus panel as brief, practical, and evidence-based.20,21 The screening issues MOHR assessed were all recommended by the US Preventive Services Task Force with the exception of sleep, quality of life, and anxiety. In response to positive depression, anxiety, alcohol, and drug screening questions, MOHR asked the Patient Health Questionnaire (PHQ9),22 Generalized Anxiety Disorder (GAD) questionnaire,23,24 Alcohol Use Disorders Identification Test (AUDIT-C),25 and the Drug Abuse Screening Test (DAST-10),26 respectively.

The electronic version of MOHR scored and categorized patients’ responses as being of “no concern,” “some concern,” or “high concern.” For responses with some or high concern, patients were asked if they were ready to change and/or discuss the topic with their clinician.27–30 If patients selected more than 1 topic to change and/or discuss, they were asked to identify the most important topic. MOHR then provided patients a summary containing motivational feedback, initial improvement steps, and space to create 3 “SMART” goals (ie, goals that are specific, measurable, achievable, realistic, and timely).31,32 A clinician summary was automatically faxed to the practice to be uploaded into the EHR. The paper version of MOHR replicated this process, but required manual scoring and lacked follow-up questions.

Practices were asked to decide which patients would be invited to complete MOHR, when and where MOHR would be completed, whether they would use the electronic or paper version, and who would counsel patients in response to summaries. Some practices received research team staff support to help perform tasks related to implementing MOHR. Practice champions and academic representatives were offered Web-based training and biweekly learning collaboratives to refine the MOHR tools, discuss implementation strategies, and share challenges and successes.33,34 Practice champions from 6 practices attended the Web-based training. Learning collaboratives were primarily attended by academic representatives, who shared experiences with their practices.

Data Sources

Data were collected from PBRN and CPCRN records, field and learning collaborative notes, the MOHR tool, and practice interviews. From PBRN and CPCRN records, we identified characteristics of practices approached for participation. From learning collaborative and field notes, we prospectively tracked each practice’s implementation strategy. Field notes included number of patients approached to complete MOHR. The electronic and paper version of MOHR included date and time information for patients completing MOHR. For each practice, after 100, 200, and 300 patients in the practice had completed MOHR, the practice was interviewed about their implementation workflow as well as the time and staff required to complete tasks.

Outcomes Assessment

We used a pragmatic application of the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework to evaluate outcomes.35–37 This report focuses on the adoption, implementation, reach, and maintenance of MOHR. We defined adoption as the percentage of practices approached for study participation who agreed to participate, with descriptions of why practices declined; implementation as a description of how practices integrated MOHR into their workflow and the time and staff needed to carry out implementation steps; reach as the proportion of eligible patients approached who completed a MOHR assessment; and maintenance as whether early intervention practices continued to use MOHR after the study.37–40

For reach, we assessed the overall reach for the study and for each practice as well as tracking variation by week, by implementation strategy, and by practice and patient characteristics. We calculated the variability in reach using a generalized linear random effect model including a random practice effect. We used a logit link and binomial distribution. We calculated the relationship between reach and implementation strategy, practice characteristics, and patient demographics using logistic regression. We used SAS version 9.4 (SAS Institute) for all analyses, with the PROC GLIMMIX procedure for random effects modeling and PROC LOGISTIC for logistic regression models.

RESULTS

Adoption

Overall, 30 practices were approached to recruit 18 study sites, for a 60% adoption rate. Among 7 of 9 practice pairs, the first 2 practices approached agreed to participate. The local teams had to approach 10 and 2 practices before recruiting practice 6 and 9, respectively. These 12 practices declined because they were doing other studies, were concerned about workload, or routinely did health risk assessments.

Implementation

Implementation strategies of the early intervention practices are detailed in Table 2. Most (8 of 9) opted to use the Web-based tool, expressing interest in the automated scoring, feedback, and follow-up. Practice 2 elected to use the paper-based version due to concerns about patients’ Web access. Four practices (practices 1, 3, 4, and 5) mailed invitations for patients to complete MOHR at home online before a visit. Due to unusually low response rates, practice 3 converted to an internal nurse-staffed calling center to contact patients by telephone, ask the questions, and enter responses on the MOHR website. The remaining practices (practices 6–9) had concerns similar to those of practice 2 and had practice or academic staff administer MOHR in the lobby or exam room and enter patient responses. With the exception of practice 9, which initially attempted to administer MOHR to all patients, practices targeted a combination of patients presenting for wellness and/or chronic care visits, representing fewer than a third of patients seen daily.

View this table:
Table 2

MOHR Implementation Strategy Developed by Practices and Time to Complete Additional Tasks

All but 1 practice relied on the clinician to counsel patients during office visits in response to MOHR-identified topics. Clinicians reported spending an average of 5 to 17 minutes per patient in counseling. No practice used ancillary staff (eg, mental health or health behavior counselors) for counseling. Six practices reported no additional patient follow-up. The remaining practices had 10% to 20% of patients return for a follow-up office visit. Practice 6 reported delays delivering the summaries to clinicians for visits and accordingly provided no additional clinician counseling at visits or follow-up (Table 2).

With the exception of practice 2, no practice was able to field MOHR without support from the academic team or external health system resources. Support included querying appointment records to identify eligible patients (practices 4 and 5), mailing MOHR invitations (practices 1, 4, and 5), identifying eligible patients before appointments (practices 6–9), or administering MOHR (practices 6–9) (Table 2). Fielding MOHR, including clinician counseling, resulted in an average increase of 28 minutes per office visit (range 16–31 minutes). In our study, 17 minutes of tasks were performed by practice staff and 11 minutes by the academic staff.

Reach

Of 3,591 patients offered MOHR, 1,782 completed the assessment, for an overall reach of 49.6% (Table 3). We observed significant variation in reach (P <.001) based on implementation approach: 2.6% to 45.6% for mailed invitations (mean 26.2%), 64.2% for phone completion, 43.9% when completed in the office by the patient on paper, and 56.8% to 94.4% (mean 73.7%) when completed with staff assistance in the office on the Web. While this study was not designed to compare the reach of different implementation strategies, in general, completion rates were higher when completed by practice or research staff than by patients (71.2% vs 30.3%; P <.001). This is exemplified by the dramatic increase in reach when practice 3 converted from mailed invitation to phone completion (Figure 1).

View this table:
Table 3

Total Number of Patients Who Were Offered and Who Completed MOHR by Practice and Implementation Strategy

Figure 1
Figure 1

Number of patients completing the MOHR assessment each week by practice.

Maintenance

Among the early intervention practices, none continued to use MOHR after study completion. Six practices have embedded elements of MOHR into their patient portal or pre-visit patient paperwork as part of a standardized HRA process.

DISCUSSION

A diverse range of primary care practices were able to implement and systematically field a comprehensive behavior and mental health assessment with their patients. Practices were willing to do the MOHR assessment as exemplified by the high study adoption rate. For many practices, MOHR aligned with ongoing initiatives focused on patient-centered care and population health management.

By tailoring and integrating MOHR into their workflow, practices got a substantial proportion of patients to complete the assessment. Our observed reach of approached patients (49.6%) was double the HRA completion rates previously published by large health systems (22.4%) and on par with worksite completion rates coupled with economic incentives (40% to 64%).41,42 Diverse practices were successful in getting patients of all ethnic, racial, and socio-economic levels to participate in MOHR. The value of coupling MOHR with primary care visits was supported by the finding that patients had an average of 6 concerning behaviors and mental health issues. These findings are fully addressed in a separate manuscript.43

While many have suggested that advances in health information technology may help to automate HRA administration and relieve practice resource burdens,2,15 we found that some element of patient assistance more than doubled MOHR completion rates. As expected, providing assistance required substantially more staff time. Whether practices would be able to maintain this level of support over time is unclear.

Asking practices to conduct a behavior and mental health assessment like MOHR is not an insignificant request. Despite their interest, most practices lacked capacity and infrastructure to field MOHR independently and required external assistance. Accordingly, no practices chose to maintain MOHR after study completion when academic staff was no longer available. Most sites, however, integrated elements of MOHR into their workflow. Still more substantial practice transformation will be necessary to integrate MOHR-like assessments routinely into primary care, and current incentives, such as the mandate to include HRAs as part of wellness care, are insufficient to facilitate this practice change.14,15

HRAs hold great promise for primary care. The high prevalence of unhealthy behaviors and mental health concerns that we observed is supported in the literature.1,43 Once such concerns are identified, primary care clinicians are well suited to initiate the counseling process. A large body of evidence demonstrates, however, that without intensive counseling, follow-up activities, and monitoring of progress toward achievement of health improvement goals (collectively termed an “HRA-plus process”), merely collecting HRA data will not change patient outcomes.10,44–48 While the greatest practice time investment was clinician counseling, it is interesting that no practices used existing co-located health behavior counselors or referred patients to community resources. Many psychosocial and health behavioral counseling interventions require more intensive counseling over an extended period of time than primary care clinicians can provide. More effectively using co-located multidisciplinary teams and developing local community resources to refer patients for intensive counseling will be essential to help patients make difficult life changes.49,50

This study has several limitations. We were not able to integrate MOHR fully into the practices’ EHRs, despite initial plans to do so. While the participating practices were very diverse, they may not have been fully representative of primary care nationally. We had a high proportion of Federally Qualified Health Centers that served more disadvantaged populations. MOHR was designed as a comprehensive psychosocial and health behavior assessment, resulting in more risk assessment questions and positive screens for practices to address. The length likely influenced our findings, and shorter assessments may prove to be more effective. Finally, this initial study addressed only an initial offering of MOHR to patients and did not assess the longitudinal management of patients over time. Future papers will address the patient experience (receipt of counseling and goal setting) and the costs of implementing MOHR.

While the MOHR study offers an initial understanding of the HRA-plus model for behavioral and mental health in primary care, additional study is needed. Further efforts are necessary for practices to gain the capacity to adopt the HRA-plus process. Practices need HRA tools that can be easily integrated into their workflows with content locally tailored to specific patient populations, that can be incorporated into existing EHRs and patient portals, and that provide automated feedback and support for both patients and clinicians. The optimal content and length of HRAs needs to be determined to maximize their feasibility and benefit. Practices need adequate resources to invest in infrastructure and staff to help patients complete an HRA and provide counseling. Future efforts will need to explore the roles of co-located health behavioral and mental health personnel and the integration of care between primary care practices and existing community resources.49,51 Merely mandating that HRAs be added to an already packed wellness visit simply increases the chances that that practices will do this poorly or not at all.

Acknowledgments

We would like to thank the research teams for their valuable efforts including Melissa Hayes, Steve Mitchell, Mark Greenwald, Mark Kelly, Rodger Kessler, Jill Arklind, Joseph Carroll, Christine Nelson, John Heintzman, Maria Fernandez, Sarah Krasny, Kayla Fair, Julie Ribardo, Beth A. Glenn, Dylan Roby, Jennifer Leeman, Alexis Moore, and Catherine Rohweder. We would also like to thank Eric Peele and RTI International for helping us to create and program the MOHR tool.

Most of all we would like to thank our practices for their partnership, insights and hard work: Vienna Primary and Preventive Medicine, Little Falls Family Practice, Charles City Regional Health Services, Chester Family MedCare, Carilion Family Medicine – Roanoke Salem, Carilion Family Medicine – Southeast, Berlin Family Health, Milton Family Practice, Humbolt Open Door Clinic, McKinleyville Community Health Center, St. Johns’ Well Child and Family Center, Spring Branch Community Health Center – Pitner Clinic, Spring Branch Community Health Center – Hillendahl Clinic, HealthPoint Community Health Centers – Navasota, HealthPoint Community Health Centers – Hearne, Murfeesboro Clinic, and Snow Hill Clinic.

Footnotes

  • Conflicts of interest: authors report none.

  • Funding support: Funding for the MOHR project was provided by the National Cancer Institute, Agency for Healthcare Research and Quality, Office of Behavioral and Social Sciences Research, and National Center for Advancing Translational Sciences (CTSA Grant Number ULTR00058). The opinions expressed in this manuscript are those of the authors and do not necessarily reflect those of the funders.

  • Trial Registration Clinicaltrials.gov identifier: NCT01825746

  • Received for publication March 26, 2014.
  • Revision received August 1, 2014.
  • Accepted for publication August 19, 2014.

REFERENCES

  1. ↵
    1. Goldstein MG,
    2. Whitlock EP,
    3. DePue J
    ; Planning Committee of the Addressing Multiple Behavioral Risk Factors in Primary Care Project. Multiple behavioral risk factor interventions in primary care. Summary of research evidence. Am J Prev Med. 2004;27(2)(Suppl):61–79.
    OpenUrlCrossRefPubMed
  2. ↵
    Agency for Healthcare Research and Quality. Health assessments in primary care: a how-to guide for clinicians and staff. http://www.ahrq.gov/professionals/prevention-chronic-care/improve/system/health-assessments/index.html. Published September 2013. Accessed February, 2014.
  3. ↵
    1. Stange KC,
    2. Flocke SA,
    3. Goodwin MA,
    4. Kelly RB,
    5. Zyzanski SJ
    . Direct observation of rates of preventive service delivery in community family practice. Prev Med. 2000;31(2 Pt 1):167–176.
    OpenUrlCrossRefPubMed
  4. ↵
    1. Stange KC,
    2. Zyzanski SJ,
    3. Jaén CR,
    4. et al
    . Illuminating the ‘black box’. A description of 4454 patient visits to 138 family physicians. J Fam Pract. 1998;46(5):377–389.
    OpenUrlPubMed
  5. ↵
    1. Robbins LC,
    2. Hall JH
    . How to Practice Prospective Medicine. Indianapolis, IN: Methodist Hospital of Indiana; 1970.
  6. ↵
    1. Schoenbach VJ,
    2. Wagner EH,
    3. Beery WL
    . Health risk appraisal: review of evidence for effectiveness. Health Serv Res. 1987;22(4):553–580.
    OpenUrlPubMed
  7. ↵
    1. Schoenbach VJ
    . Appraising health risk appraisal. Am J Public Health. 1987;77(4):409–411.
    OpenUrlCrossRefPubMed
  8. ↵
    1. Shekelle PG,
    2. Tucker JS,
    3. Maglione M,
    4. Morton SC,
    5. Roth E,
    6. Chao B
    . Health Risk Appraisals and Medicare. Santa Monica, CA: RAND Corporation; 2003.
  9. ↵
    Guide to Community Preventive Services. http://www.thecommunityguide.org/. Published 2014. Accessed February, 2014.
  10. ↵
    1. Soler RE,
    2. Leeks KD,
    3. Razi S,
    4. et al
    ; Task Force on Community Preventive Services. A systematic review of selected interventions for work-site health promotion. The assessment of health risks with feedback. Am J Prev Med. 2010;38(2)(Suppl): S237–S262.
    OpenUrlCrossRefPubMed
    1. Anderson DR,
    2. Staufacker MJ
    . The impact of worksite-based health risk appraisal on health-related outcomes: a review of the literature. Am J Health Promot. 1996;10(6):499–508.
    OpenUrlCrossRefPubMed
  12. ↵
    1. Oremus M,
    2. Hammill A,
    3. Raina P
    ; McMaster University Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality, US Department of Health and Human Services. Health risk appraisal technology assessment report. http://www.cms.gov/Medicare/Coverage/DeterminationProcess/downloads/id79ta.pdf. Published July 6, 2011. Accessed February, 2014.
  13. ↵
    1. Halpin HA,
    2. McMenamin SB,
    3. Schmittdiel J,
    4. et al
    . The routine use of health risk appraisals: results from a national study of physician organizations. Am J Health Promot. 2005;20(1):34–38.
    OpenUrlCrossRefPubMed
  14. ↵The Patient Protection and Affordable Care Act, 42 USC §18001 (2010).
  15. ↵
    1. Goetzel RZ,
    2. Staley P,
    3. Ogden L,
    4. et al
    ; US Department of Health and Human Services, Centers for Disease Control and Prevention. A framework for patient-centered health risk assessments - providing health promotion and disease prevention services to Medicare beneficiaries. http://www.cdc.gov/policy/opth/hra/. Published 2011. Accessed February, 2014.
  16. ↵
    1. Krist AH,
    2. Glenn BA,
    3. Glasgow RE,
    4. et al
    ; MOHR Study Group. Designing a valid randomized pragmatic primary care implementation trial: the my own health report (MOHR) project. Implement Sci. 2013;8:73.
    OpenUrlCrossRefPubMed
  17. ↵
    1. Rodriguez HP,
    2. Glenn BA,
    3. Olmos TT,
    4. et al
    . Real-world implementation and outcomes of health behavior and mental health assessment. J Am Board Fam Med. 2014;27(3):356–366.
    OpenUrlAbstract/FREE Full Text
  18. ↵
    Agency for Healthcare Research and Quality. Primary care practice-based research networks (PBRNs). http://pbrn.ahrq.gov. Accessed January, 2013.
  19. ↵
    National Cancer Institute. Cancer Prevention and Control Research Network (CPCRN). http://www.cpcrn.org/. Accessed January, 2013.
  20. ↵
    1. Estabrooks PA,
    2. Boyle M,
    3. Emmons KM,
    4. et al
    . Harmonized patient-reported data elements in the electronic health record: supporting meaningful use by primary care action on health behaviors and key psychosocial factors. J Am Med Inform Assoc. 2012;19(4):575–582.
    OpenUrlCrossRefPubMed
  21. ↵
    Department of Health and Human Services, National Institute of Health, Office of Behavioral and Social Sciences Research, National Cancer Institute. Identifying core behavioral and psychosocial data elements for the electronic health record. 2011; http://conferences.thehillgroup.com/OBSSR/EHR2011/resources.html.
  22. ↵
    1. Kroenke K,
    2. Spitzer RL,
    3. Williams JB
    . The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–613.
    OpenUrlCrossRefPubMed
  23. ↵
    1. Swinson RP
    . The GAD-7 scale was accurate for diagnosing generalised anxiety disorder. Evid Based Med. 2006;11(6):184.
    OpenUrlFREE Full Text
  24. ↵
    1. Spitzer RL,
    2. Kroenke K,
    3. Williams JB,
    4. Löwe B
    . A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–1097.
    OpenUrlCrossRefPubMed
  25. ↵
    1. Bush K,
    2. Kivlahan DR,
    3. McDonell MB,
    4. Fihn SD,
    5. Bradley KA
    . The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med. 1998;158(16):1789–1795.
    OpenUrlCrossRefPubMed
  26. ↵
    1. Skinner HA
    . The drug abuse screening test. Addict Behav. 1982;7(4): 363–371.
    OpenUrlCrossRefPubMed
  27. ↵
    1. Whitlock EP,
    2. Orleans CT,
    3. Pender N,
    4. Allan J
    . Evaluating primary care behavioral counseling interventions: an evidence-based approach. Am J Prev Med. 2002;22(4):267–284.
    OpenUrlCrossRefPubMed
    1. Woolf SH,
    2. Jonas S,
    3. Kaplan-Liss E
    1. Glasgow RE,
    2. Goldstein MG
    . Introduction to the principles of health behavior change. In: Woolf SH, Jonas S, Kaplan-Liss E, eds. Health Promotion and Disease Prevention in Clinical Practice. 2nd ed. Philadelphia, PA: Lippincott, Williams & Wilkin; 2007: 129–147.
    1. Krist AH,
    2. Woolf SH,
    3. Frazier CO,
    4. et al
    . An electronic linkage system for health behavior counseling effect on delivery of the 5A’s. Am J Prev Med. 2008;35(5)(Suppl): S350–S358.
    OpenUrlCrossRefPubMed
  30. ↵
    1. Glasgow RE,
    2. Davis CL,
    3. Funnell MM,
    4. Beck A
    . Implementing practical interventions to support chronic illness self-management. Jt Comm J Qual Saf. 2003;29(11):563–574.
    OpenUrlPubMed
  31. ↵
    1. Croteau J,
    2. Ryan D
    . Acheiving your SMART health goals. BeWell@Stanford. 2013; http://bewell.stanford.edu/smart-goals. Accessed Jan, 2013.
  32. ↵
    1. O’Neil J
    . SMART Goals, SMART Schools. Educ Leadersh. 2000;57(5): 46–50.
    OpenUrl
  33. ↵
    1. Woodman RW,
    2. Pasmore WA
    1. Porras JI,
    2. Robertson PJ
    . Organizational Development Theory: A Typology and Evaluation. In: Woodman RW, Pasmore WA, eds. Research in Organizational Change and Development. Greenwich, CT: JAI Press; 1987:1–57.
  34. ↵
    1. Sheinfeld Gorin S,
    2. Krist AH
    . Using MOHR for behavior change: a webinar for providers. http://healthpolicy.ucla.edu/mohr. Published March 2013. Accessed February 2014.
  35. ↵
    1. Glasgow RE,
    2. Vogt TM,
    3. Boles SM
    . Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327.
    OpenUrlCrossRefPubMed
    1. Glasgow RE,
    2. Bull SS,
    3. Gillette C,
    4. Klesges LM,
    5. Dzewaltowski DA
    . Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med. 2002;23(1):62–69.
    OpenUrlCrossRefPubMed
  37. ↵
    Reach Effectiveness Adoption Implementation Maintenance (RE-AIM). 2012; http://re-aim.org/. Accessed August, 2012.
    1. Proctor E,
    2. Silmere H,
    3. Raghavan R,
    4. et al
    . Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.
    OpenUrlCrossRefPubMed
    1. Glasgow RE,
    2. Klesges LM,
    3. Dzewaltowski DA,
    4. Estabrooks PA,
    5. Vogt TM
    . Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res. 2006;21(5):688–694.
    OpenUrlAbstract/FREE Full Text
  40. ↵
    1. Gaglio B,
    2. Shoup JA,
    3. Glasgow RE
    . The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013;103(6):e38–e46.
    OpenUrlPubMed
  41. ↵
    1. Buist DS,
    2. Knight Ross N,
    3. Reid RJ,
    4. Grossman DC
    . Electronic health risk assessment adoption in an integrated healthcare system. Am J Manag Care. 2014;20(1):62–69.
    OpenUrlPubMed
  42. ↵
    1. Haisley E,
    2. Volpp KG,
    3. Pellathy T,
    4. Loewenstein G
    . The impact of alternative incentive schemes on completion of health risk assessments. Am J Health Promot. 2012;26(3):184–188.
    OpenUrlCrossRefPubMed
  43. ↵
    1. Phillips SM,
    2. Glasgow RE,
    3. Heurtin-Roberts S,
    4. et al
    . Health risk factors in primary care. In press.
  44. ↵
    U.S. Preventive Services Task Force. Behavioral counseling to promote a healthy diet and physical activity for cardiovascular disease prevention in adults 2012; http://www.uspreventiveservicestaskforce.org/uspstf/uspsphys.htm. Accessed Feb, 2014.
    1. Barton M
    ; US Preventive Services Task Force. Screening for obesity in children and adolescents: US Preventive Services Task Force recommendation statement. Pediatrics.2010;125(2):361–367.
    OpenUrlAbstract/FREE Full Text
    1. Moyer VA
    ; US Preventive Services Task Force. Screening for and management of obesity in adults: US Preventive Services Task Force recommendation statement. Ann Intern Med. 2012;157(5):373–378.
    OpenUrlPubMed
    1. Baghelai C,
    2. Nelkin VS,
    3. Miller TR
    ; Agency for Healthcare Research and Quality. Health risk appraisals in primary care: current knowledge and potential applications to improve preventive services and chronic care. http://www.ahrq.gov/research/findings/final-reports/environmental-scan/index.html. Published July 13, 2009. Accessed February, 2014.
  48. ↵
    US Preventive Services Task Force. Screening for depression in adults. 2009; http://www.uspreventiveservicestaskforce.org/uspstf/uspsaddepr.htm. Published December 2009. Accessed February, 2014.
  49. ↵
    1. Krist AH,
    2. Shenson D,
    3. Woolf SH,
    4. et al
    . A framework to integrate existing clinical and community delivery systems for preventive care. Am J Prev Med In press.
  50. ↵
    1. Etz RS,
    2. Cohen DJ,
    3. Woolf SH,
    4. et al
    . Bridging primary care practices and communities to promote healthy behaviors. Am J Prev Med. 2008;35(5)(Suppl):S390–S397.
    OpenUrlCrossRefPubMed
  51. ↵
    1. Woolf SH,
    2. Krist AH,
    3. Rothemich SF
    . Joining Hands: Partnerships Between Physicians and the Community in the Delivery of Preventive Care. Washington, DC: Center for American Progress; 2006.
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Get Permissions

Jump to section

Subjects

Keywords