Skip to main content

Main menu

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Current Issue
  • Content
    • Current Issue
    • Early Access
    • Multimedia
    • Podcast
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • Plain Language Summaries
    • Calls for Papers
  • Info for
    • Authors
    • Reviewers
    • Job Seekers
    • Media
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • Podcast
    • E-mail Alerts
    • Journal Club
    • RSS
    • Annals Forum (Archive)
  • Contact
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleOriginal Research

Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study

Christopher B. Forrest, Efrat Shadmi, Paul A. Nutting and Barbara Starfield
The Annals of Family Medicine July 2007, 5 (4) 361-367; DOI: https://doi.org/10.1370/afm.703
Christopher B. Forrest
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Efrat Shadmi
PhD, RN
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul A. Nutting
MD, MSPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Barbara Starfield
MD, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF
Loading

Published eLetters

If you would like to comment on this article, click on Submit a Response to This article, below. We welcome your input.

Submit a Response to This Article
Compose eLetter

More information about text formats

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Author Information
First or given name, e.g. 'Peter'.
Your last, or family, name, e.g. 'MacMoody'.
Your email address, e.g. higgs-boson@gmail.com
Your role and/or occupation, e.g. 'Orthopedic Surgeon'.
Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.
Statement of Competing Interests
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Vertical Tabs

Jump to comment:

  • Untangling the web of referral coordination
    Christopher J. Stille
    Published on: 27 August 2007
  • Responses to Hon and Kuo
    Christopher B Forrest
    Published on: 21 August 2007
  • The Role of Patient-Centered Care in Specialty Referral Completion
    Dennis Z Kuo, MD, MHS
    Published on: 02 August 2007
  • Specialty Referral Completion Among Primary Care Patients: Mission impossible?
    Kam lun Ellis Hon, FAAP
    Published on: 01 August 2007
  • Published on: (27 August 2007)
    Page navigation anchor for Untangling the web of referral coordination
    Untangling the web of referral coordination
    • Christopher J. Stille, Worcester, MA USA

    I read with interest the article on specialty referral completion by Forrest and colleagues(1) in the July/August issue. If one accepts a broad definition of coordination of care that includes the degree to which care plans made between patients and multiple health care providers are carried out, this demonstrates yet another aspect of the problems with care coordination that currently exist in the US health care system....

    Show More

    I read with interest the article on specialty referral completion by Forrest and colleagues(1) in the July/August issue. If one accepts a broad definition of coordination of care that includes the degree to which care plans made between patients and multiple health care providers are carried out, this demonstrates yet another aspect of the problems with care coordination that currently exist in the US health care system. It fits well with other studies of coordination, among them Dr. Forrest’s previous work in a pediatric primary care network demonstrating low levels of communication from specialists to primary care pediatricians(2), and our own work that found infrequent communication from pediatric primary care providers (PCPs) to specialists(3).

    Differences of views between patients and physicians about referral activities are symptomatic of poor referral coordination. Only fair concordance was found between physicians and patients about the simple activity of completing the referral, similar to our finding(4) that parents of referred children often disagreed with their child’s physicians about whether communication between referring and consulting physicians had occurred. This implies strongly that physicians and patients should collaborate more closely when making referral decisions, and that patients should be supported in their involvement after a referral is made. Encouraging findings that may help to build an effective patient-centered intervention are the associations of referral completion with patient request and with the PCP’s office making the appointment. Another interesting finding is that 17% of patients did not complete the referral, mainly due to lack of time or spontaneous resolution of the problem. This suggests that a strategy for monitoring of selected problems, rather than immediate referral, may help to increase access to specialty services for those patients who need more urgent specialty care.

    Several other findings in this article support the conclusion that referral completion is closely related to other care coordination activities. Several predictors of referral completion were also found in one of our studies to be predictors of successful PCP to specialist communication, including longer duration of patient relationship with physician and lack of Medicaid insurance(3). Interestingly, whether or not the referring physician reported communication with the specialist in Dr. Forrest’s study was not associated with referral completion, suggesting that these components of coordination may not all be interdependent. Determining these relationships and their impact on care outcomes is important to construct interventions to make sure that resources are used wisely, and to deliver care that is more patient-centered.

    References

    1. Forrest CB, Shadmi E, Nutting PA, Starfield B. Specialty referral completion among primary care patients: results from the ASPN referral study. Annals of Family Medicine 2007; 5:361-367.

    2. Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Archives of Pediatrics and Adolescent Medicine 2000; 154:499-506.

    3. Stille CJ, McLaughlin TJ, Primack WA, Mazor KM, Wasserman RC. Determinants and impact of generalist-specialist communication about pediatric outpatient referrals. Pediatrics 2006; 118:1341-1349.

    4. Stille CJ, Giese A, Mazor KA, Primack WA. Parent and provider experiences with generalist-specialist communication for pediatric subspecialty referrals. Pediatric Academic Societies' Annual Meeting 2003 (Abstract).

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (21 August 2007)
    Page navigation anchor for Responses to Hon and Kuo
    Responses to Hon and Kuo
    • Christopher B Forrest, Philadelphia

    I agree with Dr. Hon that a larger study of referral completion would provide more insights into the reasons for non-attendance for all patients and important sub-groups. When we completed this study, we felt that following up 776 patients through physicians practices was a major feat! In part, this relates to our methodology--self-administered questionnaires that were mailed to patients. At Children's Hospital of Philad...

    Show More

    I agree with Dr. Hon that a larger study of referral completion would provide more insights into the reasons for non-attendance for all patients and important sub-groups. When we completed this study, we felt that following up 776 patients through physicians practices was a major feat! In part, this relates to our methodology--self-administered questionnaires that were mailed to patients. At Children's Hospital of Philadelphia, we have a large pediatric network that has a common electronic health record platform. The EHR is making large scale studies possible, and I hope soon we will have better information about the primary-specialty interface using electronic capture of data via the EHR and the personal health record as well.

    Dr. Kuo points out that lack of agreement between doctors and patients and the need for a patient-centered approach when discussing a referral as a management option. I could not agree more. The referral decision is a really a negotiation that can be made more effective with enhanced communication. Patients choose to participate in a referral, and our results indicate that many choose to follow the recommendation of their primary care doctor. Still, I support the need for practice and training modification to enhance the effectiveness of referrals. Measuring outcomes across all referrals is an incredibly difficult, perhaps unattainable task at present. We attempted to do this by assessing patient and physician perceptions of the effectiveness of the referral in terms of addressing the reason for referral and their evaluations. We found no differences in these measures between those with complete and incomplete referrals; nonetheless, I think more work is needed, probably using condition specific designs that permit more complete ascertainment of outcomes, to fully understand when specialty care is needed and for how long.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (2 August 2007)
    Page navigation anchor for The Role of Patient-Centered Care in Specialty Referral Completion
    The Role of Patient-Centered Care in Specialty Referral Completion
    • Dennis Z Kuo, MD, MHS, Baltimore, MD, USA

    The importance of collaboration and care coordination between primary and specialty care is increasingly being recognized,(1-2) but what if the patient fails to show up? Forrest and colleagues(3) find that 8 of 10 patients do complete a subspecialty referral within 3 months. The authors suggest that referral completion rates may be increased by assisting patients with appointment scheduling and increasing continuity of c...

    Show More

    The importance of collaboration and care coordination between primary and specialty care is increasingly being recognized,(1-2) but what if the patient fails to show up? Forrest and colleagues(3) find that 8 of 10 patients do complete a subspecialty referral within 3 months. The authors suggest that referral completion rates may be increased by assisting patients with appointment scheduling and increasing continuity of care by the primary care physician. I am struck by the K of .34 indicating only fair agreement between physician and patient regarding specialty completion and the specificity of 51.5% describing physician report of actual referral completion. It would be one thing if a physician was aware of patient refusal of the specialty referral, but it is another thing for the physician to assume a patient is receiving care when he in fact is not and have an adverse outcome due to delayed assessment and management.

    The study findings support the importance of patient-centered care when making subspecialty referrals. The authors report that referral completion was associated with patient requesting referral, increased physician time in practice, and longer continuity of care, which are all issues associated with patient-centered care. The duration of the physician-patient relationship and the years the physician has spent in practice obviously cannot be changed for a given situation, but the authors comments that “a shared understanding of common goals may increase patients’ trust that their physicians are acting on their behalf, thereby increasing patient’s motivation to follow specialty referral recommendations made by their doctor.� There is evidence that training providers to be more patient-centered during clinical consultations improves overall communication and increase patient satisfaction;(4) thus, in addition to practice-level interventions such as scheduling assistance, provider training in communication skills and shared decision making might also impact referral completion rates.

    Finally, it would be interesting to examine the outcomes of incomplete versus completed referrals. Reasons patients reported for not completing the referral included resolution of the health problem, lack of time, or disagreement with the physician. If patients who delay or refuse specialty referral do not have worse outcomes, then perhaps such referrals were unnecessary and the patient understood this. The examining physician may have failed to accurately assess the situation, possibly due to suboptimal communication or from liability concerns. Criteria for referrals are complex and vary widely between physicians, and physician training may be able to influence referral rates(5) and, potentially, referral completion.

    References:

    (1) McManus P, Fox H, Limb S, Carpinelli A, and the Federal Expert Work Group on Pediatric Subspecialty Capacity. Promising approaches for strengthening the interface between primary and specialty pediatric care. Maternal and Child Health Policy Research Center, March 2006. Available at www.mchpolicy.org.

    (2) Wagner EH, Glasgow RE, Davis C, Bonomi AE, Provost L, McCulloch D, Carver P, Sixta C. Quality improvement in chronic illness care: a collaborative approach. Jt Comm J Qual Improv 2001;27:63-80.

    (3) Forrest CB, Shadmi E, Nutting PA, Starfield B. Specialty referral completion among primary care patients: results from the ASPN referral study. Ann Fam Med 2007;5:361-7.

    (4) Lewin SA, Skea ZC, Entwistle V, Zwarenstein M, Dick J. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev 2001;4:CD003267.

    (5) Forrest CB, Nutting PA, von Schrader S, Rohde C, Starfield B. Primary care physician specialty referral decision making: patient, physician, and health care system determinants. Med Decis Making 2006;26:76-85.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
  • Published on: (1 August 2007)
    Page navigation anchor for Specialty Referral Completion Among Primary Care Patients: Mission impossible?
    Specialty Referral Completion Among Primary Care Patients: Mission impossible?
    • Kam lun Ellis Hon, FAAP, Hong Kong, China

    The paper by Forrest and colleagues(1) is important and investigate the problem of specialty referral completion among primary care patients at outpatient clinics that have been studied infrequently since the early 70s.(2) Nonattendance has been a common problem not only affecting medical costs but also hospital resources. Studies of the patterns and predictors of non-attendance have yielded contradicting results. Impor...

    Show More

    The paper by Forrest and colleagues(1) is important and investigate the problem of specialty referral completion among primary care patients at outpatient clinics that have been studied infrequently since the early 70s.(2) Nonattendance has been a common problem not only affecting medical costs but also hospital resources. Studies of the patterns and predictors of non-attendance have yielded contradicting results. Importantly, very few prospective or randomized trials have addressed this problem at the preventive levels.(3-4)

    The study by Forrest and colleagues, however, is limited by the small and potentially non-representative number of only 776 (predominately white) patients from 113 physicians from 30 states over a 2 year-period between August 1997 to April 1999. This averages to only 13 patients per state per year. A larger scale prospective study may better enlighten us on the issue.

    The investigators find that a longer duration of the patient relationship with the primary care physician and physician/staff scheduling of the specialty appointment are both positive predictors of referral completion, and conclude that the referral completion rates may be increased by assisting patients with scheduling their specialty appointments and promoting continuity of care. Various interventions have been reported to promote attendance (including a telephone reminder and mail reminder prior to the appointment date) with variable rates of success.(3-5) As "lack of time" and "health problem had resolved" were found to be the most common reasons for not completing the referral, various forms of reminders may prove to be of limited efficacy. The increased human and financial resources required to accomplish this might offset its worthiness. Prospective studies are called for to evaluate these hypotheses.

    References

    1. Forrest CB, et al. Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study. Ann Fam Med 2007;5:361-367.

    2. Anderson FP, et al. An approach to the problem of non-attendance in a paediatric outpatients clinic. Am J Dis Child 1971;122:142-3.

    3. Sawyer SM, et al. Telephone reminders improve adolescent clinic attendance: A randomized controlled trial. J Paediatr Child Health 2002;38:79-83.

    4. Hahim MJ, et al. Effectiveness of telephone reminders in improving rate of appointments kept at an outpatient clinic: A randomized controlled trial. J Am Board Fam Pract 2001;114:193-6.

    5. Hon KL, et al. Reasons for new referral non-attendance at a pediatric dermatology center: A telephone survey. J Dermatol Treat 2005;16:113-116.

    Competing interests:   None declared

    Show Less
    Competing Interests: None declared.
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 5 (4)
The Annals of Family Medicine: 5 (4)
Vol. 5, Issue 4
1 Jul 2007
  • Table of Contents
  • Index by author
  • In Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
3 + 4 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study
Christopher B. Forrest, Efrat Shadmi, Paul A. Nutting, Barbara Starfield
The Annals of Family Medicine Jul 2007, 5 (4) 361-367; DOI: 10.1370/afm.703

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
Specialty Referral Completion Among Primary Care Patients: Results From the ASPN Referral Study
Christopher B. Forrest, Efrat Shadmi, Paul A. Nutting, Barbara Starfield
The Annals of Family Medicine Jul 2007, 5 (4) 361-367; DOI: 10.1370/afm.703
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • Acknowledgments
    • Footnotes
    • REFERENCES
  • Figures & Data
  • eLetters
  • Info & Metrics
  • PDF

Related Articles

  • PubMed
  • Google Scholar

Cited By...

  • Patient Perspectives on Delayed Specialty Follow-Up After a Primary Care Visit
  • Does use of GP and specialist services vary across areas and according to individual socioeconomic position? A multilevel analysis using linked data in Australia
  • The Association of Practice Type and the Scope of Care of Family Physicians
  • Referral Rates Vary Widely Between Family Medicine Practices
  • Electronic Consultations to Improve the Primary Care-Specialty Care Interface for Cardiology in the Medically Underserved: A Cluster-Randomized Controlled Trial
  • Barbara Starfield: Passage of the Pathfinder of Primary Care
  • In This Issue: Clinical Diagnosis and Management
  • Google Scholar

More in this TOC Section

  • Shared Decision Making Among Racially and/or Ethnically Diverse Populations in Primary Care: A Scoping Review of Barriers and Facilitators
  • Convenience or Continuity: When Are Patients Willing to Wait to See Their Own Doctor?
  • Feasibility and Acceptability of the “About Me” Care Card as a Tool for Engaging Older Adults in Conversations About Cognitive Impairment
Show more Original Research

Similar Articles

Subjects

  • Methods:
    • Quantitative methods
  • Other research types:
    • Health services
    • PBRN research
  • Core values of primary care:
    • Continuity
    • Coordination / integration of care

Content

  • Current Issue
  • Past Issues
  • Early Access
  • Plain-Language Summaries
  • Multimedia
  • Podcast
  • Articles by Type
  • Articles by Subject
  • Supplements
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Job Seekers
  • Media

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2025 Annals of Family Medicine