Skip to main content

Main menu

  • Home
  • Content
    • Current Issue
    • Online First
    • Multimedia
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • The Issue in Brief (Plain Language Summaries)
    • Call for Papers
  • Info for
    • Authors
    • Reviewers
    • Media
    • Job Seekers
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • RSS
    • Email Alerts
    • Journal Club
  • Contact
    • Feedback
    • Contact Us
  • Careers

User menu

  • My alerts

Search

  • Advanced search
Annals of Family Medicine
  • My alerts
Annals of Family Medicine

Advanced Search

  • Home
  • Content
    • Current Issue
    • Online First
    • Multimedia
    • Collections
    • Past Issues
    • Articles by Subject
    • Articles by Type
    • Supplements
    • The Issue in Brief (Plain Language Summaries)
    • Call for Papers
  • Info for
    • Authors
    • Reviewers
    • Media
    • Job Seekers
  • About
    • Annals of Family Medicine
    • Editorial Staff & Boards
    • Sponsoring Organizations
    • Copyrights & Permissions
    • Announcements
  • Engage
    • Engage
    • e-Letters (Comments)
    • Subscribe
    • RSS
    • Email Alerts
    • Journal Club
  • Contact
    • Feedback
    • Contact Us
  • Careers
  • Follow annalsfm on Twitter
  • Visit annalsfm on Facebook
Research ArticleMethodology From PBRNS

Opportunities, Challenges, and Lessons of International Research in Practice-Based Research Networks: The Case of an International Study of Acute Otitis Media

Larry A. Green, George E. Fryer, Paul Froom, Larry Culpepper and Jack Froom
The Annals of Family Medicine September 2004, 2 (5) 429-433; DOI: https://doi.org/10.1370/afm.224
Larry A. Green
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
George E. Fryer Jr
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul Froom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Larry Culpepper
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jack Froom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Abstract

The requirements of research become more complex and demanding in international collaborations. The opportunity to study naturally occurring variation in treatment prompted networking primary care research networks in the United Kingdom, the Netherlands, and North America to study acute otitis media. Additional challenges faced and addressed in this study included (1) differing national requirements for protecting human subjects; (2) variation in data collection processes in primary care practices; (3) data transmission among participants; (4) duties and tariffs on necessary instruments; (5) fluctuation in currency exchange rates; (6) incapacitation of coinvestigators; (7) complex administration of funds; (8) financing the additional, legitimate costs of collaboration; (9) sustaining strong personal relationships among coinvestigators; and (10) accepting longer time frames than would otherwise be expected. Overall, international practice-based research can be productive, affect millions of people, and be extremely rewarding to investigators. It is not, however, for the faint-hearted.

  • International research
  • international cooperation
  • practice-based research/methods
  • research design
  • otitis media

INTRODUCTION

When viewed retrospectively through the lens of scientific papers with an emphasis on methods and results, studies typically appear to have been done in a logical and orderly manner as planned by the investigators. In reality, successful researchers are called upon to address the unexpected and overcome a host of challenges that are not usually elaborated in their reports. The purpose of this article is to illustrate how international research can be important and rewarding while confessing a set of challenges that beset an international study, thereby reminding investigators that international primary care research is extremely valuable but infused with additional requirements.

Research becomes even more complex and demanding when investigators unite to study phenomena across national boundaries. One of the common reasons for international studies is to identify and examine the consequences of variation in illness, disease, health care, and health system arrangements.1 Whereas scientific research principles are accepted globally, their expression and implementation may vary when specific methods and strategies are applied in different countries.2 These variations can be simultaneously a source of rewarding stimulation, new knowledge, and frustration for investigators.3 Such was the case for an international study of acute otitis media (AOM) funded by the US Agency for Healthcare Research and Quality (AHRQ) and administered by The Ambulatory Sentinel Practice Network (ASPN), a pioneering, national primary care practice-based research network in the United States.4

OPPORTUNITY

The attractive opportunity that prompted this international study of AOM in children 6 months to 18 years of age was grounded in naturally occurring variation in how children with AOM were treated in the United Kingdom (UK), the Netherlands, and the United States and Canada (North America).4 In the UK and North America usual care included routine prescribing of antibiotics, but for different lengths of time (5 to 7 vs 10 days, respectively). In the Netherlands a wait-and-see approach was applied for most children, and this strategy resulted in prescribing antibiotics for a minority of children. At the time of the study, 1994 to 1996, a placebo-controlled trial of antibiotic vs no antibiotic was not acceptable to human subjects review committees, but a study in which the requirement was to deliver to all enrolled subjects the care usually rendered by the nation’s clinicians was considered ethical. Thus, a concurrent observational study of children with acute otitis media receiving usual care in the Netherlands, the UK, and North America was undertaken by experienced, practice-based research networks5 comprised of frontline clinicians caring for people living in their communities in Europe and North America.6

The value of such research is illustrated in Figures 1⇓ and 2⇓. Despite different treatment practices, there was little variation in the time from initial treatment to the last report of pain or fever for children either younger than 2 or 2 to 18 years of age, based on children who had both initial visit data and a 10-day parent diary (85% of enrolled children), adjusted for sex of the child, breast feeding (ever), exposure to household tobacco smoke, and a parental history of otitis media. Results from this study were cited in subsequent randomized trials7 and were part of an evolution of care for acute otitis media during the 1990s toward current guidelines that overall encourage restrained use of antibiotics.8

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Pain resolution for children with acute otitis media.

Figure 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 2.

Fever resolution for children with acute otitis media.

Results such as these are important, verify the feasibility of practice-based research in networks of frontline clinicians, and reward the work and effort of international collaboration. Underlying these results, however, were a number of challenges that threatened the viability of the study, diminished its potential impact, and from which important lessons can be identified that might inform future international research efforts.

CHALLENGES

Challenge 1: Human Subjects Review

Each of the participating countries had established mechanisms to assure the ethical conduct of research and the protection of human subjects, but the processes differed. Because AHRQ was the funding agency, compliance with its rules and regulations was a necessity. Thus, the investigators outside the United States had to meet the dual expectations of their own review processes and also modify or augment them to accommodate the requirements from the United States.

For example, the organization of this process in the UK required the UK network to submit the study to 9 different institutional review boards because of the geographic dispersion of their practices. The work and vacation schedules of the chairpersons of these 9 review boards delayed approvals that had to occur in a defined sequence. There was considerable reluctance to accommodate the needs of this particular project, given that these review boards were being asked to do something different from what they required and expected. When the UK process was completed, the UK network launched the study only to learn that the US Office of Protection from Research Risks (OPRR) had not signed off on their review. An untimely change of staff at OPRR created further delays. ASPN, the administrator of the research funding, based on an interpretation of OPRR regulations, insisted that the UK network stop enrollment until the OPRR clearance was obtained. The UK suspended enrollment to the consternation of their clinicians.

There were substantial consequences of the multiple human subject reviews and the resulting delays. Data collection was extended into the final year of the project. The UK missed much of the wintertime enrollment period when acute otitis was more common, and never caught up with the other networks’ enrollment. Analysis and comparative feedback to the networks and their physicians were delayed, disappointing clinicians. Project analysis and publication of results could not be accomplished as planned.

Challenge 2: Instrumentation

This project was one of the first practice-based research projects to incorporate handheld computers as a means of data collection by the participating clinicians. The UK network was responsible for the development of this system, and after creating the software for what was thought to be a suitable platform, the Dutch clinicians simply said, “No!” The keyboard requirements and sequencing of data collection processes did not make sense in the Dutch practice environment. Consequently, an alternative computer, a Newton, was selected, and the software to support the study was completely rebuilt.

The new equipment had to be distributed across national boundaries and became subject to tariff and duty regulations. Grant management regulations forbade the use of grant funds for taxes and tariffs, and there were no alternative funds to cover this cost of several thousands of dollars. This problem was solved by preprogramming all of the handheld computers with the study materials, thereby converting them into research instruments that were exempt from duties. Handheld tympanometers were also required for the study, and a similar problem was overcome by importing the tympanometers into Europe as research instruments.

These instrumentation challenges resulted in relocation of programming from one country to another, duplication of effort, unfunded work, and delayed implementation of data collection.

Challenge 3: Data Transmission

Once in place, the handheld computer system functioned well, connected through a satellite system called eWorld. The database was assembled and coordinated at Brown University, with the study grant paying the costs of data transmission. Being able to see the database build in real time and being able to respond to problems promptly was a splendid, successful development.

Inadvertently, however, in each of the 2 years of data collection, ASPN exceeded limits on the credit card accounts to which the charges for eWorld were applied, resulting in the immediate discontinuation of eWorld services. This appeared to the research networks as a failure of the data transmission system. Clinicians and study coordinators suddenly could not transfer their data and would scramble to identify what had gone wrong, only to discover not a technical computing problem, but that a billing problem in the United States had “shut them down.”

Months later, the owner and operator of eWorld, decided to discontinue the service as part of a comprehensive new business plan. This decision took effect in the middle of the second winter enrollment season. Follow-up of enrolled patients required a 2-month visit with data to be transmitted via eWorld, and a few patients required a later visit.

Consequently, an alternative mechanism was required, immediately, not just in the vast North American continent, but across the Atlantic. The result was the invention, near the end of data collection, of another data collection system for each network, including a paper, pencil, and mail method. Data acquisition, closing of the data sets, and analysis were delayed, and all the networks and the central coordinating office had to adapt and absorb the additional expenses.

Challenge 4: Financial Administration

This acute otitis media study was designed and funded as a 3-year project, with each of the collaborating networks having its own subbudget, financial regulations, indirect cost requirements, and cash flow needs. From the perspective of each network, the processes of grant review and budget negotiations resulted in work that was more complex and a budget that was reduced. Subsequently, in the middle of the study, the US Congress cut the funding for AHRQ (then known as the Agency for Health Care Policy and Research), which led to a further 25% reduction in the budget that had already been reduced by negotiations at the start of the project. Because of delays in implementation, study enrollment did not conclude until month 33, requiring in some instances follow-up 9 months later, necessitating an unfunded extension on the heels of the US-imposed 25% reduction in budget. Also, during the study international monetary policies evolved, resulting in a devaluation of the US dollar against European currencies, compounding the effects of other budget reductions by an additional 25% reduction in Europe.

As a result of all of these situations, the networks were faced with the expected workload and the additional workloads associated with study delays and modifications—with a recurrently shrinking budget. These reductions had to be absorbed, somewhere, but preferably “not in my country’s budget.” Also, mismatches between when grant payments were made to networks and when costs were accrued by the project required skillful adaptation by local managers to cope with cash flow requirements. Other fiscal regulations, such as use of US airlines for travel, were reduced to being viewed as a mere nuisance by the European collaborators.

Challenge 5: Death and Despair

Midway into the project, the Dutch project director, in his mid-40s, suffered a fatal myocardial infarction while ice-skating with his daughter. There was no equally qualified person to assume his role. More importantly, his intellectual command of the project and his very strong and productive relationships with Dutch practices and the international collaborators could not be simply reinvented. Even though his colleagues ably stepped into the gap and continued the project, the hole he left was never completely filled.

After the project succeeded in completing enrollment and a robust data set was clean and established, the funding and time intended for analysis were long gone. Principled principal investigators persisted with analysis and reporting, but progress was inevitably slow because of competing requirements and loss of interest in an old project. Nonetheless, as the months passed, much of the analytic plan was achieved, and further reports were in draft stages when the principal investigator died.

SOME PRACTICAL LESSONS

Accomplishing research in primary care practice-based networks is complicated, and international research adds even more complexities that can be expected to prolong investigations. The story of the international acute otitis media study describes lessons that are probably most accurately viewed as old lessons, relearned. Some of these lessons are highlighted in Table 1⇓.

View this table:
  • View inline
  • View popup
Table 1.

Practical Lessons in International Primary Care Research

Compared with routine single-site or single-country research, international research, particularly in practice-based research networks, requires additional effort, administrative skill, and patience. Pilot testing is even more important in international studies, but no amount of pilot testing can foresee or preclude environmental changes that may destabilize international research. International projects almost certainly will take longer than expected and should include more generous time frames than would otherwise be required. Strong, trusting relationships among coinvestigators can be the basis of overcoming much adversity; and the practical means of cultivating these relationships through conference calls, meetings, and social engagements is a legitimate and necessary research expense.

Trivial questions do not justify the effort, but for important, compelling questions, such as comparisons of treatments for conditions that afflict millions of people, the yield of findings can be worth the additional effort and expense. Overall, international practice-based research can be productive and extremely gratifying, but it is not for the fainthearted.

Acknowledgments

Ruut A. deMelker, Paul Grob, Max Jacobs, Frank van Balen and the practices in the research networks as named in Froom et al5 were indispensable to the international study on which this article is based.

Dr. Jack Froom (1923–2002) inspired a generation of primary care investigators, and this essay is dedicated to him and his lifetime partner, Selma, with our deepest admiration and sincere thanks—for everything.

Footnotes

  • ↵† Deceased.

  • Conflicts of interest: none reported

  • Funding support: This research was supported by the Agency for Healthcare Research and Quality.

  • Received for publication April 1, 2004.
  • Revision received June 15, 2004.
  • Accepted for publication June 27, 2004.
  • © 2004 Annals of Family Medicine, Inc.

REFERENCES

  1. ↵
    World Health Organization. The World Health Report 2003: Shaping the Future. Geneva, Switzerland: World Health Organization; 2003:146–153.
  2. ↵
    Kohn R, White KL, eds. Health Care. An International Study. Report of the World Health Organization/International Collaborative Study of Medical Care Utilization. London: Oxford University Press; 1976.
  3. ↵
    Walley T, Folino-Gallo P, Schwabe U, van Gause E. Variation and increase in use of statins across Europe: data from administrative databases. BMJ. 2004;328:385–386.
    OpenUrlFREE Full Text
  4. ↵
    Green LA, Wood M, Becker L, et al. The ambulatory sentinel practice network: purpose, methods, and policies. J Fam Pract. 1984;18:275–280.
    OpenUrlPubMed
  5. ↵
    Froom J, Culpepper L, Grob P, et al. Diagnosis and antibiotic treatment of acute otitis media: report from International Primary Care Network. BMJ. 1990;300:582–586.
    OpenUrlAbstract/FREE Full Text
  6. ↵
    Froom J, Culpepper L, Green LA, et al. A cross-national study of acute otitis media: risk factors, severity, and treatment at initial visit. Report from the International Primary Care Network (ICPN) and the Ambulatory Sentinel Practice Network (ASPN). JABFP. 2001;14:406–417.
    OpenUrlPubMed
  7. ↵
    Damoiseaux RAMJ, van Balen FAM, Hoes AW, Verheij TJM, de Melker RA. Primary care based randomized, double blind trial of amoxicillin versus placebo for acute otitis media in children aged under 2 years. BMJ. 2000;320:350–354.
    OpenUrlAbstract/FREE Full Text
  8. ↵
    American Academy of Pediatrics and American Academy of Family Physicians Subcommittee on Management of Acute Otitis Media. Clinical Practice Guideline. Diagnosis and Management of Acute Otitis Media. Available at: http://www.aafp.org/x26481.xml. Accessed March 12, 2004.
PreviousNext
Back to top

In this issue

The Annals of Family Medicine: 2 (5)
The Annals of Family Medicine: 2 (5)
Vol. 2, Issue 5
1 Sep 2004
  • Table of Contents
  • Index by author
  • The Issue in Brief
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word on Annals of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Opportunities, Challenges, and Lessons of International Research in Practice-Based Research Networks: The Case of an International Study of Acute Otitis Media
(Your Name) has sent you a message from Annals of Family Medicine
(Your Name) thought you would like to see the Annals of Family Medicine web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
7 + 10 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Citation Tools
Opportunities, Challenges, and Lessons of International Research in Practice-Based Research Networks: The Case of an International Study of Acute Otitis Media
Larry A. Green, George E. Fryer, Paul Froom, Larry Culpepper, Jack Froom
The Annals of Family Medicine Sep 2004, 2 (5) 429-433; DOI: 10.1370/afm.224

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Get Permissions
Share
Opportunities, Challenges, and Lessons of International Research in Practice-Based Research Networks: The Case of an International Study of Acute Otitis Media
Larry A. Green, George E. Fryer, Paul Froom, Larry Culpepper, Jack Froom
The Annals of Family Medicine Sep 2004, 2 (5) 429-433; DOI: 10.1370/afm.224
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • INTRODUCTION
    • OPPORTUNITY
    • CHALLENGES
    • SOME PRACTICAL LESSONS
    • Acknowledgments
    • Footnotes
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Related Articles

  • PubMed
  • Google Scholar

Cited By...

  • Acute sore throat
  • Data Collection Outcomes Comparing Paper Forms With PDA Forms in an Office-Based Patient Survey
  • The Creation and Development of the Dental Practice-Based Research Network
  • Electronic Data Collection Options for Practice-Based Research Networks
  • In this Issue: The Patient-Clinician Relationship and Practice-Based Network Research
  • Google Scholar

More in this TOC Section

  • Rapid Assessment of Agents of Biological Terrorism: Defining the Differential Diagnosis of Inhalational Anthrax Using Electronic Communication in a Practice-Based Research Network
  • How and Why to Study the Practice Content of a Practice-Based Research Network
Show more Methodology From PBRNS

Similar Articles

Subjects

  • Other research types:
    • PBRN research
  • Other topics:
    • Research capacity building
    • Social / cultural context

Content

  • Current Issue
  • Past Issues
  • Past Issues in Brief
  • Multimedia
  • Articles by Type
  • Articles by Subject
  • Multimedia
  • Supplements
  • Online First
  • Calls for Papers

Info for

  • Authors
  • Reviewers
  • Media
  • Job Seekers

Engage

  • E-mail Alerts
  • e-Letters (Comments)
  • RSS
  • Journal Club
  • Submit a Manuscript
  • Subscribe
  • Family Medicine Careers

About

  • About Us
  • Editorial Board & Staff
  • Sponsoring Organizations
  • Copyrights & Permissions
  • Contact Us
  • eLetter/Comments Policy

© 2023 Annals of Family Medicine