HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) In Brief TRACK Comments: Submit a response TRACK Comments: View responses Alert me when this article is cited Alert me when TRACK Comments are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Green, L. A. Articles by Froom, J. Search for Related Content PubMed PubMed Citation Articles by Green, L. A. Articles by Froom, J.

Opportunities, Challenges, and Lessons of International Research in Practice-Based Research Networks: The Case of an International Study of Acute Otitis Media

Larry A. Green, MD¹, George E. Fryer, Jr, PhD¹, Paul Froom, MD², Larry Culpepper, MD³ and Jack Froom, MD^4,

¹ The Robert Graham Center, Washington, DC
² Hematology Laboratories, Clalit Health Services, Nesher, Israel
³ Department of Family Medicine, Boston University, Boston, Mass
⁴ Department of Family Medicine, State University of New York, Stoney Brook, Stoney Brook, NY

CORRESPONDING AUTHOR: Larry A. Green, MD The Robert Graham Center, Suite 201 1350 Connecticut Avenue, NW Washington, DC 22036 Lgreen{at}aafp.org

	ABSTRACT

TOP ABSTRACT INTRODUCTION OPPORTUNITY CHALLENGES SOME PRACTICAL LESSONS REFERENCES

 
The requirements of research become more complex and demanding in international collaborations. The opportunity to study naturally occurring variation in treatment prompted networking primary care research networks in the United Kingdom, the Netherlands, and North America to study acute otitis media. Additional challenges faced and addressed in this study included (1) differing national requirements for protecting human subjects; (2) variation in data collection processes in primary care practices; (3) data transmission among participants; (4) duties and tariffs on necessary instruments; (5) fluctuation in currency exchange rates; (6) incapacitation of coinvestigators; (7) complex administration of funds; (8) financing the additional, legitimate costs of collaboration; (9) sustaining strong personal relationships among coinvestigators; and (10) accepting longer time frames than would otherwise be expected. Overall, international practice-based research can be productive, affect millions of people, and be extremely rewarding to investigators. It is not, however, for the faint-hearted.

Key Words: International research • international cooperation • practice-based research/methods • research design • otitis media

	INTRODUCTION

TOP ABSTRACT INTRODUCTION OPPORTUNITY CHALLENGES SOME PRACTICAL LESSONS REFERENCES

 
When viewed retrospectively through the lens of scientific papers with an emphasis on methods and results, studies typically appear to have been done in a logical and orderly manner as planned by the investigators. In reality, successful researchers are called upon to address the unexpected and overcome a host of challenges that are not usually elaborated in their reports. The purpose of this article is to illustrate how international research can be important and rewarding while confessing a set of challenges that beset an international study, thereby reminding investigators that international primary care research is extremely valuable but infused with additional requirements.

Research becomes even more complex and demanding when investigators unite to study phenomena across national boundaries. One of the common reasons for international studies is to identify and examine the consequences of variation in illness, disease, health care, and health system arrangements.¹ Whereas scientific research principles are accepted globally, their expression and implementation may vary when specific methods and strategies are applied in different countries.² These variations can be simultaneously a source of rewarding stimulation, new knowledge, and frustration for investigators.³ Such was the case for an international study of acute otitis media (AOM) funded by the US Agency for Healthcare Research and Quality (AHRQ) and administered by The Ambulatory Sentinel Practice Network (ASPN), a pioneering, national primary care practice-based research network in the United States.⁴

	OPPORTUNITY

TOP ABSTRACT INTRODUCTION OPPORTUNITY CHALLENGES SOME PRACTICAL LESSONS REFERENCES

 
The attractive opportunity that prompted this international study of AOM in children 6 months to 18 years of age was grounded in naturally occurring variation in how children with AOM were treated in the United Kingdom (UK), the Netherlands, and the United States and Canada (North America).⁴ In the UK and North America usual care included routine prescribing of antibiotics, but for different lengths of time (5 to 7 vs 10 days, respectively). In the Netherlands a wait-and-see approach was applied for most children, and this strategy resulted in prescribing antibiotics for a minority of children. At the time of the study, 1994 to 1996, a placebo-controlled trial of antibiotic vs no antibiotic was not acceptable to human subjects review committees, but a study in which the requirement was to deliver to all enrolled subjects the care usually rendered by the nation’s clinicians was considered ethical. Thus, a concurrent observational study of children with acute otitis media receiving usual care in the Netherlands, the UK, and North America was undertaken by experienced, practice-based research networks⁵ comprised of frontline clinicians caring for people living in their communities in Europe and North America.⁶

The value of such research is illustrated in Figures 1 and 2. Despite different treatment practices, there was little variation in the time from initial treatment to the last report of pain or fever for children either younger than 2 or 2 to 18 years of age, based on children who had both initial visit data and a 10-day parent diary (85% of enrolled children), adjusted for sex of the child, breast feeding (ever), exposure to household tobacco smoke, and a parental history of otitis media. Results from this study were cited in subsequent randomized trials⁷ and were part of an evolution of care for acute otitis media during the 1990s toward current guidelines that overall encourage restrained use of antibiotics.⁸

View larger version (27K): [in this window] [in a new window]   Figure 1. Pain resolution for children with acute otitis media.

View larger version (27K): [in this window] [in a new window]   Figure 2. Fever resolution for children with acute otitis media.

	CHALLENGES

TOP ABSTRACT INTRODUCTION OPPORTUNITY CHALLENGES SOME PRACTICAL LESSONS REFERENCES

 
Challenge 1: Human Subjects Review
Each of the participating countries had established mechanisms to assure the ethical conduct of research and the protection of human subjects, but the processes differed. Because AHRQ was the funding agency, compliance with its rules and regulations was a necessity. Thus, the investigators outside the United States had to meet the dual expectations of their own review processes and also modify or augment them to accommodate the requirements from the United States.

For example, the organization of this process in the UK required the UK network to submit the study to 9 different institutional review boards because of the geographic dispersion of their practices. The work and vacation schedules of the chairpersons of these 9 review boards delayed approvals that had to occur in a defined sequence. There was considerable reluctance to accommodate the needs of this particular project, given that these review boards were being asked to do something different from what they required and expected. When the UK process was completed, the UK network launched the study only to learn that the US Office of Protection from Research Risks (OPRR) had not signed off on their review. An untimely change of staff at OPRR created further delays. ASPN, the administrator of the research funding, based on an interpretation of OPRR regulations, insisted that the UK network stop enrollment until the OPRR clearance was obtained. The UK suspended enrollment to the consternation of their clinicians.

There were substantial consequences of the multiple human subject reviews and the resulting delays. Data collection was extended into the final year of the project. The UK missed much of the wintertime enrollment period when acute otitis was more common, and never caught up with the other networks’ enrollment. Analysis and comparative feedback to the networks and their physicians were delayed, disappointing clinicians. Project analysis and publication of results could not be accomplished as planned.

Challenge 2: Instrumentation
This project was one of the first practice-based research projects to incorporate handheld computers as a means of data collection by the participating clinicians. The UK network was responsible for the development of this system, and after creating the software for what was thought to be a suitable platform, the Dutch clinicians simply said, "No!" The keyboard requirements and sequencing of data collection processes did not make sense in the Dutch practice environment. Consequently, an alternative computer, a Newton, was selected, and the software to support the study was completely rebuilt.

The new equipment had to be distributed across national boundaries and became subject to tariff and duty regulations. Grant management regulations forbade the use of grant funds for taxes and tariffs, and there were no alternative funds to cover this cost of several thousands of dollars. This problem was solved by preprogramming all of the handheld computers with the study materials, thereby converting them into research instruments that were exempt from duties. Handheld tympanometers were also required for the study, and a similar problem was overcome by importing the tympanometers into Europe as research instruments.

These instrumentation challenges resulted in relocation of programming from one country to another, duplication of effort, unfunded work, and delayed implementation of data collection.

Challenge 3: Data Transmission
Once in place, the handheld computer system functioned well, connected through a satellite system called eWorld. The database was assembled and coordinated at Brown University, with the study grant paying the costs of data transmission. Being able to see the database build in real time and being able to respond to problems promptly was a splendid, successful development.

Inadvertently, however, in each of the 2 years of data collection, ASPN exceeded limits on the credit card accounts to which the charges for eWorld were applied, resulting in the immediate discontinuation of eWorld services. This appeared to the research networks as a failure of the data transmission system. Clinicians and study coordinators suddenly could not transfer their data and would scramble to identify what had gone wrong, only to discover not a technical computing problem, but that a billing problem in the United States had "shut them down."

Months later, the owner and operator of eWorld, decided to discontinue the service as part of a comprehensive new business plan. This decision took effect in the middle of the second winter enrollment season. Follow-up of enrolled patients required a 2-month visit with data to be transmitted via eWorld, and a few patients required a later visit.

Consequently, an alternative mechanism was required, immediately, not just in the vast North American continent, but across the Atlantic. The result was the invention, near the end of data collection, of another data collection system for each network, including a paper, pencil, and mail method. Data acquisition, closing of the data sets, and analysis were delayed, and all the networks and the central coordinating office had to adapt and absorb the additional expenses.

Challenge 4: Financial Administration
This acute otitis media study was designed and funded as a 3-year project, with each of the collaborating networks having its own subbudget, financial regulations, indirect cost requirements, and cash flow needs. From the perspective of each network, the processes of grant review and budget negotiations resulted in work that was more complex and a budget that was reduced. Subsequently, in the middle of the study, the US Congress cut the funding for AHRQ (then known as the Agency for Health Care Policy and Research), which led to a further 25% reduction in the budget that had already been reduced by negotiations at the start of the project. Because of delays in implementation, study enrollment did not conclude until month 33, requiring in some instances follow-up 9 months later, necessitating an unfunded extension on the heels of the US-imposed 25% reduction in budget. Also, during the study international monetary policies evolved, resulting in a devaluation of the US dollar against European currencies, compounding the effects of other budget reductions by an additional 25% reduction in Europe.

As a result of all of these situations, the networks were faced with the expected workload and the additional workloads associated with study delays and modifications—with a recurrently shrinking budget. These reductions had to be absorbed, somewhere, but preferably "not in my country’s budget." Also, mismatches between when grant payments were made to networks and when costs were accrued by the project required skillful adaptation by local managers to cope with cash flow requirements. Other fiscal regulations, such as use of US airlines for travel, were reduced to being viewed as a mere nuisance by the European collaborators.

Challenge 5: Death and Despair
Midway into the project, the Dutch project director, in his mid-40s, suffered a fatal myocardial infarction while ice-skating with his daughter. There was no equally qualified person to assume his role. More importantly, his intellectual command of the project and his very strong and productive relationships with Dutch practices and the international collaborators could not be simply reinvented. Even though his colleagues ably stepped into the gap and continued the project, the hole he left was never completely filled.

After the project succeeded in completing enrollment and a robust data set was clean and established, the funding and time intended for analysis were long gone. Principled principal investigators persisted with analysis and reporting, but progress was inevitably slow because of competing requirements and loss of interest in an old project. Nonetheless, as the months passed, much of the analytic plan was achieved, and further reports were in draft stages when the principal investigator died.

	SOME PRACTICAL LESSONS

TOP ABSTRACT INTRODUCTION OPPORTUNITY CHALLENGES SOME PRACTICAL LESSONS REFERENCES

 
Accomplishing research in primary care practice-based networks is complicated, and international research adds even more complexities that can be expected to prolong investigations. The story of the international acute otitis media study describes lessons that are probably most accurately viewed as old lessons, relearned. Some of these lessons are highlighted in Table 1.

Trivial questions do not justify the effort, but for important, compelling questions, such as comparisons of treatments for conditions that afflict millions of people, the yield of findings can be worth the additional effort and expense. Overall, international practice-based research can be productive and extremely gratifying, but it is not for the fainthearted.

	ACKNOWLEDGMENTS

 
Ruut A. deMelker, Paul Grob, Max Jacobs, Frank van Balen and the practices in the research networks as named in Froom et al⁵ were indispensable to the international study on which this article is based.

Dr. Jack Froom (1923–2002) inspired a generation of primary care investigators, and this essay is dedicated to him and his lifetime partner, Selma, with our deepest admiration and sincere thanks—for everything.

	FOOTNOTES

 
Deceased.

Conflicts of interest: none reported

Funding support: This research was supported by the Agency for Healthcare Research and Quality.

Received for publication April 1, 2004. Revision received June 15, 2004. Accepted for publication June 27, 2004.

	REFERENCES

TOP ABSTRACT INTRODUCTION OPPORTUNITY CHALLENGES SOME PRACTICAL LESSONS REFERENCES

 

1. World Health Organization. The World Health Report 2003: Shaping the Future. Geneva, Switzerland: World Health Organization; 2003:146–153.
2. Kohn R, White KL, eds. Health Care. An International Study. Report of the World Health Organization/International Collaborative Study of Medical Care Utilization. London: Oxford University Press; 1976.
3. Walley T, Folino-Gallo P, Schwabe U, van Gause E. Variation and increase in use of statins across Europe: data from administrative databases. BMJ. 2004;328:385–386.[Free Full Text]
4. Green LA, Wood M, Becker L, et al. The ambulatory sentinel practice network: purpose, methods, and policies. J Fam Pract. 1984;18:275–280.[Medline]
5. Froom J, Culpepper L, Grob P, et al. Diagnosis and antibiotic treatment of acute otitis media: report from International Primary Care Network. BMJ. 1990;300:582–586.[Medline]
6. Froom J, Culpepper L, Green LA, et al. A cross-national study of acute otitis media: risk factors, severity, and treatment at initial visit. Report from the International Primary Care Network (ICPN) and the Ambulatory Sentinel Practice Network (ASPN). JABFP. 2001;14:406–417.[Medline]
7. Damoiseaux RAMJ, van Balen FAM, Hoes AW, Verheij TJM, de Melker RA. Primary care based randomized, double blind trial of amoxicillin versus placebo for acute otitis media in children aged under 2 years. BMJ. 2000;320:350–354.[Abstract/Free Full Text]
8. American Academy of Pediatrics and American Academy of Family Physicians Subcommittee on Management of Acute Otitis Media. Clinical Practice Guideline. Diagnosis and Management of Acute Otitis Media. Available at: http://www.aafp.org/x26481.xml. Accessed March 12, 2004.

This article has been cited by other articles:

				J. M. Galliher, T. V. Stewart, P. K. Pathak, J. J. Werner, L. M. Dickinson, and J. M. Hickner Data Collection Outcomes Comparing Paper Forms With PDA Forms in an Office-Based Patient Survey Ann. Fam. Med, March 1, 2008; 6(2): 154 - 160. [Abstract] [Full Text] [PDF]

				G. H. Gilbert, O. D. Williams, D. B. Rindal, D. J. Pihlstrom, P. L. Benjamin, M. C. Wallace, and for the DPBRN Collaborative Group The Creation and Development of the Dental Practice-Based Research Network J Am Dent Assoc, January 1, 2008; 139(1): 74 - 81. [Abstract] [Full Text] [PDF]

				W. D. Pace and E. W. Staton Electronic Data Collection Options for Practice-Based Research Networks Ann. Fam. Med, May 1, 2005; 3(suppl_1): S21 - S29. [Abstract] [Full Text] [PDF]

				K. C. Stange In this Issue: The Patient-Clinician Relationship and Practice-Based Network Research Ann. Fam. Med, September 1, 2004; 2(5): 386 - 387. [Full Text] [PDF]

TRACK Comments:

This Article Abstract Figures Only Full Text (PDF) In Brief TRACK Comments: Submit a response TRACK Comments: View responses Alert me when this article is cited Alert me when TRACK Comments are posted Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Green, L. A. Articles by Froom, J. Search for Related Content PubMed PubMed Citation Articles by Green, L. A. Articles by Froom, J.