Article Text
PDF
Abstract
The publication of insights from a quality improvement project recently precipitated a ruling by the lead federal regulatory agency that regulations providing protection for human subjects of research should apply. The required research review process did not match the rapid changes, small samples, limited documentation, clinician management, and type of information commonly used in quality improvement. Yet quality improvement can risk harm to patients, so some review might be in order. The boundaries and processes are not clear. Efforts have been made to determine what constitutes “research”, but this has proved difficult and often yields irrational guidance with regard to protection of patients. Society needs a workable way to separate activities that will improve care, on the one hand, and those that constitute research, on the other. Practitioners who lead both quality improvement and research projects claim that those which rapidly give feedback to the care system that generated the data, aiming to change practices within that system, are “quality improvement” no matter whether the findings are published, whether the project is grant funded, and whether contemporaneous controls do not have the intervention. This criterion has not previously been proposed as a possible demarcation. The quandaries of which projects to put through research review and how to ensure ethical implementation of quality improvement need to be resolved.
- quality improvement
- human subject protection
- informed consent
- ethics
Statistics from Altmetric.com
In 2001 a faculty member at the University of Pittsburgh reported a published paper to their Institutional Review Board (IRB), the committee that reviews research for compliance with requirements for human subject protection, as part of a comprehensive review of all faculty publications by the IRB (box 1).1 The publication described a quality improvement project sponsored by an End Stage Renal Disease (ESRD) network under a Medicare contract which aimed to improve end dialysis measures of adequate blood cleaning through better compliance with dialysis prescriptions. Although the project succeeded in improving the adequacy of dialysis, the pathway for improvement was problematic. Improvement in end dialysis laboratory values resulted from physicians prescribing longer than necessary dialysis times which counterbalanced a tendency for the dialysis centers to deliver shorter dialysis times than prescribed. The article told the story, aiming to alert readers to watch for creating compensating errors since counterbalancing errors do not create reliable excellence. The project had started with a review of routinely collected data for all dialysis centers in the state, which identified those that had persistently inadequate end dialysis laboratory values. All dialysis episodes for all patients at low performing centers contributed data at baseline and after 9 ??months of improvement activities (which were developed and implemented at each site). Data from a few random patients at better performing centers monitored secular trends.
Box 1 Institutional Review Boards
In the US, organizations sponsoring research or allowing research upon patients in their care require that the project be reviewed by an Institutional Review Board (IRB), which aims to ensure that the project is worthwhile, that patient’s well being and privacy are defended, that human subjects have the opportunity for meaningful consent, and that the burdens and opportunities of research are fairly distributed. The IRB must include a representative of the public as well as professionals and researchers. The work of the IRB is funded only as part of the overhead of research organizations, and many IRBs are overwhelmed with their routine tasks.
In their retrospective review the university’s IRB ruled that the activity itself, not just publication of the results, was “research” and should have been reviewed by the IRB. The university’s inquiry to a regional official of the federal Centers for Medicare and Medicaid Services yielded the opinion that the project was not research.2 The university and its IRB appealed to the Federal Office for Human Research Protection (OHRP)3 which eventually ruled that the activity was research and should have had IRB review.4
This case brought a long avoided problem to the attention of many people involved in improving quality of health care. The social arrangements currently designed to protect human subjects of research seem to apply to many quality improvement projects, but applying them would create almost overwhelming barriers for improving quality. People concerned with the prospects for quality improvement point out, reasonably, that no one could possibly have known that this project would teach us something worth publishing, and that no patient was actually at increased risk of harm from the improvement activity. Those who feel strongly about human subject protection, on the other hand, would point out that there were privacy issues for the subjects and that the project entailed having a “control group” and imposed potentially disruptive interventions. Furthermore, strong advocates of human subject protection would worry that some projects that impose substantial risks could be inappropriately cast as quality improvement in order to evade careful review for protection of human subjects. On the other hand, if OHRP’s ruling prevailed, requiring projects like the one at Pittsburgh to undergo review, then quality improvement would be much slower and more difficult to sustain and virtually all quality improvement would come under IRB review. Still, patients involved in quality improvement are often at some increased risk of bodily harm or affronts to their dignity, usually without their knowledge or consent, and society has established no responsible way to limit risks prospectively except the protection for human subjects of research.5
The anxiety and concerns that this case caused for the ESRD networks have greatly impeded their quality improvement work. Obviously, if OHRP’s ruling becomes more broadly applicable, reform and improvement activity would slow greatly and research review would be inundated with new demands.
SOME ATTEMPTED RESOLUTIONS
At about the same time, the National Center for Ethics in Health Care at the Veterans Health Administration (VHA) was working on the same issues, after being consulted about large scale projects that were designed as quality improvement but resembled research. The Center initiated a long process of deliberation and consensus building that resulted in a formal report by VHA’s National Ethics Committee in May 2002. The report outlines 10 ethical principles that should be used by healthcare organizations to “balance the ethical imperative to adequately protect patients and the ethical imperative to continuously improve patient care”.6
The ESRD networks brought their concerns to the American Medical Association (AMA). This led the AMA House of Delegates to pass a resolution seeking better policy7 and the AMA staff to produce a report that is working its way through the AMA approval process.8
A few scholars have published their approaches to these issues but their claims are quite divergent. The starting point for most of these analyses has been how to differentiate quality improvement from “true” research. For example, Casaratt et al proposed that projects are “research” if most patients will not benefit or if additional risks or burdens are imposed upon patients in order to make the results generalizable,9 while Bellin and Dubler concluded that studies that used a control group were “research”.10 Lo and Groman did not look for a distinction between improvement and research but anchored their recommendations in the distinction between minimal risk projects (which require little special oversight) and riskier projects that should consider at least an independent review of some sort.11
Likewise, the regulatory language that addresses this issue has mainly claimed simply that one issue or another separates research from quality improvement. The problems of ambiguous projects or imprecision of the criteria have not yet been addressed in regulations, even though Medicare will have to respond to the situation brought about by the University of Pittsburgh case in order to proceed with quality initiatives in the ESRD networks and in the quality improvement organizations. The common rule that governs the protection of human subjects in research focuses on being “designed to develop generalizable knowledge”.5 The new US regulations governing privacy of medical records simply excluded quality improvement as “health care operations” and did not elaborate on a definition or how to categorize difficult projects.12 The National Bioethics Advisory Commission added the consideration of whether a program is already established or “new, modified, or untested”, and likened quality improvement activities more to clinical practice than to research.13 Each of these legal pronouncements leaves many projects in an ambiguous zone, and also would probably move many improvement projects into review as research involving human subjects for fear that serious penalties might be imposed if the project was not subjected to review as research.
AN ALTERNATIVE APPROACH
Most of this discussion on the ethics of quality improvement frames the issue as first determining what is research and what is not. That which is research is then regulated in the established way,5 while that which is not research follows a different and largely unregulated path. Very often, though, where one ends up depends a great deal on where one starts. It might be different and somewhat more useful—or at least illuminating—to start from a different framing of the issues, which is what I outline here.
At any one time and place, certain practices combine as the “system” for providing health care to the population. At all times and places those arrangements are known to fall short of optimal, whether they are judged in contrast to published reports, anecdotal reports, research projects, or just the thoughtful perspectives of those most closely concerned. A part of the obligation of those who serve in health care, especially to the extent that they are seen as having “professional” status, is to improve their own care settings, always moving toward more effective, reliable, and efficient arrangements.14,15 Improving quality and efficiency is not an occasional and optional “add on” to the lives of healthcare professionals; it is intrinsic to the endeavor.
Nevertheless, improvement activities can have a serious downside for healthcare workers. Improving a set of activities necessarily implies that prior practices were not optimal. Recognizing and accepting that reality is often difficult for the people who provided that care, resulting in denial, resistance, shame, and other dysfunctional responses.
Innovation and reform already face substantial disincentives. Ordinarily, improved care strategies are not financially attractive, are controversial among some providers, or are simply disruptive to routines. In addition, making substantial change very quickly leaves the reformer open to allegations of failing to practice in accordance with professional standards, which can precipitate legal trouble or adverse publicity.
Trying out changes that aim at improvement also creates the potential for adverse effects, sometimes obvious or well known, and sometimes quite unexpected. Thus, innovation aimed at improving the performance of the healthcare “system” always carries the risk of disadvantaging or harming some patients. Risks can be mitigated by strategies like learning from prior implementation of similar changes in other settings, incrementally trying new approaches (that is, by starting with small numbers of patients with sequential partial changes), closely monitoring effects of changes, and revising innovations to limit adverse effects. Nevertheless, quality improvement can still harm patients.
However, small harms are ordinarily seen as part of the cost of living in complex community settings. No one expects to compensate a small business that loses income while the county fixes the road out front. The patient who prefers a large open ward while in a nursing home is disadvantaged by current regulations that require private and semi-private rooms. People routinely would prefer that venereal diseases be kept private and not reported to public health authorities, but their discomfort is set aside for the greater public good. In short, in numerous ways, citizens expect that society will adversely affect our self-interests and that improving societal function will incur some adversities that each person must simply bear. The costs of keeping the society civil and appealing should be equitably borne across the population, but that concern ordinarily takes shape only when disproportionate burdens are apparent.
Minor adjustments in operations are usually left to the responsibility of those who implement them. As reforms become more extensive or have more impact, however, they often come under more scrutiny by supervisors, regulators, and the public; and substantial changes are often subject to the political process. For example, the local transit authority can routinely realign bus routes without much more than notice to bus riders, but eliminating most bus routes would require public hearings and ratification by the local governing body.
One marker that the “size” of reform and innovation is within the range of the “ordinarily expected” is that a proposed rearrangement is within the scope of what would have been acceptable if it had happened within the ordinary erratic variations in the system. Evidence that a proposed quality improvement activity in health care is acceptable without review might well be that the practice is widely accepted in some part of the country, that it has been proved to be effective in a research trial, or that it is already within the range of practices that are being implemented by individual clinicians. What makes it an “improvement activity” is then only that it serves a deliberate aim to improve and its effects are being monitored. As discussed below, the use of measurement and feedback aimed to change care practices may be the most defining element in quality improvement assessment.
If the model of quality improvement as intrinsic to the work of healthcare professionals is acceptable, society would need to develop ways that allow directly engaged professionals to innovate toward improved practices and also to provide appropriate scrutiny when the changes proposed are more extensive or have more substantial impact.
Since a regulatory structure already protects human subjects of research, its interface with the professional obligation to pursue quality improvement will require definition. Research projects are morally justifiable only if the methods are likely to produce useful and reliable knowledge and the human subjects give knowledgeable consent. Any human subject research project is morally optional and cannot go forward unless subjects are willing to take on the risk.
Oversight for research came into existence to guard against people being tricked into taking risks for the benefit of “science”, “researchers”, or generally “other people” in the future. The problem is that the review apparatus for research is quite formal, rather ponderous and costly, and nearly overwhelmed with its current workload.16,17 The research review process does not match the rapid changes, small samples, limited documentation, clinician management, and type of information of ordinary quality improvement. Yet many improvement activities might qualify as research under the expansive definition of “research” used in the federal regulations5: “a systematic investigation ... designed to develop or contribute to generalizable knowledge”. This is evident in the OHRP ruling that incited the current concerns. Society will need a workable way to separate activities that will have no oversight as research because they are part of the professional obligation to improve care, on the one hand, and activities that are required to have oversight as research, on the other.
Perhaps a project is “research” rather than quality improvement if it is a substantial deviation from established good practices (more risk of unexpected adverse impact), has substantial funding (more risk of bias), has substantial non-therapeutic aims (more risk of investigator self-interest), has individual patients as subjects (rather than care systems or providers), or involves randomization and blinding (only appropriate when the merits of the intervention are really unknown or considered to be equal to the standard treatment,18 except perhaps in trials that otherwise count as quality improvement but are initiated consecutively among randomly assigned practice teams19). In preparation for a conference on ethics of quality improvement, Bottrell interviewed 24 persons who were engaged in quality improvement,20 most of whom also worked in clinical or research settings. The most substantial and persistent differentiating factor that they articulated was that quality improvement always feeds back measurements rapidly to the care system that is generating the data, which leads to an ongoing shaping of the intervention. These respondents often felt that the generalizable knowledge that quality improvement projects sometimes yield were not the defining element, as they are for conventional research. Instead, they said that the projects that gave rapid feedback to the care system that generated the data, aiming to change practices within that system, were “quality improvement” no matter whether the findings were published, whether the project was grant funded, or whether contemporaneous controls did not have the intervention. This idea is not yet part of any regulation or prior literature.
Quality improvement endeavors are dynamic, with changing packages of interventions and time series charts as the usual mode of measurement. Conventional research principles hold that this method can yield only hunches and hypotheses which cannot be considered to be reliable until proved in later formal research. Since quality improvement cannot generate a formal statistical test of the likelihood of a significant difference and the domain for generalizations is not clear, its findings ordinarily are not publishable in peer reviewed literature. However, surely one does gain reliable insights about both the subject matter and the practical efficacy of improvement strategies by working on a series of related quality improvement projects. For example, our work with 35 teams on improved care for persons with advanced heart or lung failure taught us specific insights about advance care planning and dyspnea relief, as well as about how to move conventional care systems toward optimal services.21 Settling how one responsibly assesses the validity and reliability of insights arising from quality improvement is central to the debate about ethics. If the findings of quality improvement projects are largely worthwhile in shaping the care system, one would certainly not want to make it difficult to measure the changing system performance in order to guide changes.22 If the findings are largely shunned as a way to gain reliable and hence publishable knowledge, then it would be a troublesome irony to require review as if the project were research.
MOVING FORWARD
Coming to a workable accord on how to allow quality improvement to proceed responsibly has recently become much more urgent. More quality improvement projects are being delayed or stopped by requiring review under regulations for “human subject research”. At the same time, the research review process itself is often overwhelmed and underfunded. Requirements for protecting privacy have also added a new layer of problematic regulation to clinical practice, research, and improvement activities,12,23 yet public and professional interest in serious reform is burgeoning.24–26
Key messages
-
Requiring quality improvement projects to go through formal review as research has become more frequent, is often onerous and time consuming, and imposes rigidity on the improvement process that threatens its success.
-
Yet subjects are often put at risk in improvement projects, and no reasoned strategy is in place to assure that the process meets ethical requirements.
-
Various efforts to articulate a clear dividing line between research and quality improvement have yielded disparate answers, none of which appears to be straightforward.
-
While research is usually seen as optional, quality improvement might well be more often morally required, both for the patient as citizen and for the care provider as professional responsibility.
-
Practitioners of quality improvement and research often see quality improvement as being characterized by relatively rapid feedback of trends and effects to shape ongoing changes, which is not acceptable in research.
-
Various parties have different views on these issues, and a project to develop scholarship and understanding is underway, led by The Hastings Center in New York.
At least until stymied by pre-emptive ruling or regulation, we still have the opportunity to “get it right”. When I started asking leaders in quality improvement, ethics, public policy, and research about this set of problems about 5 years ago, every leader had a strong view and little awareness that others had strikingly different strongly held views. People are mostly involved in one or another of the affected enterprises and see the issues only from that one perspective. If leaders of ethics, privacy law, research regulation, improvement activities, and health services research could quickly gain wider knowledge and generate shared wisdom, we may be able to reshape current rules and interpretations to protect participants in research and improvement activities from harm, without unduly discouraging activities that make the care system more effective, reliable, and efficient. To that end, the Agency for Health Care Quality and Research has partially funded a series of meetings and papers under the leadership of The Hastings Center, the well known bioethics think tank in New York. The project will have a working online conference for interested parties. Readers can sign on by going to www.medicaring.org and going to “online conference”. Information is available from infomedicaring.org.
Generating a wise consensus on policy in this area depends upon quick action. If some adverse ruling pushes quality improvement toward research review, improvement will be greatly slowed and professional responsibility for quality will wither. If congressional or executive branch action exempted quality improvement from oversight, patients will be left at risk of harms. Surely we can craft a workable approach, but doing so will require thoughtful leadership, development of a body of scholarship, and possibly trials of various strategies in small systems which would be an intriguing and important application of quality improvement to refinement of its own procedures and regulatory policies.
REFERENCES
Footnotes
-
See editorial commentary, p 11
Linked Articles
- Quality lines
- Commentary