Cumulative Incidence of False-Positive Results in Repeated, Multimodal Cancer Screening

INTRODUCTION

Numerous cancer screening tests are promoted to the healthy public.^1–8 The motivating factor behind regular cancer screening is the theory that the earlier one detects a malignancy or premalignancy, the more likely treatment is to be effective in increasing lifespan while minimizing harms caused by the therapy.⁹ Although this model has intuitive appeal, it is often used without actual proof in hand and without full consideration of potential adverse consequences. The most common potential adverse consequence is a false-positive result, which often brings with it physical, psychological, and economic burdens of further diagnostic testing.^10–13

The false-positive rate of a single screening test has been studied, but the cumulative false-positive rate of repeating the test at regular intervals is infrequently reported,^14–17 and the cumulative false-positive rate of multiple tests has not, to our knowledge, been reported at all. The ongoing Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial is designed to assess the benefits and harms of screening for 4 major causes of cancer mortality. As such, it represents an ideal opportunity to assess the cumulative false-positive rate and resulting diagnostic procedure rates of a combined-modality screening program.

METHODS

Methods for the PLCO False-Positive Screening Study

Study Population and Eligibility

Enrollment exclusions used by the PLCO trial as a whole applied. We began with the intervention arm of the PLCO trial (n = 77,464). Additional exclusion criteria for this study were (1) death before the first screening test (n = 39); (2) missing all screening tests (n = 5,028); and (3) inadequate follow-up time (less than 3 years after the last screening test taken) (n = 3,961) (Figure 1⇓). We observed the remaining 68,436 participants up to a maximum possible 14 tests (although not all participants consented to or received all 14 examinations). At the time of the initial screening and 3 years later, participants were offered all 4 tests (chest radiograph, flexible sigmoidoscopy, CA-125, and transvaginal ultrasonography for women; and chest radiograph, flexible sigmoidoscopy, PSA, and DRE for men); 1 and 2 years after the initial screening, participants were offered all tests except flexible sigmoidoscopy.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Study population selection.

Flexible sigmoidoscopy frequency was decreased from every 3 to every 5 years during the trial to better match changing clinical practices. Given our 3-year timeframe, flexible sigmoidoscopies offered at year 5 were not included in this study. Approximately 25% of participants had the opportunity to undergo 2 flexible sigmoidoscopies; the rest were scheduled for only 1 (at baseline). As a posteroanterior chest radiograph was offered only twice after the baseline examination for nonsmokers, not all participants underwent the fourth chest radiograph at year 3.

RESULTS

The demographic characteristics of the PLCO trial have been described elsewhere.^23,29 There were 68,436 participants who underwent at least 1 screening test and had adequate follow-up (Figure 1⇑). Most participants (65%) ranged in age from 55 to 64 years, were white, non-Hispanic (88.3%), and had a college education or higher (57.5%). Participant sex was equally distributed (50% men, 50% women).

Cumulative Risk of a False-Positive Result

For men the risk of having at least 1 false-positive finding is 36.7% (95% CI, 36.2%–37.3%) by the 4th screening test (the end of the first day of screening), and 60.4% (95% CI, 59.8%–61.0%) by the 14th examination. Women have a 26.2% risk (95% CI, 25.7%–26.8%) by the 4th screening test, and 48.8% risk (95% CI, 48.1%–49.4%) by the 14th examination. Sensitivity analysis by modified Kaplan-Meier approach provides similar results at 14 tests: 57.2% for men (95% CI, 56.3%–58.2%) and 44.1% for women (95% CI, 43.4%–44.7%). The cumulative risk of receiving at least 1 false-positive result in the 14 tests is displayed in Figure 2⇓, panel A. The exact estimates and 95% confidence intervals for each test number (t) can be found in Supplemental Table 1, available online at http://www.annfammed.org/cgi/content/full/7/3/212/DC1.

• Download figure
• Open in new tab
• Download powerpoint

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Cumulative probability of a false-positive result for a multimodal cancer screening regimen.

When comparing individual screening modalities, flexible sigmoidoscopy accounted for the highest cumulative false-positive risk: 41.8% (95% CI, 40.9%–42.7%) for men and 29.2% (95% CI, 28.3%–30.0%) for women after 2 examinations. Chest radiograph accounted for the next highest false-positive risk, with a 22.3% (95% CI, 21.4%–23.2%) probability after 4 tests for men and a 21.5% (95% CI, 20.6%–22.4%) probability for women. The cumulative risks of receiving at least 1 false-positive result by screening modality is displayed in Figure 2⇑, panel B. The exact estimates and 95% confidence intervals for each test number (t) can be found in Supplemental Table 2, available online at http://www.annfammed.org/cgi/content/full/7/3/212/DC1.

Cumulative Risk of an Invasive Diagnostic Procedure Resulting from a False-Positive Result

For a woman the cumulative risk of undergoing a false-positive–prompted invasive diagnostic procedure* is 12.3% (95% CI, 11.8–12.8%) after 4 tests, and 22.1% (95% CI, 21.4%–22.7%) after 14 examinations. For men, the risk is slightly higher: 17.2% (95% CI, 16.7%–17.6%) after 4 tests, and 28.5% (95% CI, 27.8%–29.3%) after 14 examinations. Sensitivity analysis confirmed similar percentages: for women, the risk after 14 tests is 20.6% (95% CI, 20.4%–20.9%); for men, 27.5% (95% CI, 27.2%–27.9%). The cumulative probability of receiving at least 1 invasive diagnostic procedure is displayed in Figure 3⇓, panel A. The exact estimates and 95% confidence intervals for each test number (t) are shown in Supplemental Table 3, available online at http://www.annfammed.org/cgi/content/full/7/3/212/DC1.

• Download figure
• Open in new tab
• Download powerpoint

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Cumulative probability of receiving an invasive diagnostic procedure^a as a direct result of a false-positive test in a multimodal cancer screening regimen.

We also compared cumulative risks of false-positive–prompted invasive diagnostic interventions by individual modalities. Flexible sigmoidoscopy accounts for the greatest probability of a false-positive–prompted invasive diagnostic procedure: a 30.1% (95% CI, 29.2%–31.1%) probability after 2 tests for men, and a 22.1% (95% CI, 21.2%–23.0%) probability for women. A false-positive transvaginal sonogram is associated with the next highest risk, with a 6.7% (95% CI, 5.6%–7.7%) probability after 4 tests. The probability of receiving at least 1 invasive diagnostic procedure by screening modality is displayed in Figure 3⇑, panel B. The exact estimates and 95% confidence intervals for each test number (t) are shown shown for all modalities in Supplemental Table 4, available online at http://www.annfammed.org/cgi/content/full/7/3/212/DC1.

DISCUSSION

The cumulative risk of an individual obtaining a false-positive result in a multimodal screening program increases with number of screening tests; by the 4th screening test—which in the PLCO trial would mean the end of day 1—the risk is about 37% for men and 26% for women. By the 14th test, the risk is approximately 60% and 49% for men and women. The risk of undergoing any false-positive–prompted invasive diagnostic procedure is about 17% for men and 12% for women after 4 tests, and 29% for men and 22% for women after 14 screening tests.

Historically, recommendations for individual screening modalities have varied and will likely continue to vary across health care systems. The overarching principle shown by this study, however—that of rapidly increasing cumulative false-positive rates associated with multiphasic screening—remains important and underappreciated when recommending screening. To our knowledge, this study is the first to use randomized trial data to estimate the general magnitude of false positives that might be expected with a multimodal screening program; the fundamental approach and techniques used in this work should be applicable across a wide range of screening regimens.

The PLCO trial does not include an evaluation of mammography; however, as this modality is in widespread use, the associated false-positive rates are of clear importance to women and clinicians. Elmore et al estimated the cumulative incidence of false positives associated with mammography to be 49.1% after 10 examinations.¹⁴ To put this estimate in context with our own findings, the cumulative incidence of a false-positive mammogram was estimated at approximately 18% after 3 examinations (the maximum number that would have been observed during our study’s timeframe). Although this percentage is not directly additive to our cumulative incidence curve, it is a mathematical certainty that adding another testing modality would increase the final risk of obtaining at least 1 false positive for women.

Our study was a test-to-event rather than time-to-event analysis: we included any participant who took at least 1 screening examination sometime during the course of the 3 rounds (or 14 possible tests), and we analyzed the test results by total number rather than by time elapsed to allow for appropriate handling of missing tests while maximizing the use of available data. Individuals in the intervention arm who received no screening examinations were not included in the results: we chose this approach because we believe it to be most accurate and informative for the real-world scenario of a patient visiting a clinic with the intention of being screened and who wants to understand the risks involved if he or she undergoes at least some proportion of a multiphasic program of screening.

Additionally, not all participants received all 14 tests; as a result, the calculated false-positive cumulative incidence curves may underestimate true false-positive rates for a multiphasic screening program with 100% compliance. We believe, however, that the population included in this analysis—with good but not perfect compliance—and the cumulative risks obtained from this population best approximate what could be expected in real-world practice.

Because a given individual could receive a different screening modality at each point along the cumulative risk curves, the curves show the general accumulated burden better than they serve as precise indicators of risk at a given time. That is, the curves provide rule-of-thumb estimates at the point of each screening examination taken (t), but most accurately predict exact percentages at the end of the first day of screening and once all of the tests have been undergone (at t = 14). The curves provide rough approximations of time-to-event: if all tests are taken according to protocol, estimated rates at t = 4 are equivalent to the end of day 1 of testing, rates at t = 7 are equal to year 1 of testing, rates at t = 10 are equal to year 2 of testing, and rates at t = 14 are equal to year 3 of testing. Our methods, however, maximize the use of available data from variably compliant individuals and analyze tests by total number rather than elapsed time. For that reason, it is more accurate to interpret results by final number of tests taken than by years within the screening program.

A steep rise in the cumulative risk for false-positive examination results can be seen during the initial screening session (tests 1 to 4, Figure 2⇑, panel A) There is likely to be a subset of healthy individuals in the general population who do not have cancer but chronically express a benign abnormality detectable by blood test, radiographic test, or endoscopy. Upon initial screening, these individuals would likely have been immediately evaluated further, and once a false positive was identified, these individuals did not contribute further to the overall multimodal risk curve.

The classification of follow-up procedures into 3 categories is subjective in that the diagnostic tests performed were widely disparate; different clinicians may have conflicting opinions about what relegates a procedure to a minimally or moderately invasive category. For this reason, we have presented the full list of procedures and how they fit in our own classification system for the sake of transparency.

Clinicians may have varying thresholds for what they consider a true- vs false-positive result. Some researchers currently estimate a 5- to 15-year lead-time for PSA and prostate cancer.³⁰ Our study used a standardized 3-year negative follow-up period across all modalities to define a false positive; we did not feel it appropriate, absent definitive knowledge, to vary the time-interval criteria for a false positive by individual modality.

In the case of colorectal cancer, as previously noted, some clinicians argue that any adenoma, regardless of size, should be considered a true-positive result. We performed a sensitivity analysis to examine the effect that including any adenoma would have on false-positive rates in the PLCO trial. The percentage of men receiving false-positive flexible sigmoidoscopy results using this broader definition changes from 26.8% to 20.2%, and for women, from 17.2% to 13.5%. (In this situation, the remaining false positives encompassed such findings as inflammation, hyperplastic polyps, or a lack of demonstrable lesion on diagnostic follow-up.) The percentage of participants receiving at least 1 false positive of any modality shifts from 43.1% to 39.5% (for men, 50.9% to 46.6%; for women, 35.3% to 32.3%). If, conversely, the definition of a true-positive is restricted to overt cancer only, as some PLCO investigators favored, the percentage of men receiving at least 1 false-positive flexible sigmoidoscopy result increases to 30.3%; in women, 19.3%. For all modalities combined, the absolute percentage of participants receiving at least 1 false positive rises to 45.2% (men, 53.5%; women, 36.8%). Establishing a threshold for referral to colonoscopy after abnormal flexible sigmoidoscopy is complex. Randomized trials in Europe used biopsy at the time of flexible sigmoidoscopy to assist in triaging patients to reduce the number sent for colonoscopy,^31,32 an approach not used in the PLCO trial.³³ Ultimately, understanding the benefit of a higher compared with a lower rate of referral for colonoscopy after flexible sigmoidoscopy must await the outcome of the randomized trials.

Since the PLCO trial began, the use of flexible sigmoidoscopy in the United States has become less common, while the use of screening colonoscopy has increased substantially with the introduction of Medicare reimbursement coverage for this procedure.³⁴ The Centers for Disease Control and Prevention estimates that approximately 2.8 million flexible sigmoidoscopy examinations were performed in 2002; flexible sigmoidoscopy also remains a recommended screening modality by many professional organizations, including the US Preventive Services Task Force and, jointly, the American Cancer Society, the US Multi-society Task Force on Colorectal Cancer, and the American College of Radiology.^35–37 Because colonoscopy allows evaluation of more of the total area of the colon than does flexible sigmoidoscopy (and thus, has the potential to discover more abnormalities of uncertain significance), it is likely that substituting this modality would have increased the cumulative incidence of false positives. Our observed flexible sigmoidoscopy false-positive rate could thus be considered a conservative estimate of the false-positive risks for colonoscopy.

The rate of prostatic biopsy follow-up (ie, the percentage of moderately invasive procedures) for abnormal PSA results may appear low to some in the urologic community (44%). This percentage does not include men whose prostatic biopsies resulted in a diagnosis of cancer (ie, true positives), and thus it represents only a fraction of the total number of biopsies performed in the PLCO trial. Our cutoff point for an abnormal PSA was 4 ng/mL; however, because there is no absolute value below which a man can be assured he does not have prostate cancer,³⁸ some researchers and clinicians have used lower cut-points for screening (between 2.5 to 4 ng/mL). Attempting to increase sensitivity in this manner would likely increase the cumulative false-positive rates we report here.

Our study has several limitations. The PLCO trial examines a series of cancer screening tests, but not all of these modalities are used by a majority of the public. Nevertheless, all have been publicly advocated at one time by professional organizations or advocacy groups. The trial participants may not be entirely representative of the general population in that by agreeing to join a long-term cancer screening trial, they may be more involved in health promotion activities and may have higher screening compliance rates than nonparticipants.²⁹

The false-positive and diagnostic follow-up rates found in this study have important practical implications, especially given that the study was limited to 4 rounds of screening examations. Most formal screening recommendations are open-ended; none advise stopping after 4 rounds. Given the rate at which false-positive results can accumulate in multimodal screening programs and the potential iatrogenic burden these regimens can generate in healthy individuals, we propose that future guidelines begin determining risk-benefit equations for entire screening regimens rather than continuing to evaluate individual tests separately.

This study has developed a clearer picture in a previously unexplored aspect of burden and risks associated with scheduled multimodal cancer screening programs.

The benefit of regular combined-modality cancer screenings in reducing mortality is not yet known; we will not have that information until the conclusion of the PLCO trial. Physicians and patients must therefore examine the balance of known risks vs potential benefits to determine the most appropriate course of action for each individual.