Abstract
PURPOSE The purposes of this study were to determine if (1) certain demographic characteristics (potential predictors) of participants, and (2) clock-drawing test results (as a screening test for cognitive impairment) were associated with fecal immunochemical test (FIT) sample collection errors.
METHODS Patients scheduled for an upcoming colonoscopy were asked to collect stool samples using 5 different FITs. Patients completed a questionnaire that included the clock-drawing test. Errors included mistakes or omissions in recording the stool collection date and errors in stool collection. Each clock drawing was scored by 2 reviewers using 2 established methods.
RESULTS Of the 1,448 participants with a clock drawing, 63% were female with a mean age of 63 years. In this population there were 83% White, 6% Black, and 24% Hispanic persons. Cognitive impairment was found in 292 patients by the Mendes-Santos method. Kappa coefficient for the 2 clock-drawing scores was 0.79 (P <.001). The multivariable generalized linear mixed model for FIT collection errors indicated being female (adjusted odds ratio [AOR], 1.64; 95% CI, 1.09-2.48), having an 8th grade or less education (AOR, 3.40; 95% CI, 1.87-6.18), and having an abnormal Mendes-Santos method clock score (AOR, 1.65; 95% CI, 1.08-2.54) were associated with significantly more errors.
CONCLUSION Among the participants who do not have dementia, FIT collection errors were made not only by those who had abnormal clock drawing, but also, by those with normal clock drawings. Subjects being female, having 8th grade education or less, and having an abnormal clock drawing scored by Mendes-Santos’s method were associated with FIT collection errors.
INTRODUCTION
Approximately 16% to 20% of people ages 60 or more years have mild cognitive impairment.1 Mild cognitive impairment is described as an intermediate state between cognitively intact persons and those with dementia.2 Individuals with mild cognitive impairment demonstrate impairment in any cognitive domain, such as memory, executive function, language, or visuospatial skills.1,3 Screening for colorectal cancer, which is recommended to start at age 45 years, can be accomplished by several methods including a fecal immunochemical test (FIT) or colonoscopy.4 Both require the ability to correctly follow a sequence of steps.
During a research study, patients can make mistakes that may be considered careless (eg, not answering a question on a survey). Mistakes made in our study, comparing the effectiveness of FITs with colonoscopy, included reporting an incorrect date, not including the date of the stool sample collection, or collecting the stool sample incorrectly.5 Forgetting to return a specimen or provide the date of stool collection are tasks in which the participant forgets to do something, possibly due to short-term memory loss or inattention.6 Collecting the stool specimen incorrectly demonstrates failure in executive function, since collection involves reading, understanding, and following instructions. It may also be due to an inability to consider the sequence of steps to complete the task or visual perception problems.2,7,8 An underlying cause for these mistakes may be mild cognitive impairment, major depression, a low education level, or being careless.9,10
Clock-drawing tests are used with batteries of other cognitive tests to discriminate among individuals with normal cognition, mild cognitive impairment, and dementia.11,12 Studies have demonstrated that individuals with cognitive impairment score worse on the clock-drawing test than individuals with normal cognition.13–18 In a mailed survey to determine if a clock-drawing test can be accomplished by mail, 4,843 community-dwelling persons aged 52 or more years, in a southern California retirement community, completed a clock-drawing test scored by 2 methods and found that 510 (11%) respondents had abnormal clock drawings.19 That study demonstrated that community-dwelling persons were able to complete a clock-drawing test by mail. The results indicated that for both males and females, the mean total clock scores worsened with each successive 5-year age group. In another study, a mailed questionnaire with a clock-drawing test was used to compare the accuracy of colonoscopy self-report with information from the medical record.20 Of the 493 respondents from 16 primary care offices across Iowa, 15% had abnormal clock drawings; individuals with normal scores had higher accuracy in self-report compared with their medical record.20
In the context of a large comparative effectiveness study of FIT testing, we sought to evaluate the possible contribution of impaired cognition on FIT test completion and errors. Our hypothesis was that impairment demostrated with a clock-drawing test may be associated with errors in FIT sample collection. The purposes of this study were to determine if (1) certain demographic characteristics (potential predictors) of participants, and (2) clock-drawing test results (as a screening test for cognitive impairment) were associated with FIT sample collection errors.
METHODS
This study used data from our comparative effectiveness study to assess test characteristics of 5 FITs used for detection of advanced colorectal neoplasia, using optical colonoscopy as the gold standard.5 Healthy patients scheduled for screening or surveillance colonoscopy were asked to collect stool samples using 5 different FITs from 1 bowel movement. Detailed step-by-step written and pictorial directions for collecting each of the FITs were provided. Individuals aged 50 to 85 years who could read/write in English or Spanish were eligible. Individuals with medical diagnoses indicating cognitive impairment were not invited to participate. Institutional Review Board approval was received for this study at each of the 3 participating sites: University of Iowa at Iowa City, Iowa; Univeristy of North Carolina at Chapel Hill, North Carolina; and Texas Tech University Health Sciences Center at El Paso, Texas. Study methods for the comparative effectiveness project have been described previously and are summarized in Supplemental Appendix 1.5
Health Questionnaire
Each participant completed a 19-item health questionnaire that was adapted from our previous research.21 Questions focused on the results of recent colorectal cancer screening tests, family history of colorectal cancer, perception of general health, medication use, smoking status, and demographics.
Clock-Drawing Test
On the 13-item investigator-developed product questionnaire designed to assess perceptions of ease of stool collection using the various FITs, the final 2 questions instructed patients to “Please draw the face of a clock with numbers and hands to signify the time ten minutes after eleven” and to indicate if anyone helped them draw the clock. The questionnaire included a pre-drawn circle. Two scoring methods, Watson et al17 and Mendes-Santos et al22 (referred to as the Watson method and Mendes-Santos method) were used to analyze the clock drawings (See Table 1 for scoring and adaptations).
Scoring Rubric for Mendes-Santos and Watson Methodsa
Clock scoring was done by authors J.M.D., M.S., and P.K., and acknowleded clock scorers, T.S. and G.J. The researchers who completed the clock-drawing scoring were trained by the geriatrician (G.J.) who facilitated consensus when needed. All scorers were blinded to the patients’ study errors. Two different scorers scored each clock using each method. When consensus was not reached, a third scorer scored the clock, and the 3 scorers discussed the clock until consensus was reached. If clocks were not attempted, they were coded as “missing.”
Errors in FIT Collection
The errors in FIT collection included: (1) failure to return a vial/dry-slide card, (2) mistakes or omissions in recording the stool collection date, and (3) errors in the collection of the stool on the vial or dry-slide card. The stool collection products included 4 vials with screw-on caps, and 1 dry-slide card. While patients may have made 1 or more errors, for data analysis the presence of any error on any one of the products was considered a “yes” for stool collection error. Most returned FITs were still testable for analysis of occult blood even when collection errors were made. Each of the 5 FITs had collection errors (Supplemental Appendix 1).
Statistical Analysis
The outcome variable for this study was the FIT collection errors. The clock-drawing test scores were collapsed into dichotomous variables. For the Watson method, scores of 0-3 were considered normal and scores of 4-7 were considered abnormal.17 For the Mendes-Santos method, scores of 6-10 were classified as normal and scores of 1-5 abnormal.22 Error in FIT collection was coded yes/no.
Standard descriptive statistics were used to summarize the variables. Chi-square tests were used for comparison of categorical variables between groups. The kappa statistic was used to assess agreement between the 2 dichotomous clock-drawing scores. Kappa values of 0.81 and above represent almost perfect agreement. Substantial agreement values were 0.61-0.80, moderate agreement values were 0.41-0.60, fair agreement values were 0.21-0.40, and slight agreement values were 0-0.20.23
The generalized linear mixed models using the SAS GLIMMIX (SAS Institute Inc) procedure were used to examine a binary outcome to indicate if having any errors in FIT collection was associated with demographic factors and each of the clock-drawing scores. The location variable of site (Iowa, North Carolina, or Texas) was specified as the random intercept in the model to account for the correlation among participants at a site. Each of the potential associated factors was tested individually with the outcome variable in the model. Variables tested included patient’s age category, sex, race, ethnicity, household income, and each of the 2 clock-drawing scores separately. Variables with P <.15 in these univariable predictor models were included in the multivariable analyses, and age was forced into the final model because of its clinical implications. Subsequently, the backward stepwise method was used to remove the variables one at a time, and variables with P <.05 were considered significant in the final multivariable model. Interaction terms were tested in the multivariable model. All analyses were performed using SAS version 9.4 (SAS Institute Inc).
RESULTS
A total of 1,490 participants completed the informed consent, health questionnaire, FITs, and product questionnaire. Of these, 28 (1.9%) did not attempt the clock drawing and 14 (0.9%) reported they had help drawing the clock. Those 42 individuals were excluded from analysis which left a total of 1,448 participants. The mean age was 63 years (SD 8 years), 908 (63%) were female, and 354 (24%) participants identified as Hispanic. The group had 1,201 (83%) White people, 94 (6%) Black, and 153 (11%) of other races (Native American/Alaska Native, Asian, Native Hawaiian/Other Pacific Islander). There were 75 (5%) with an eighth-grade education or less and 573 (41%) with an income of $80,000 or higher (Table 2).
Demographic Characteristics of Study Participants (N = 1,448)
Overall, 153 (10.6%) participants made 1 or more errors in FIT collection. The 1,448 participants were sent 7,240 FITs and 7,234 (99.9%) were returned. Six (0.4%) participants failed to return a vial or dry-slide card, 22 (1.6%) made a mistake or omission in recording the stool collection date; 133 (9.2%) made 145 errors in the collection of the stool, with 10 (0.7%) making more than 1 collection error. The most frequent errors for FITs were too much stool in the vial requiring a repeat of the analysis for 34 (2.3%) of the 1,448 participants, stool on wrong end of the vial by 29 (2.0%), stool on wrong side of the slide by 13 (0.9%), stool collected incorrectly on slide by 11 (0.8%), the buffer tip was broken off by 9 (0.6%), and 42 (2.8%) made other errors in the collection of the stool. Of the 133 participants that made collection errors, 62 (4.3%) had at least 1 FIT that was not testable.
Clock-drawing results indicated 292 (20%) participants had cognitive impairment by the Mendes-Santos method, and 378 (26%) by the Watson scoring method (Table 3). The kappa coefficient for the 2 clock-drawing scores was 0.79 (P <.001), indicating substantial agreement between the 2 methods.23 Demographic characteristics and errors associated with abnormal clocks were calculated according to each scoring method. Significant differences were found by the Watson method for age, sex, race, ethnicity, and income. Using the Mendes-Santos method, significant differences were found for age, sex, ethnicity, and income (Table 3). Examples of abnormal clock drawings are found in Figure 1.
Participant Clock Drawing Scores by Demographics (N = 1,448)
Examples of abnormal clock drawings.
In the univariate generalized linear mixed model for FIT collection errors, female sex, Hispanic ethnicity, and an 8th grade education or less compared with college or higher were significantly associated with having a FIT collection error. The Mendes-Santos score was close to being significant (P = .073). The multivariable generalized linear mixed model indicated being female (adjusted odds ratio [AOR], 1.49; 95% CI, 1.02-2.19), having an eighth grade education or less compared with college or higher (AOR, 3.76; 95% CI, 2.14-6.62), and having an abnormal clock by the Mendes-Santos score (AOR, 1.57; 95% CI, 1.05-2.36) were significantly associated with having FIT collection errors (Table 4). In the multivariable generalized linear models, the Mendes-Santos score and Watson scores were tested separately with other variables; the Watson score was not a significant predictor.
Generalized Linear Mixed Model of Factors Associated With Errors in FIT Collection (N = 1,448)
DISCUSSION
Use of FITs, as a stool-based screening test for colorectal cancer, is a cost-effective and potentially life-saving test. Screening helps to detect precancerous polyps and early-stage cancer when treatment has the greatest chance of success. Screening remains underused, and even when samples are collected many are unusable due to collection errors.24 This study examines factors that are associated with FIT collection errors with the hypothesis that undiagnosed mild cognitive impairment may be associated with collection errors. Individuals with a prior cognitive impairment diagnoses were excluded from the study.
We found an overall FIT collection error rate of 11% and an abnormal clock-drawing rate of 20% to 26% depending on scoring method. Our study showed the proportion of clock-drawing errors increased with increasing age, similar to other studies.25–27 Regardless of the type of clock-drawing scoring method, males were significantly more likely to draw an abnormal clock. Other studies have found that females do worse on the clock drawing compared with males, different from our results.17,28–30 Factors such as depression, caregiver stress, aspirin use, estrogen use, and smoking could be associated with our findings and would need further study, especially to determine if those factors influence our differing finding based on sex.19,31
Even after controlling for education, we found abnormal clock drawing was associated with FIT collection errors. Regardless of the type of clock-drawing scoring method, males were significantly more likely to draw an abnormal clock; however, females were more likely to make stool collection errors. Age, sex, education level, and abnormal clock drawings continued to be significantly associated with FIT collection errors. In an earlier study of 4,843 community-dwelling individuals aged 52 years and older, regression analysis showed a significant association for abnormal clock drawing with older age, lower education, and being female.19 In our results, abnormal clock drawing scores (by either scoring method) were significantly associated with educational level, with persons having an 8th grade educational level or less being more likely to draw an abnormal clock. Lower education level was a consistent predictor of poorer clock drawings in previous studies.19,32
Clock drawing and FIT stool collection involve a diverse range of cognitive abilities including comprehension, planning, visual memory and reconstruction, visuospatial abilities, motor programming and execution, numerical knowledge, abstract thinking, concentration, and frustration tolerance.11,33,34,35 The participants needed to be able to understand and follow directions for clock drawing and FIT collection. Without additional cognitive testing, it is unknown if participants who had trouble completing these tasks had mild cognitive impairments. Errors in the collection of stool samples may occur for many other reasons including arthritis, medication side effects, physical limitations, or vision problems.
Stool collection errors occur in research studies and in sample collection for routine laboratory tests.36 In a study where FIT samples were received from 1,871 patients, researchers found that 20% were not usable, with the majority failing to provide the stool collection date.36 At one laboratory, the colorectal cancer screening FIT rejection rate was 29%.37 The main causes for rejection were expiration of the FIT, no collection date recorded, no physician order, and no patient information, some of which were not the patient’s responsibility.37 To reduce the rejection rate, multiple interventions were implemented.37 In their study, these interventions reduced their rejection rate to 7%37 which was similar to our error rate, indicating our results are likely generalizable to the real world. Most errors in specimen testing occur in the pre-analytical phase which includes patient preparation and sample collection.38 In the clinical setting, accurate FIT collection is important for correct colorectal cancer screening.
Future research could evaluate other medical procedures patients complete at home, such as colonoscopy preparation, irrigating and changing a wound dressing, or medication schedules, to assess errors in completion. If problems occur for certain patients, exploring whether cognitive screening using clock drawing may be appropriate to identify individuals who may need help completing tasks.
A limitation of this study is that the clock-drawing test is only one component of a detailed exam to assess cognitive impairment and should not be used alone to classify someone with cognitive impairment. Any individual screened for mild cognitive impairment should undergo a complete history and physical examination including extensive cognitive testing, medication use, functional status, neurologic and psychiatric evaluation, and laboratory testing.39
Because the clock-drawing test was completed at the participant’s home, it is difficult to know for certain whether the respondent completed the clock themselves. Most respondents indicated “no” to the question “did someone help you complete this clock drawing?” This study is different from other mailed fecal occult blood test studies, in that each participant was asked to use 5 different FIT devices for stool collection, which is a more complex task compared with a typical take-home test where individuals are only asked to perform 1 test. Because the FITs were collected on the same stool sample, however, and had generally similar collection procedures, we believe this has minimal impact on the validity of our findings. Another limitation is that participants were not asked if they had help completing the stool collection. Strengths of this study include a relatively large racially and ethnically diverse population who completed a moderately complex task of collecting 5 FITs and a clock drawing.
CONCLUSION
In this study of patients (without dementia, serious psychiatric diagnoses, or paralysis) scheduled for screening or surveillance colonoscopies, FIT collection errors were made by those with both normal and abnormal clock drawings. This could indicate an inability to follow directions for complex tests, such as FIT collection. Thus, the clock-drawing test may be beneficial in the clinical setting when a patient is asked to complete medical tests or preparation for tests at home. Being female, having an 8th grade education or less, and having an abnormal clock drawing score by Mendes-Santos’s method were associated with FIT collection errors.
Acknowledgments
Clock scorers: Tia Saunders, BS, and Gerald Jogerst, MD, University of Iowa.
Footnotes
Conflicts of interest: authors report none.
Funding support: Research reported in this publication was supported by the National Institutes of National Cancer Institute R01 CA215034 (BT Levy, PI) and the National Center For Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR002537. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Trial registration: NCT03264898 (clinicaltrials.gov) – Comparative Effectiveness of FITs with Colonoscopy.
- Received for publication May 7, 2021.
- Revision received February 28, 2022.
- Accepted for publication May 11, 2022.
- © 2022 Annals of Family Medicine, Inc.
