Interventions to mitigate the impact of low literacy on patients' recall of information by simplifying language have had limited success. The current study examines the extent to which cognition explains the relationship between literacy and retention of health information. Primary care patients aged 40 to 85 years watched a video about colorectal cancer (CRC) screening and then answered knowledge-based questions about the video's content as well as a literacy assessment and cognitive assessments measuring processing speed, working memory, and-long term memory. A week later, available participants completed the knowledge assessment a second time. In regression models for immediate knowledge, literacy significantly predicted knowledge. However, once cognition (i.e., processing speed, working memory, and long-term memory) was added to the model, it explained 70.7% of the relationship between literacy and performance. A week later, literacy again significantly predicted knowledge, but entering cognition into the model explained 45.9% of the relationship between literacy and performance. These results suggest that cognition explains much of the association between literacy and both immediate and delayed recall of health information. Design and intervention strategies for educational tools should consider cognitive factors such as working memory demands in addition to focusing on the readability of materials.