Use of the MERSQI to Evaluate ABFM Journal Club Articles

Context: To augment our research curriculum, our three family medicine residency programs participated in the American Board of Family Medicine (ABFM) Journal Club Pilot. We implemented the Medical Education Research Study Quality Instrument (MERSQI) to score journal articles and identify potential curriculum gaps.

Objective: 1) summarize the body of literature included in the ABFM Journal Club Pilot by scoring each article for methodological quality; 2) identify research curriculum strengths and areas for growth

Study Design: A bibliometric analysis was conducted to examine the methodologic quality of research studies included in the ABFM Journal Club Pilot. Additionally, we surveyed residents to document their confidence critically appraising journal articles.

Dataset: MERSQI quality scores were calculated for 40 studies published in 25 journals selected by the ABFM.

Intervention/Instrument: The 10-item MERSQI was used to assess methodological quality across six domains: study design, sampling (number of institutions and response rate), type of data, validity (internal structure, content, and relationships to other variables), data analysis (appropriateness and complexity), and outcomes. Previous studies document strong validity evidence for the MERSQI. The resident survey included 12 Likert scale items measuring confidence appraising different elements of journal club articles (e.g. interpreting confidence intervals, statistical power, etc.).

Results: MERSQI scores ranged from 13 to 18, with the average being 16.31 (higher scores indicate higher quality). A majority of articles (80%) implemented a randomized control trial. Most articles (82%) with a survey had a response rate of 75% or above. Most studies were multi-institutional (90%) and presented objective measurements (87.2%) as opposed to self-assessment data alone (12.8%). At baseline before implementing the journal club pilot, a majority of residents indicated they had none or minimal experience evaluating journal articles (n=22, 52.4%).

Conclusions: On average, ABFM journal club articles had relatively high MERSQI scores compared to other bibliometric analyses. The MERSQI was a useful tool to identify gaps in our journal club curriculum. This information may guide the selection of future journal articles and refine the curriculum moving forward.