Original article
Written case simulations: Do they predict physicians' behavior?

https://doi.org/10.1016/0895-4356(90)90241-GGet rights and content

Abstract

Written case simulations are often used to investigate physicians' decision making and clinical competence. Their use rests on the assumption that physicians' responses to written simulations closely agree with their responses to actual clinical encounters, yet this assumption of criterion validity has received little attention. To determine the ability of written case simulations to predict actual clinical behavior, we applied methodologic criteria to published articles that used written simulations. Only 11 (15%) of 74 articles included an assessment of the criterion validity of their written case simulations. Only 2 of those 11 studies were designed and executed in such a way that criterion validity could be fully interpreted. No clear consensus emerged from an examination on the 11 studies on how well responses to written case simulations perform as proxy measures of actual behavior. More work is needed before assuming that written case simulations measure actual behavior.

References (93)

  • T. Borkovec et al.

    The utility of analogue research

    Behav Res Ther

    (1979)
  • I. Ajzen

    Attitudes, traits, and actions: dispositional prediction of behavior in personality and social psychology

    Adv Exp Soc Psychol

    (1987)
  • R.M. Hartley et al.

    Case history questionnaires in the study of doctors' use of resources: are they measuring what we want?

    Med Care

    (1985)
  • J.J. Norcini et al.

    Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence

    Med Educ

    (1985)
  • A.R. Feinstein

    Clinimetrics

  • J.C. Nunnally

    Psychometric Theory

  • D.T. Campbell et al.

    Convergent and discriminant validation by the multitrait-multimethod matrix

    Psychol Bull

    (1959)
  • W.H. McCarthy

    An assessment of the influence of cueing items in objective examinations

    J Med Educ

    (1966)
  • J.I. Balla et al.

    Diagnostic cues in gastroenterology

    Aust NZ J Med

    (1983)
  • J. Benbassat et al.

    A comparison of initial diagnostic hypotheses of medical students and internists

    J Med Educ

    (1984)
  • S.P. Damrosch

    Graduate nursing students' attitudes toward sexually active older persons

    Gerontologist

    (1984)
  • S.P. Damrosch

    Nursing students' assessments of behaviorally self-blaming rape victims

    Nurs Res

    (1985)
  • J.G. Dolan et al.

    An evaluation of clinicians' subjective prior probability estimates

    Med Decis Making

    (1986)
  • J.A. Farrow

    The use of vignette analysis of dangerous driving situations involving alcohol to differentiate adolescent DWI offenders and high school drivers

    Am J Drug Alcohol Abuse

    (1987)
  • L.D. Gruppen et al.

    The influence of general and case-related experience on primary care treatment decision making

    Arch Intern Med

    (1988)
  • K. Hawton et al.

    The attitudes of psychiatrists to deliberate self-poisoning: comparison with physicians and nurses

    Br J Med Psychol

    (1981)
  • D. Hemenway et al.

    Testing for physician-induced demand with hypothetical cases

    Med Care

    (1985)
  • J. Hoey et al.

    Physician sensitivity to the price of diagnostic tests: a U.S. Canadian analysis

    Med Care

    (1982)
  • G.F. Longstreth

    Detecting colorectal neoplasms: assessment based on hypothetical cases

    J Clin Gastroenterol

    (1987)
  • C.A. Mancuso et al.

    A model for physicians' therapeutic decision making

    Arch Intern Med

    (1987)
  • F.K. Orkin et al.

    A study of decision making: how faculty define competence

    Anesthesiology

    (1978)
  • J. Overall et al.

    Decisions about drug therapy 11: expert opinion in a hypothetical situation

    Behav Sci

    (1972)
  • K.E. Rudestam et al.

    The clinical judgment process in the prescribing of psychotropic drugs

    Int J Addict

    (1981)
  • I.M. Rutkow et al.

    Surgical decision making: the reliability of clinical judgment

    Ann Surg

    (1979)
  • R. Saylor-Bronstein et al.

    The use of written simulations and trained patients to evaluate a course in rheumatology

    J Med Educ

    (1985)
  • D.L. Simel et al.

    Blood alcohol measurements in the emergency department: who needs them?

    Am J Public Health

    (1988)
  • W.M. Tierney et al.

    Jehovah's Witnesses and blood transfusion: physicians' attitudes and legal precedents

    South Med J

    (1984)
  • R.M. Wachter et al.

    Attitudes of medical residents regarding intensive care for patients with the acquired immunodeficiency syndrome

    Arch Intern Med

    (1988)
  • T.S. Wallsten

    Physician and medical student bias in evaluating diagnostic information

    Med Decis Making

    (1981)
  • D.T. Watts et al.

    Nurses' and physicians' attitudes toward tube-feeding decisions in long-term care

    J Am Geriatr Soc

    (1986)
  • J.S. Yudin et al.

    Interpreting survival rates for the treatment of decompensated diabetes: are we saving too many lives?

    Lancet

    (1987)
  • J.R. Clarke et al.

    Development of an objective rating system for residency applications

    Surgery

    (1984)
  • H.U. Fisch et al.

    An experimental study of the clinical judgment of general physicians in evaluating and prescribing for depression

    Br J Psychiat

    (1981)
  • H.U. Fisch et al.

    A cross-national study of drug treatment decisions in psychiatry

    Med Decis Making

    (1982)
  • J.S. Gillis et al.

    Drug therapy decisions: a social judgment analysis

    J Nerv Ment Dis

    (1981)
  • Cited by (227)

    • Treatment manuals, training and successful provision of stop smoking behavioural support

      2015, Behaviour Research and Therapy
      Citation Excerpt :

      The analysis also relied on self-report. There is evidence that stop smoking practitioners over-report the extent to which they deliver behaviour change techniques (Lorencatto et al. 2014), and discrepancies between self-reported and actual practice have been documented more generally for physicians (Cabana et al. 1999; Jones, Gerrity, Earp, 1990). However, self-reported success rates have been shown to be reasonably valid (Brose, West, Michie, & McEwen, 2013).

    View all citing articles on Scopus

    Presented in part at the February 1988 Meeting of the Southern Region of the Society of General Internal Medicine, New Orleans, Louisiana.

    View full text