This paper considers how educational interventions should be evaluated for their effectiveness. Five levels of effectiveness are clarified and illustrated: outcomes, behaviour, learning, reaction and participation. These levels are then discussed, within the context of research evidence for education within the medical profession. Methodological and practical research conclusions are then drawn. From an analysis of over 300 abstracts the evidence shows that only limited research on healthcare outcomes has been undertaken, and there is considerable ambiguity over what constitutes an effectiveness measure. The benefits of a consistent set of terms are then considered. Finally, five key messages are distilled from the discussion and summarized.