Critical incident technique and explicitation interviewing in studies of information behavior
Introduction
The critical incident technique (CIT), which developed from work in the U.S. Army Air Forces Aviation Psychology Program by Flanagan (1954), is regarded as a flexible set of principles, to be modified for the situation under study. Explicitation, however, is relatively unknown beyond its country of origin, France.
The CIT was originally used to assess performance in professional practice; a recent example of this usage includes the performance of health professionals—for example, the prescribing decisions of physicians (Bradley, 1992) and the quality of care for cancer patients (Bjorklund, 1999). More complex sets of behavioral intentions can be examined, such as the factors that affect patient compliance (Stromberg, Bostrom, Dahlstrom, & Fridlund, 1999), customer perceptions and reaction across a range of service industries (de Ruyter, Wetzels, & van Birgelen, 1999), and the quality of nursing care (Redfern & Norman, 1999). Other applications include supported reflection for student learning (Naidu & Oliver, 1999). The aim in health services research is often a better understanding of the interaction between patient and professional, and therefore the behaviors (patient, professional, or both) that appear to lead to effective care outcomes. Similarly, the studies that examine service quality in hospitality research or other service industries are generally concerned with the dimensions of service quality and identification of the key factors that affect customers' positive and negative perceptions.
Flanagan (1954) advocated five steps: (1) determine the general aim of the activity; (2) develop plans and specifications for collecting factual incidents regarding the activity; (3) collect the data (either through interview or written up by the observer); (4) analyze, as objectively as possible, and (5) interpret and report on the requirements, particularly those that make a significant contribution to the activity. According to these steps, studies of quality of care, or quality of service, can be categorized as follows:
- •
General aims. These require a care or service episode, which may be discrete, or what Strauss, Fagerhaugh, Suczek, and Wiender (1985) termed a more extended “trajectory of care” for chronic conditions, for example. These situations are nevertheless bounded, with some purposes, agreed to by the customers or clients and the service providers, and therefore they fall within Flanagan's specification. Some studies have included all stakeholders; others concentrated on provider or user; and one study, unusually, considered the impact of other customers on the service experience (Grove & Fisk, 1997).
- •
Plans and specifications. Flanagan indicated that for direct observation of behavior, observers should be familiar with the activity, the groups whose behavior is being studied should be specified, and the behaviors should be categorized, or some criteria developed. Health services researchers are frequently from the same profession as the group studied, but in other organization studies, researchers are more likely to be academic social scientists than business professionals.
- •
Observational and retrospective data collection. Flanagan implied that observational data collection is preferable, although retrospective data collection of incidents fresh in the mind of the study subjects is acceptable, if informed consent procedures assure the participants that they are not being singled out, and that they will remain anonymous. Flanagan noted (without reference to the specific studies) that slight changes in the question can produce substantial differences in the responses. He suggested that proper piloting is necessary, with checking of the interpretation of the question and instructions. The question should have a brief specification of the type of relevant behavior, and subjects should be restricted to recall of certain incidents, such as the most recent. In the actual interview, the subject should be allowed to do most of the talking to permit an unbiased account.
- •
Variations on data collection. Flanagan also mentioned group interviews (recent examples include Jackson & Stevenson, 2000, Liefooghe & Olafsson, 1999) and questionnaires.
- •
Scope of studies. Most recent studies examined between 50 to 100 incidents, although more complex activities may require several thousand incidents. Few studies can afford the latter level of research effort, and very few studies exceed 300 incidents for analysis. Each research subject usually contributes one or two incidents, although Janson and Becker (1998) used three incidents per subject. Several studies do not detail how piloting was achieved, and more emphasis seems to be put on the data analysis, whether through content analysis, phenomenonological approaches Chell, 1998, Johnson & Fauske, 2000, or grounded theory. In several health care studies Luker et al., 2000, Redfern & Norman, 1999, the emphasis is on delineation of the positive and negative quality indicators. A similar emphasis is evident in some educational studies (e.g., good and bad supervisory behaviors; Curtis, Helion, & Domsohn, 1998) and business research (e.g., attribution of positive and negative tourism experiences; Jackson, White, & Schmierer, 1996). The depth of detail that can be provided by the subject appears important (Kemppainen, 2000).
- •
Data analysis. This includes the frame of reference, or what the underlying purpose or purposes might be; the formulation of categories (acknowledged to be a subjective process); and general behaviors (the appropriate level of specificity). Several of the studies (e.g., Callan, 1998) could be viewed as extended pilot studies to refine the research methods and define provisional classification of categories. Several studies Runeson et al., 2001, Vettor & Kosinski, 2000, Von Post, 1998 described the research approach as the analysis of “stories” of care. In others, the quality of the dilemma itself is of interest, as in Bendtsen, Hensing, McKenzie, and Stridman's (1999) study of prescribing of analgesics in primary care. The organizational behavior studies often use a quantitative approach, to study the components of a transaction (O'Driscoll & Cooper, 1996) or citizenship behavior (Skarlicki & Latham, 1995).
- •
Interpreting and reporting, with full acknowledgment of limitations (e.g., researcher bias and groups that are not representative). Typically, the difficulty is one common to many qualitative research studies, that of coder interreliability, a problem acknowledged by Perry (1997) and addressed by Kemppainen, Levine, Mistal, and Schmidgall (2001).
The preponderance of critical incident studies in the health services research area is notable. Bradley (1992) suggested that the perennial popularity of anecdotes in medicine is because they make interesting stories and that as “stories” they offer insights to a social anthropologist about the social and conceptual world inhabited by the group. Some of the studies, as indicated previously, capitalize on this approach to understanding the nature of care. More particularly, perhaps, despite the bandwagon of evidence-based practice, the continuing popularity of the case report, the medical case presentation, or the nursing care study as formats for disseminating knowledge suggests that health professionals structure their knowledge according to story schemata (Urquhart, 1998) or illness scripts (Schmidt, Norman, & Boshuizen, 1990). Health-related incidents and stories are perhaps more memorable to all concerned than is a visit to a supermarket.
The CIT relies on recall of an actual event. In contrast, vignettes (Urquhart, 1999) may be used to examine the likely behavior of subjects in certain situations. The technique is useful when it is likely that attitudes or behavior would be less likely to be revealed using a direct approach. Subjects may be unwilling to be truthful when the situation is controversial.
Section snippets
Explicitation
The explicitation data collection technique (Vermersch, 1994) has some similarities with the retrospective data collection use in the CIT. What emerges from studying explicitation is a thorough and specific set of guidelines that are largely compatible with the goals of Flanagan and others, but which, in emphasizing how data are gathered and in providing theoretical grounding for why this should be so, also offers an interesting tool for examining purposes in gathering qualitative data on
Explicitation and Flanagan's five vital steps
For comparison with the CIT, Flanagan's (1954) five vital steps are used to explore the scope of the explicitation method:
- •
General aims. Any event, phenomenon, activity, or process can be investigated using the technique, but participants in an explicitation study only give an account of one instance of experience. This agreement forms part of the contract made with interviewees, and interviewers may remind interviewees of this in the interview if the interviewer suspects a more general
Review of the use of cit in studies of information seeking and use
CIT information behavior studies frequently occur in the health sector. Similar to an earlier small-scale study by Northup, Moore-West, Skipper, and Teaf (1983), many of these studies examine the motivations for information seeking, urgency of the request, type of information required, sources used, and reasons for selecting those sources (see Table 1).
Larger-scale studies include a National Library of Medicine study of the use of MEDLINE among 552 health professionals (mostly physicians), who
Use of the CIT in value and impact studies
The CIT has been developed in different ways for various studies of the effectiveness, value, and impact of service provision. Unlike the market research literature, most of these studies do not attempt to determine the dimensions of an information service encounter in terms of what was good or bad, but more often the aim is to determine the outcome of information use, and hence, by implication, the value ascribed to the service providing the information. Examples of this approach include
The JUSTEIS project
The JUSTEIS project Armstrong et al., 2000, Armstrong et al., 20011
Problems of integrating CIT interviews and questionnaires
The variations between the questionnaire and interview responses concerning particular EIS used for the critical incident seem to reflect Flanagan's (1954) observation that subtle changes in the wording of the question can produce large changes in response. If respondents are unclear about their own classification and description of EIS, then it is vital that interviews form a substantial component of any survey work, to clarify the terminology used by users to describe particular EIS. It would
Adapting the CIT to an evaluation project (VIVOS)
The Value and Impact of Virtual Outreach Services (VIVOS) project3 Yeoman et al., 2001a, Yeoman et al., 2001b used the CIT in its evaluation of outreach services in health libraries. The VIVOS research team worked closely with librarians at
Explicitation: Text-entering project
The explicitation technique recently was used in a study of the mental processes of experienced users when performing the types of Web-based task required on dynamic pages (Light & Wakeman, 2001). The study examined how people responded to entering text into interactive components on Web sites, such as comment boxes, search fields, and order forms. The explicitation technique was used to describe the following: (1) thoughts that went through the minds of the users as they approached and started
Conclusion
Explicitation has developed along parallel tracks to some of the critical incident studies, but there is a far stronger emphasis on the processes required for evocation to encourage full recall. What explicitation offers is an approach that might disentangle some of the deeper-seated perceptions from the information user. The approach appears to encourage a better recall of the episode and some of the steps taken. The particular difficulties encountered in the JUSTEIS project suggests that the
Acknowledgements
The authors thank the Joint Information Systems Committee of the Higher Education Funding Councils for their support of JUSTEIS and Resource (formerly the Library and Information Commission) for their funding of VIVOS. The authors also wish to thank the JISC Scientific Adviser Professor Jenny Rowley for her support on the JUSTEIS project and the School of Cognitive and Computer Sciences at the University of Sussex for its support for the text-entering project. Other members of the research team
References (74)
- et al.
Prescribing benzodiazepines—a critical incident study of a physician dilemma
Social Science and Medicine
(1999) The critical incident technique in hospitality research: An illustration from the UK lodge sector
Tourism Management
(1998)- et al.
Information needs and information-seeking behaviours of on-call radiology residents
Academic Radiology
(1999) - et al.
The impact of other customers on service experiences: A critical incident examination of “getting along”
Journal of Retailing
(1997) - et al.
Tourism experiences within an attributional framework
Annals of Tourism Research
(1996) - et al.
Beyond the interface: Users' perceptions of interaction and audience on Websites
Interacting with Computers
(2001) - et al.
Information behavior in dynamic group work contexts: Interwoven situational awareness, dense social networks and contested collaboration in command and control
Information Processing and Management
(2000) - et al.
Factors influencing patient compliance with therapeutic regimens in chronic heart failure: A critical incident analysis
Heart and Lung
(1999) - et al.
Information needs of physicians at the University Clinic Hospital in Valencia, Spain
Explanations, communication and social cognition