The Delphi method as a research tool: an example, design considerations and applications
Introduction
The Delphi method has proven a popular tool in information systems (IS) research [4], [6], [13], [14], [16], [24], [25], [35]. Citing “a lack of a definitive method for conducting the research and a lack of statistical support for the conclusions drawn,” Schmidt [34] presented a step-wise methodology for conducting such studies. Building on the framework that Schmidt developed, we offer two contributions towards increasing the value of Delphi studies in investigating research questions. First, we fill in many details in the context of Schmidt’s framework by providing guidelines on how to conduct a rigorous Delphi study that identifies the most important issues of interest by soliciting qualified experts. Second, we demonstrate how to use a Delphi survey as a research tool to serve a variety of different purposes in the theorizing process. Increasing the rigor will increase the confidence with which researchers can use the results in subsequent studies and managers can make decisions based on information gathered using these methods.
Bricolage is a French term that means “to use whatever resources and repertoire one has to perform whatever task one faces” [40]. Characterizations of the research process as bricolage and the researcher as bricoleur [10] serve to remind us of the improvisation and opportunism inherent in the research process and the need to put our research tools to multiple use. A third goal, then, is to encourage researchers to incorporate the Delphi method into their research repertoire and to suggest some of the various ways they could apply the method in the theorizing process.
Section snippets
Overview of the Delphi method
The Delphi method originated in a series of studies that the RAND Corporation conducted in the 1950s. The objective was to develop a technique to obtain the most reliable consensus of a group of experts [8]. While researchers have developed variations of the method since its introduction, Linstone and Turoff [17] captured common characteristics in this description:
Delphi may be characterized as a method for structuring a group communication process so that the process is effective in allowing a
Example research study design using the Delphi method
Schmidt presented a guideline focusing on the major phases of the process and on analysis issues. However, the example we present in this paper focuses on perhaps the most important yet most neglected aspect of the Delphi method—choosing appropriate experts. This neglect is problematic, considering that most Delphi researchers characterize the technique as a method for soliciting information from experts. We based our guidelines primarily on those initially developed by Delbecq et al. [9].
Conclusion: uses of the Delphi method for theory building
The Delphi method is a versatile research tool that researchers can employ at various points in their research. Use of the Delphi method for forecasting and issue identification/prioritization can be valuable in the early stages, particularly in selecting the topic and defining the research question(s). A major contribution of the MISRC/SIM studies of key managerial IS concerns, published in MIS Quarterly, [5], for example, was to provide IS researchers with an understanding of the most
Acknowledgements
We thank Victor Mbarika for his help in the e-commerce study that we used as an illustrative framework for this paper. We also thank Casey Cegielski for his valuable references that gave us a literature base to study the Delphi method.
Chitu Okoli is an assistant professor in the Department of Decision Sciences and Management Information Systems of the John Molson School of Business at Concordia University, Montréal, Canada. He obtained a PhD in 2003 from the Department of Information Systems and Decision Sciences at Louisiana State University, USA. He primarily researches applications of the Internet in developing countries, and strategically using the Internet for competitive advantage. In addition to the Delphi method,
References (41)
- et al.
A comparative analysis of critical issues facing Canadian information systems personnel: a national and global perspective
Information & Management
(2000) Managing information technology in modernizing “against the odds”: lessons from an organization in a less-developed country
Information & Management
(1998)Specification of a capability-based IT classification framework
Information & Management
(2002)Improving the quality of group judgments: social judgment analysis and the Delphi technique
Organizational Behavior and Human Performance
(1979)- et al.
Delphi: a re-evaluation of research and theory
Technological Forecasting and Social Change
(1991) - R.H. Ament, Comparison of Delphi forecasting studies in 1964 and 1969, Futures March, 1970, p....
- et al.
A systemic framework for the field of information systems
Database for Advances in Information Systems
(2001) - N.H. Barry, Nonparametric statistics, http://faculty-staff.ou.edu/B/Nancy.H.Barry-1/nonpar.html,...
- et al.
Key issues in information systems management: 1994–95 SIM Delphi results
MIS Quarterly
(1996) - et al.
Key issues in information systems management
MIS Quarterly
(1987)
International business and trade in the next decade: report from a Delphi study
Journal of International Business Studies
An experimental application of the Delphi method to the use of experts
Management Science
The physical infrastructure for electronic commerce in developing nations: historical trends and the impact of privatization
International Journal of Electronic Commerce
Knowledge manipulation activities: results of a Delphi study
Information & Management
SEER: a divergent methodology applied to forecasting the future roles of the systems analyst
Human Systems Management
Managing international data communications
Information & Management
Cited by (0)
Chitu Okoli is an assistant professor in the Department of Decision Sciences and Management Information Systems of the John Molson School of Business at Concordia University, Montréal, Canada. He obtained a PhD in 2003 from the Department of Information Systems and Decision Sciences at Louisiana State University, USA. He primarily researches applications of the Internet in developing countries, and strategically using the Internet for competitive advantage. In addition to the Delphi method, Chitu Okoli also uses quantitative surveys and stochastic simulation.
Suzanne Pawlowski is an assistant professor in the Department of Information Systems and Decision Sciences at Louisiana State University. She holds a PhD degree in Computer Information Systems from Georgia State University and MBA and BA degrees from the University of California, Berkeley. Research interests include enterprise systems and organizational learning, retention of IT professionals, and collaborative visualization technologies. Scholarly publications include papers in ACM Computing Surveys, Communications of the ACM, IEEE Transactions on Software Engineering, and Communications of AIS. Her previous work experience includes a 20-year career in IT at Lawrence Livermore National Laboratory, including manager of application development.