Home   >   CSC-OpenAccess Library   >    Manuscript Information
Delphi Method: A Comprehensive Literature Review
Rafa E. Al-Qutaish
Pages - 1 - 15     |    Revised - 31-12-2024     |    Published - 01-02-2025
Volume - 12   Issue - 1    |    Publication Date - February 2025  Table of Contents
MORE INFORMATION
KEYWORDS
Delphi Method, Structuring, Collecting Experts' Opinions
ABSTRACT
The Delphi method, originally conceived in the 1950s by Dalkey and Helmer, has emerged as a robust and versatile technique for eliciting expert opinions to tackle complex and multifaceted problems. It was first implemented in a study conducted by the Rand Corporation, aimed at forecasting the impact of technology on warfare. Over the years, its framework has been systematically refined and adapted by numerous researchers, leading to its establishment as a reliable and standard research tool across a broad spectrum of disciplines. Linstone and Turoff, notable contributors to the evolution of this method, describe it as a structured communication process designed to achieve group consensus through iterative rounds of feedback while preserving the anonymity of participants. This critical feature ensures that individual opinions are uninfluenced by group dynamics or hierarchical pressures, fostering a more egalitarian and unbiased exchange of ideas. Subsequent refinements to the method have further emphasized its utility in gathering informed judgments from experts using structured questionnaires, enabling a systematic exploration of diverse perspectives. The iterative nature of the process allows for the gradual refinement of opinions, with participants revisiting their responses in light of feedback from the group. This approach not only facilitates consensus but also enhances the depth and precision of insights generated. The Delphi method has gained particular acclaim for its effectiveness in addressing long-term forecasting and strategic decision-making challenges. It excels in distilling intricate and often tacit knowledge into coherent, actionable insights, making it invaluable in areas where direct empirical data may be limited or unavailable. This paper aims to provide a comprehensive exploration of the Delphi method, beginning with its historical development and foundational principles. It will examine the method's inherent advantages, such as its ability to harness collective intelligence while minimizing biases, alongside its limitations, including potential challenges in expert selection and response attrition. Furthermore, the paper will delve into the various adaptations of the method, such as the Real-Time Delphi, Policy Delphi, and Decision Delphi, each tailored to specific research contexts. Finally, it will analyze the diverse applications of the Delphi method across disciplines, ranging from healthcare and education to technology forecasting and public policy, underscoring its enduring relevance as a powerful tool for structured problem-solving and collaborative inquiry.
Ali, A. K. (2005). Using the Delphi Technique to Search for Empirical Measures of Local Planning Agency Power. The Qualitative Report, 10(4), 718-744. https://doi.org/10.46743/2160-3715/2005.1829.
Balasubramanian, R., & Agarwal, D. (2012). Delphi Tech-nique: A Review. International Journal of Public Health Dentistry, 3(2), 16-25. https://doi.org/10.12968/ijtr.2009.16.2.38892.
Baumann, M. H., Strange, C., Heffner, J. E., Light, R., Kirby, T. J., Klein, J., Luketich, J. D., Panacek, E. A., & Sahn, S. A. (2001). Management of Spontaneous Pneumothorax: An American College of Chest Physicians Delphi Consensus Statement. Chest, 119(2). https://doi.org/10.1378/chest.119.2.590.
Bazzani, C., & Canavari, M. (2013). Forecasting a Scenario of the Fresh Tomato Market in Italy and in Germany Using the Delphi Method. British Food Journal, 115(3), 448-459. https://doi.org/10.1108/00070701311314246.
Bevilacqua, M., Ciarapica, F. E., Giacchetta, G., & Marchetti, B. (2011). Implementation of a Quality Procedure Based on Delphi Method and the ISO/TS 16949:2009 in the Production of Stainless-Steel Tubes for Automotive Exhaust Systems. 28(8), 841-866. https://doi.org/10.1108/02656711111162514.
Buckley, C. (1995). Delphi: A Methodology for Preferences More than Predictions. Library Management, 16(7), 16-19. https://doi.org/10.1108/01435129510093737.
Campos-Climent, V., Apetrei, A., & Chaves-Avila, R. (2012). Delphi Method Applied to Horticultural Cooperatives. Management Decision, 50(7), 1266-1284. https://doi.org/10.1108/00251741211247003.
Chevron, J. R. (1998). The Delphi Process: a Strategic Branding Methodology. Journal of Consumer Marketing, 15(3), 254-264. https://doi.org/10.1108/07363769810219125.
Colton, S., & Hatcher, T. (2004). The Web-based Delphi Research Technique as a Method for Content Validation in HRD and Adult Education Research. the Academy of Human Resource Development International Conference, Austin, TX, USA.
Cuhls, K. (2003). Delphi Method. Fraunhofer In-stitute for Systems and Innovation Research.
Custer, R. L., Scarcella, J. A., & Stewart, B. R. (1999). The Modified Delphi Technique: A Rotational Modification. Journal of Vocational and Technical Education, 15(2), 1-8. https://doi.org/10.21061/jcte.v15i2.702.
Czinkota, M. R., & Ronkainen, I. A. (1997). International business and trade in the next decade: Report from a Delphi study. Journal of International Business Studies, 28(4), 827-844. https://doi.org/10.1057/palgrave.jibs.8490121.
Dalkey, N. C., & Helmer, O. (1963). An Experimental Ap-plication of the Delphi Method to the Use of Experts. Management Science, 9(9), 458-467. https://doi.org/10.1287/mnsc.9.3.458.
Day, J., & Bobeva, M. (2005). A Generic Toolkit for the Successful Management of Delphi Studies. Electronic Journal of Business Research Methods, 3(2), 103-117.
De Las Heras, J., Salvatore, R., Rodrigues, M. J., Lovreglio, R., & Leone, V. (2007). Wildfire Motivation Survey through the Delphi Method. Proceedings of the 4th International Wildland Fire Conference. , Sevilla, Spain.
De Meyrick, J. (2003). The Delphi Method and Health Research. Health Education, 103(1), 7-16. https://doi.org/10.1108/09654280310459112.
Di Zio, S., & Pacinelli, A. (2011). Opinion Convergence in Location: A Spatial Version of the Delphi Method. Technological Forecasting & Social Change, 78(9), 1565-1578. https://doi.org/10.1016/j.techfore.2010.09.010.
Fathian, M., Akhavan, P., Hoorali, M., & Jafari, M. (2007). Application of Delphi Technique for Development of E-Readiness Assessment Model: A Study on Small and Medium Enterprises of Iran. the Information Resources Management Association Conference, Vancouver, BC, Canada.
Faucher, J.-B., Everett, A. M., & Lawson, R. (2008). Applying a Modified Delphi Approach to Determine the Current State of the Concept of Knowledge. the 39th Annual Meeting of the Decision Sciences Institute, Baltimore, Maryland, USA.
Gnatzy, T., Warth, J., Gracht, H., & Darkow, I.-L. (2011). all Validating an Innovative Real-Time Delphi Approach: A Methodological Comparison between Real-Time and Con-ventional Delphi Studies. Technological Forecasting and Social Change, 78(9), 1681-1694. https://doi.org/10.1016/j.techfore.2011.04.006.
Grisham, T. (2009). The Delphi Technique: a Method for Testing Complex and Multifaceted Topics. International Journal of Managing Projects in Business, 2(1), 112-130. https://doi.org/10.1108/17538370910930545.
Gupta, U. G., & Clarke, R. E. (1996). Theory and Applications of the Delphi Technique: A bibliography (1975-1994). Technological Forecating and Social Change, 53, 185-211. https://doi.org/10.1016/S0040-1625(96)00094-7.
Hanafin, S. (2004). Review of Literature on the Delphi Technique. The National Children's Office.
Hanafin, S., & Brooks, A.-M. (2005). The Delphi Technique: A Methodology to Support the Development of a National Set of Child Well-being Indicators. The National Children's Office.
Hatcher, T., & Colton, S. (2007). Using the Internet to Im-prove HRD Research: The Case of the Web-Based Delphi Research Technique to Achieve Content Validity of an HRD-Oriented Measurement. Journal of European Industrial Training, 31(7), 570-587. https://doi.org/10.1108/03090590710820060.
Howze, P. C., & Dalrymple, C. (2004). Consensus Without the Meetings: Using the Delphi Method to Determine Course Content for Library Instruction. Reference Services Review, 32(2), 174-184. https://doi.org/10.1108/00907320410537694.
Hsu, C.-C., & Sandford, B. A. (2007). The Delphi Technique: Making Sense of Consensus. Practical Assessment. Research & Evaluation, 12(10), 1-8. https://doi.org/10.7275/pdz9-th90.
Hsu, P.-F., & Lin, F.-L. (2013). Developing a Decision Model for Brand Naming Using Delphi Method and Analytic Hierarchy Process. Asia Pacific Journal of Marketing and Logistics, 25(2), 187-199. https://doi.org/10.1108/13555851311314013.
Kang, T.-H., Tsai, L.-M., Jia-Horng, S., & Yuan, B. (2009). A Study of Applying the Internet Platform on Technology Foresight. the Portland International Center for Management of Engineering and Technology Conference, Portland, Oregon, USA.
Kaynak, E., Bloom, J., & Leibold, M. (1994). Using the Delphi Technique to Predict Future Tourism Potential. Marketing Intelligence & Planning, 12(7), 18-29. https://doi.org/10.1108/02634509410065537.
Lindqvist, P., & Nordänger, U. K. (2007). (Mis- ?) using the E-Delphi Method: An Attempt to Articulate the Practical Knowledge of Teaching. Journal of Research Methods and Methodological Issues, 1(1).
Linstone, H. A., & Turoff, M. (2002). The Delphi Method: Techniques and Applications. Addison-Wesley Publishing Company.
Loo, R. (2002). The Delphi Method: A Powerful Tool for Strategic Management. Policing: An International Journal of Police Strategies & Management, 25(4), 762-769. https://doi.org/10.1108/13639510210450677.
Loyens, K., Maesschalck, J., & Bouckaert, G. (2011). Delphi in Criminal Justice Policy: A Case Study on Judgmental Forecasting. The Qualitative Report, 16(6), 1477-1495. https://doi.org/10.46743/2160-3715/2011.1313.
Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi methodology? Journal of Extension, 35 (5), 1-4.
Minghat, A. D., Yasin, R. M., & Udin, A. (2012). The Appli-cation of the Delphi Technique in Technical and Vocational Education in Malaysia. the International Conference on Management and Education Innovation, Kuala Lumpur, Malaysia.
Mullen, P. M. (2003). Delphi: Myths and Reality. Journal of Health Organization and Management, 17(1), 37-52. https://doi.org/10.1108/14777260310469319.
Nelson, K. G., Brem, S. K., & Husman, J. (2012). Work in Progress: Identification of Misconceptions governed by Emergent Phenomena in Photovoltaics Content using the Delphi Method. Seattle, WA, USA.
O’Halloran, C., Stewart, J., Harrigan, P., Spencer, J. A., & Barton, J. R. (1999). Identifying Appropriate Tasks for the Preregistration Year. British Medical Journal, 319, 224-229. https://doi.org/10.1136/bmj.319.7204.224.
O’Neill, S., Scott, M., & Conboy, K. (2009). What’s Tech-nology Got to Do With it? A Delphi Study on Collaborative Learning in Distance Education. . the 17th European Conference on Information Systems, Verona, Italy.
Okoli, C., & Pawlowski, S. (2004). The Delphi Method as a Research Tool: an Example, Design Considerations and Applications. Information & Management, 42(1), 15-29. https://doi.org/10.1016/j.im.2003.11.002.
Parente, R., & Anderson-Parente, J. (2011). Case Study of Long-Term Delphi Accuracy. Technological Forecasting & Social Change, 78(9), 1705-1711. https://doi.org/10.1016/j.techfore.2011.07.005.
Paul, C. L. (2008). A Modified Delphi Approach to a New Card Sorting Methodology. Journal of Usability Studies, 4(1), 7-30.
Powell, C. (2003). The Delphi Technique: Myths and Realities. Journal of Advanced Nursing, 41(4), 376-382. https://doi.org/10.1046/j.1365-2648.2003.02537.x.
Riggs, W. E. (1983). The Delphi Technique: An Experimental Evaluation. Technological Forecasting and Social Change, 23, 89-94. https://doi.org/10.1016/0040-1625(83)90073-2.
Rikkonen, P., Kaivo-oja, J., & Aakkula, J. (2006). Delphi Expert Panels in the Scenario-Based Strategic Planning of Agriculture. foresight, 8(1), 66-81. https://doi.org/10.1108/14636680610647156.
Rogers, M. R., & Lopez, E. C. (2002). Identifying Critical Cross-Cultural School Psychology Competencies. Journal of School Psychology, 40(2), 115-141. https://doi.org/10.1016/S0022-4405(02)00093-6.
Roy, T. K., & Garai, A. (2012). Intuitionistic Fuzzy Delphi Method: More Realistic and Interactive Forecasting Tool. Notes on Intuitionistic Fuzzy Sets, 18(2), 37-50.
Scarcella, J. A., & Custer, R. L. (1999). Competencies Identi-fied as Important for 21st Century Plastering Contractors: A Rotational Delphi. Journal of Industrial Technology, 15(1), 1-7.
Shah, H. A., & Tillman, T. (2011). An International Delphi Study to Build a Foundation for an Undergraduate-Level Lean Manufacturing Curriculum. the Interna-tional Association of Journals & Conferences (IAJC) and American Society for Engineering Education (ASEE) Inter-national Joint Conference, University of Hartford, West Hartford, CT, USA.
Shah, H. A., & Tillman, T. (2012). An International Delphi Study to Build a Foundation for an Undergraduate Level Lean Manufacturing Curriculum. . International Transaction Journal of Engineering, Management & Applied Sciences & Technologies, 2(5), 521-530.
Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi Method for Graduate Research. Journal of Information Technology Education, 6, 001-021.
Snyder-Halpern, R. (2001). Indicators of Organizational Readiness for Clinical Information Technology / Systems Innovation: A Delphi Study. International Journal of Medical Informatics, 63(3), 179-204. https://doi.org/10.1016/s1386-5056(01)00179-4.
Sobaih, A. E., Ritchie, C., & Jones, E. (2012). Consulting the Oracle? Applications of Modified Delphi Technique to Qual-itative Research in the Hospitality Industry. International Journal of Contemporary Hospitality Management, 24(6), 886-906. https://doi.org/10.1108/09596111211247227.
Stitt-Gohdes, W. L., & Crews, T. B. (2004). The Delphi Technique: A Research Strategy for Career and Technical Education. Journal of Career and Technical Education, 20(2), 10.21061/jcte.v21020i21062.21636.
Stylianides, M., & Pashiardis, P. (2007). The Future of our Schools: An Example of the Delphi Technique in Action and the Case of Cyprus. International Journal of Educational Management, 21(5), 384-406. https://doi.org/10.1108/09513540710760174.
Tiwary, A., Viswanathan, J., Jhatta, M., & Ravisankar, R. (1983). Delphi Study of Futurist Management Pedagogy. Vikalpa, 8(3). https://doi.org/10.1177/02560909198303.
Turoff, M. (1970). The Design of a Policy Delphi. Technological Forecasting and Social Change, 2(2), 149-171. https://doi.org/10.1016/0040-1625(70)90161-7.
Vakani, F., & Sheerani, M. (2012). How to Gain Consensus from a Group of Non-Experts: An Educationist Perspective on Using the Delphi Technique. Development and Learning in Organizations, 26(4), 20-22. https://doi.org/10.1108/14777281211249923.
Welty, G. (1973). Some Problems of Selecting Delphi Experts for Educational Planning and Forecasting Exercises. California Journal of Educational Research, 24(3), 129-134.
Wu, C., Ma, L., Zhang, B., & Hu, H. (2009). Study on Indicators Choosing for Navigation Safety Assessment of Three Gorges Reservoir Areas Based on Delphi Method. the Asia-Pacific Conference on Information Processing, Shenzhen, China.
Xu, H.-M., Wei, J., & Yuan, M.-H. (2009). An Improved Delphi Method for Machinability Evaluation and Examination in Process Planning. the International Conference on Measuring Technology and Mechatronics Automation, Zhangjiajie, China.
Yonghong, A., Bohan, Y., Fan, Y., & Gang, Z. (2012). The Application of Modified Delphi-AHP Method in the College Students’ Comprehensive Quality Evaluation System. International Journal of Information and Education Technology, 2(4), 389-393. https://doi.org/10.7763/IJIET.2012.V2.160.
Yousuf, M. I. (2007). Using Experts’ Opinions through Delphi Technique. Practical Assessment, Research & Evaluation, 12(4), 1-8.
Zaloom, V., & Subhedar, V. (2009). Use of the Delphi Method to Prioritize Events Impacting Operations in the Maritime Domain. Lamar University.
Zhu, Q., Du, J. T., Meng, F., Wu, K., & Sun, X. (2011). Using a Delphi Method and the Analytic Hierarchy Process to Evalu-ate Chinese Search Engines: A Case Study on Chinese Search Engines. Online Information Review, 35(6), 942-956. https://doi.org/10.1108/14684521111193210.
Associate Professor Rafa E. Al-Qutaish
Associate Professor of Software Engineering, 412-1682 Chemin du Tremblay, Longueuil, QC J4N 1E1 - Canada
rafa.alqutaish@gmail.com


CREATE AUTHOR ACCOUNT
 
LAUNCH YOUR SPECIAL ISSUE
View all special issues >>
 
PUBLICATION VIDEOS