THE INFLUENCE OF THE FORM OF THE 9-POINT SCALE IN THE AHP METHOD ON THE CONSISTENCY OF JUDGMENTS

Quality of decisions depends largely on the ability to correctly define and assess the problem. In the case of complex issues, it is recommended to use decision support methods, e.g. multicriteria methods. The objective of this paper is to report the studies related to the influence of the graphic form of the 9-point, fundamental Saaty’s comparison scale used in the AHP method on the consistency of judgments, that is, to measure the fraction of judgments with CR>0,10. Since the AHP is one of the most frequently used decision support methods in management, in terms of modeling decision problems, there is a need to explore one of its most frequently discussed problems – inconsistency of results. It will improve the quality of decisions made with the use of this tool. The empirical study was conducted among 540 respondents, using the AHP method. Due to errors in filling in the questionnaire, only 424 questionnaires were included in further analysis. Individual model was prepared and analyzed for each respondent. All results were then entered into a spreadsheet and subjected to statistical analysis. It examines four most commonly used graphic forms of scale (specifically: numerical, two-stage tabular verbal, tabular verbal horizontal and vertical). Chi-square test and F-test showed no significant difference between them in relation to the existence of inconsistent results (CR > 0.1). It allows suggest that the form of graphic scale does not affect the consistency of answers. However, additional analysis showed that it affects errors in questionnaires.

Full text (pdf)

1. Alonso J.A., Lamata M.T., Consistency in the Analytic Hierarchy Process – A New Approach, “International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems” 2006, Vol. 14, No. 4, pp. 445–459.
2. Apostolou B., Hassell J.M., An empirical examination of the sensitivity of the analytic hierarchy process to departures from recommended consistency ratios, “Mathematical and Computer Modelling” 1993, Vol. 17, No. 4–5, pp. 163–170.
3. Chu P., Liu J.K., Note on consistency ratio, “Mathematical and Computer Modelling” 2002, Vol. 35, No. 9–10, pp. 1077–1080.
4. Finan J.S., Hurley W.J., The analytic hierarchy process: Does adjusting a pairwise comparison matrix to improve the consistency ratio help?, “Computers & Operations Research” 1997, Vol. 24, No. 8, pp. 749-755.
5. Gastes D., Gaul W., The Consistency Adjustment Problem of AHP Pairwise Comparison Matrices [in:] A. Diamantopoulos, W. Fritz, L. Hildebrandt, eds., Quantitative Marketing and Marketing Management, Gabler Verlag, Wiesbaden 2012, pp. 51–62.
6. Ishizaka H., Labib A., Analytic Hierarchy Process and Expert Choice: Benefits and Limitations, “ORInsight” 2009, Vol. 22, No. 4, pp. 201–220.
7. Miller G.A., The Magical Number Seven, Plus or Minus Two: Some Limits on our Capacity for Processing Information, “Psychological Review” 1956, Vol. 63, No. 2, pp. 81–97.
8. Preston C.C., Colman A.M., Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences, “Acta Psychologica” 2000, Vol. 104, No. 1, pp. 1–15.
9. Prusak A., Stefanów P., AHP – analityczny proces hierarchiczny. Budowa i analiza modeli decyzyjnych krok po kroku, C.H. Beck, Warszawa 2014.
10. Prusak A., Stefanów P., Badania nad właściwościami metody AHP, Folia Oeconomica Cracoviensia, 2011, nr LII, s. 80–104.
11. Saaty T.L., Vargas L.G., The Logic of Priorities, Kluwer Nijhoff Publishing, Massachusetts 1982.
12. Saaty T.L., Vargas L.G., A ratio scale metric and the compatibility of ratio scales: The possibility of arrow’s impossibility theorem, “Applied Mathematics Letters” 1994, Vol. 7, No. 6, pp. 45-49.
13. Saaty T.L., Decision Making with Dependence and Feedback. The Analytic Network Process, RWS Publications, Pittsburgh 2001.
14. Saaty T.L., Ozdemir M.S., Why the magic number seven plus or minus two, “Mathematical and Computer Modelling” 2003, Vol. 38, No. 3–4, pp. 233–244.
15. Saaty T.L., Peniwati K., Group Decision Making: Drawing out and Reconciling Differences, RWS Publications, Pittsburgh 2007.
16. Saaty T.L., Relative measurement and its generalization in decision making. Why Pairwise comparisons are central in mathematics for the measurement of intangible factors. The Analytic Hierarchy/Network Process, „Revista de la Real Academia de Ciencias Exactes, Fisicas y Naturales. Serie A. Matematicas” 2008, Vol. 102, No. 2, pp. 251–318.
17. Steczkowski J., Woźniak M., Zając K., Zeliaś A., Statystyka matematyczna w zastosowaniach, Akademia Ekonomiczna w Krakowie, Kraków 1996.
18. Stefanów P., Prusak A., Badanie wiarygodności i skuteczności skali porównań Saaty’ego w metodzie AHP i ANP [w:] Przedsiębiorcze aspekty rozwoju organizacji i biznesu, red. A. Chodyński, Oficyna Wydawnicza AFM, Kraków 2011, s. 271–298.
19. Targalski J., Podejmowanie decyzji [w:] Organizacja i zarządzanie, red. A. Stabryła, J. Trzcieniecki, Warszawa 1986.
20. Vaidya O.S., Kumar S., Analytic hierarchy process: An overview of applications, “European Journal of Operational Research” 2006, Vol. 169, No. 1, pp. 1–29.
21. Weathers D., Sharma S., Niedrich R.W, The Impact of the Number of Scale Points, Dispositional Factors, and the Status Quo Decision Heuristic on Scale Reliability and Response Accuracy, “Journal of Business Research” 2005, Vol. 58, pp. 1516–1524.
22. Webber S.A., Apostolou B., Hassel J.M., The sensitivity of the analytic hierarchy process to alternative scale and cue presentations, “European Journal of Operational Research” 1996, Vol. 96, No. 2, pp. 351–362.
23. Weijters B., Cabooter E., Schillewaert N., The effect of rating scale format on response styles: The number of response categories and response category labels, “International Journal of Research in Marketing” 2010, Vol. 27, pp. 236–247.