A three-level aggregation model for evaluating software usability by fuzzy logic

Rakovská, Eva ; Hudec, Miroslav

International Journal of Applied Mathematics and Computer Science, Tome 29 (2019) no. 3, pp. 489-501.

Résumé

Rapid deployment of IT brings about new issues with software usability measurement. Usability is based on users’ experience and is strongly subjective, having a qualitative character. The users’ comfort is usually collected by surveys in their daily work. The present article stems from an experimental study related to the evaluation of the usability of tools by a rule-based system. The work suggests a robust computational model that will be able to avoid the main problems arising from the experimental study (a large and less-legible rule base) and to deal with the vagueness of IT user experience, different levels of skills and various numbers of filled questionnaires in different departments. The computational model is based on three hierarchical levels of aggregation supported by fuzzy logic. Choices for the most suitable aggregation functions in each level are advocated and illustrated with examples. The number of questions and granularity of answers in this approach can be adjusted to each user group, which could reduce the response burden and errors. Finally, the paper briefly describes further possibilities of the suggested approach.

Détail

Keywords: measuring software usability, rule-based system, fuzzy quantifiers, aggregation function, questionnaire
Mots-clés : system regułowy, kwantyfikator rozmyty, funkcja agregacji

Bibliographie

[1] Albert, W. and Tullis, T. (2013). Measuring the User Experience, Collecting, Analyzing, and Presenting Usability Metrics (Interactive Technologies), 2nd Edition, Elsevier, Amsterdam.

[2] Allen, I.E. and Seaman, C. (2007). Likert scales and data analyses, Technical report, QP-Quality Progress, http://asq.org/quality-progress/2007/07/statistics/likert-scales-and-data-analyses.html.

[3] Bavdaž, M. (2010). Sources of measurement errors in business surveys, Journal of Official Statistics 26(1): 25–42.

[4] Bavdaž, M., Biffignandi, S., Bolko, I., Giesen, D., Gravem, D. and Haraldsen, G. (2011). Final report integrating findings on business perspectives related to NSIS’ statistics, Technical report, Deliverable 3.2., FP7 Blue-Ets Project, European Commission, Brussels, https://cordis.europa.eu/project/rcn/94081/results/en?rcn=143042.

[5] Beliakov, G., Pradera, A. and Calvo, T. (2007). Aggregation Functions: A Guide for Practitioners, Springer-Verlag, Berlin/Heidelberg.

[6] Calinescu, M. and Schouten, B. (2012). Adaptive survey designs that minimize nonresponse and measurement risk, Technical report, Statistics Netherlands, The Hague/Heerlen.

[7] Calvo, T., Kolesárová, A., Komorníková, M. and Mesiar, R. (2002). Aggregation operators: Properties, classes and construction methods, in T. Calvo et al. (Eds), Aggregation Operators: New Trends and Applications, Physica, Heidelberg, pp. 3–104.

[8] Dubois, D. and Prade, H. (2004). On the use of aggregation operations in information fusion processes, Fuzzy Sets and Systems 142(1): 143–161.

[9] Dujmović, J. (2007). Continuous preference logic for system evaluation, IEEE Transactions on Fuzzy Systems 15(6): 1082–1099.

[10] Dujmović, J. (2018). Soft Computing Evaluation Logic: The LSP Decision Method and Its Applications, Wiley/IEEE Computer Society, Hoboken, NJ.

[11] Grabisch, M., Marichal, J.-L., Mesiar, R. and Pap, E. (2009). Aggregation Functions, Encyclopedia of Mathematics and its Applications, Cambridge University Press, Cambridge.

[12] Greiner, L. and White, S. (2019). What is ITIL? Your guide to the it infrastructure library, in digital magazine CIO from IDG, https://www.cio.com/article/2439501/infrastructure-it-infrastructure-library-itil-definition-and-solutions.html.

[13] Gupta, M. and Qi, J. (1991). Theory of t-norms and fuzzy inference methods, Fuzzy Sets and Systems 40(3): 431–450.

[14] Hensher, A., Rose, J. and Greene, W. (2015). Applied Choice Analysis, Cambridge University Press, Cambridge.

[15] Herrera, F. and Martínez, L. (2001). A model based on linguistic 2-tuples for dealing with multigranular hierarchical linguistic contexts in multiexpert decision-making, IEEE Transactions on Systems, Man, and Cybernetics B: Cybernetics 31(2): 227–234.

[16] ISACA (2018). Service it governance professionals, COBIT5, an ISACA framework, http://www.isaca.org/cobit/pages/default.aspx.

[17] ISO (2011). Systems and software engineering, ISO/IEC 25010:2011: Systems and software quality requirements and evaluation (square). System and software quality models, https://www.iso.org/standard/35733.html.

[18] ISO (2018). ISO, online browsing platform, ISO 9241-11:2018: Ergonomics of human–system interaction. Part 11: Usability: Definitions and concepts, https://www.iso.org/standard/63500.html.

[19] Kacprzyk, J. and Yager, R. (2001). Linguistic summaries of data using fuzzy logic, International Journal of General Systems 30(2): 133–154.

[20] Kacprzyk, J., Yager, R.R. and Zadrożny S. (2000). A fuzzy logic based approach to linguistic summaries of databases, International Journal of Applied Mathematics and Computer Science 10(4): 813–834.

[21] Králiková, L. (2017). Testovanie efektívnosti softvéru v podnikovej praxi z hladiska užívatelov (Software Effectiveness Testing in Business Practice from a User Perspective), Master thesis, University of Economics in Bratislava, Bratislava.

[22] Likert, R. (1932). A technique for the measurement of attitudes, Archives of Psychology 22(140): 1–55.

[23] Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information, Psychological Review 63(2): 81–97.

[24] Morente-Molinera, J., Kou, G., Pérez, I., Samuylov, K., Selamat, A. and Herrera-Viedma, E. (2018). A group decision making support system for the web: How to work in environments with a high number of participants and alternatives, Applied Soft Computing 68: 191–201.

[25] Olson, D. (1996). Decision Aids for Selection Problems, Springer-Verlag, London.

[26] Pavlík, L. (2018). Metrics for evaluating information systems, Posterus, Portl pre odborn publikovanie, http://www.posterus.sk/?p=18957.

[27] Piegat, A. and Pluciński, M. (2015). Computing with words with the use of inverse RDM models of membership functions, International Journal of Applied Mathematics and Computer Science 25(3): 675–688, DOI:10.1515/amcs-2015-0049.

[28] Rakovská, E. and Hudec, M. (2020). Two approaches for the computational model for soft-ware usability in practice, in J. Kacprzyk et al. (Eds), Information Technology, System Research and Computational Physics, ITSRCP 2018, Advances in Intelligent Systems and Computing, Vol. 945, Springer, Cham, pp. 191–202.

[29] Ruspini, E. (1969). A new approach to clustering, Information and Control 15(1): 22–32.

[30] Seffah, A., Kececi, N. and Donyaee, M. (2001). QUIM: A framework for quantifying usability metrics in software quality models, 2nd Asia-Pacific Conference on Quality Software, Hong Kong, pp. 311–318.

[31] Snijkers, G., Haraldsen, G., Jones, J. and Willimack, D. (2013). Designing and Conducting Business Surveys, Wiley, Hoboken, NJ.

[32] Tudorie, C. (2008). Qualifying objects in classical relational database querying, in J. Galindo (Ed.), Handbook of Research on Fuzzy Information Processing in Databases, Information Science Reference, Hershey, pp. 218–245.

[33] Yager, R. (1982). A new approach to the summarization of data, Information Sciences 28(1): 69–86.

[34] Yager, R. and Rybalov, A. (1996). Uninorm aggregation operators, Fuzzy Sets and Systems 80(1): 111–120.

[35] Zadeh, L. (1965). Fuzzy sets, Information and Control 8(3): 338–353.

[36] Zimmermann, H. (2001). Fuzzy Set Theory and Its Applications, Kluwer Academic Publishers, Dordrecht.