Website quality: An analysis of scientific production



Palabras clave:

Website quality, Websites, Scientific production analysis, Systematic literature review, Review article, Heuristics, Expert analysis, Usability, Contents, User experience, Standards, Guidelines


A range of different methods and tools have been proposed by both academics and professionals in recent years for evaluating the quality of websites. Some are of general application and can be used to assess any type of website, while others have been adapted to the specialized characteristics of the websites employed in a given sector. This paper undertakes an analysis of existing scientific production in this field, with the aim of identifying its most relevant publications, its principal authors and the specific sectors served by the sites under evaluation. By triangulating review methods, 716 texts, published between 2000 and 2018, were identified in Scopus, Web of Science and other databases and examined. In addition to basic bibliographic information, the number of citations received by each text was recorded using Google Scholar. The area of knowledge in which each author works was also categorized based on his or her specific affiliations. The results point to a growing interest in website quality in a scientific community that has a presence in more than 70 countries. Its authors are drawn from various disciplines, although the highest number of publications is recorded in computer science, business and medical informatics. The most frequently cited texts are in fact seminal books in the associated disciplines of usability, information architecture and user experience. However, as of 2007, the number of texts describing evaluation tools for the websites of specific sectors – most notably, education (33%), health (27%) and commerce (21%) – increased their share. In conclusion, it is evident that website quality is a field of study undergoing constant growth and increasing specialization and one that offers ample opportunities for research.


Abadal, Ernest; Guallar, Javier (2018). “Scientific production on interaction in digital news media”. In: Pérez-Montoro, Mario (ed.). Interaction in digital news media: From principles to practice. Palgrave Macmillan, pp. 175-191. ISBN: 978 3 319 96252 8

Abdallah, Salam; Jaleel, Bushra (2015). “Website appeal: development of an assessment tool and evaluation framework of e-marketing”. Journal of theoretical and applied electronic commerce research, v. 10, n. 3, pp. 45-62.

Afonso, Ana-Paula; Reis-Lima, Jorge; Pérez-Cota, Manuel (2012). “A heuristic evaluation of usability of web interfaces”. In: 7th Iberian Conference on Information Systems and Technologies (CISTI 2012), pp. 1-6.

Agarwal, Ritu; Venkatesh, Viswanath (2002). “Assessing a firm’s web presence: A heuristic evaluation procedure for the measurement of usability”. Information systems research, v. 13, n. 2, pp. 168-186.

Aguillo, Isidro F.; Ortega, José-Luís; Fernández, Mario (2008). “Webometric ranking of world universities: introduction, methodology, and future developments”. Higher education in Europe, v. 33, n. 2-3, pp. 233-244.

Aladwani, Adel M.; Palvia, Prashant C. (2002). “Developing and validating an instrument for measuring user-perceived web quality”. Information & management, v. 39, n. 6, pp. 467-476.

Allen, Mureen; Currie, Leanne M.; Bakken, Suzanne; Patel, Vimla L.; Cimino, James J. (2006). “Heuristic evaluation of paper-based web pages: a simplified inspection usability methodology”. Journal of biomedical informatics, v. 39, n. 4, pp. 412-423.

Anusha, R. (2014). “A study on website quality models”. International journal of scientific and research publications, v. 4, n. 12.

Barnes, Stuart J.; Vidgen, Richard (2001). “An evaluation of cyber-bookshops: the webqual method”. International journal of electronic commerce, v. 6, n. 1, pp. 11-30.

Beredjiklian, Pedro K.; Bozentka, David J.; Steinberg, David R.; Bernstein, Joseph (2000). “Evaluating the source and content of orthopaedic information on the internet: the case of carpal tunnel syndrome”. The journal of bone and joint surgery, v. 82, n. 11, pp. 1540-1543.

Bevan, Nigel (2005). “Guidelines and standards for web usability”. In: Proceedings of the 11th International Conference on Human-Computer Interaction. Las Vegas (HCI International), pp. 22-27.

Bevan, Nigel; Carter, James; Harker, Susan (2015). “ISO 9241-11 Revised: What have we learnt about usability since 1998?”. In: Human-computer interaction: Design and evaluation, pp. 143-151.

Bilsel, R. Ufuk; Büyüközkan, Gülçin; Ruan, Da (2006). “A fuzzy preference-ranking model for a quality evaluation of hospital web sites”. International journal of intelligent systems, v. 21, n. 11, pp. 1181-1197.

Booth, Andrew; Sutton, Anthea; Papaioannou, Diana (2016). Systematic approaches to a successful literature review. London: SAGE Publications Ltd. ISBN: 978 1 473912465Calero, Coral; Ruiz, Julián; Piattini, Mario (2005). “Classifying web metrics using the web quality model”. Online information review, v. 29, n. 3, pp. 227-248.

Cao, Mei; Zhang, Qingyu; Seydel, John (2005). “B2C e-commerce web site quality: an empirical examination”. Industrial management & data systems, v. 105, n. 5, pp. 645-661.

Chiou, Wen-Chih; Lin, Chin-Chao; Perng, Chyuan (2010). “A strategic framework for website evaluation based on a review of the literature from 1995-2006”. Information & management, v. 47, n. 5-6, pp. 282-290.

Chua, Alton Y. K.; Goh, Dion H; Ang, Rebecca P. (2012). “Web 2.0 applications in government web sites: prevalence, use and correlations with perceived web site quality”. Online information review, v. 36, n. 2, pp. 175-195.

Clifton, Brian (2012). Advanced web metrics with Google Analytics. Indianapolis, Indiana: John Wiley & Sons, Inc. ISBN: 978 1 118 16844 8

Codina, Lluís (2008). “Sistema general de análisis y evaluación de sitios web: parámetros e indicadores”. Metodología de análisis y evaluación de recursos digitales en línea.

Codina, Lluís (2018). Revisiones bibliográficas sistematizadas: procedimientos generales y framework para ciencias humanas y sociales.

Codina, Lluís; Pedraza-Jiménez, Rafael (2016). “Características y componentes de un sistema de análisis de medios digitales: el SAAMD”. En: Pedraza-Jiménez, Rafael; Codina; Lluís; Guallar, Javier (coords.). Calidad en sitios web: Método de análisis general, e-commerce, imágenes, hemerotecas y turismo. Barcelona: Editorial UOC, pp. 15-40. ISBN: 978 84 9064 487 4

Codina, Lluís; Pedraza-Jiménez, Rafael; Díaz-Noci, Javier; Rodríguez-Martínez, Ruth; Pérez-Montoro, Mario; Cavaller-Reyes, Víctor (2014). “Sistema Articulado de Análisis de Cibermedios (SAAC): Una propuesta sobre el qué y el cómo para estudiar medios de comunicación digitales”., n. 12.

Cutroni, Justin (2010). Google analytics. Sebastopol: O’Reilly Media, Inc. ISBN: 978 0 596 15800 2

Dey, Arindam; Billinghurst, Mark; Lindeman, Robert W.; Swan, J. Edward (2018). “A systematic review of 10 years of augmented reality usability studies: 2005 to 2014”. Frontiers in robotics and AI, v. 5.

Díaz, Ediber; Arenas, Juan-Jesús; Moquillaza, Arturo; Paz, Freddy (2019). “A systematic literature review about quantitative metrics to evaluate the usability of e-commerce web sites”. In: Intelligent human systems integration 2019, pp. 332-338.

Drǎgulǎnescu, Nicolae-George (2002). “Website quality evaluations: Criteria and tools”. The international information & library review, v. 34, n. 3, pp. 247-254.

Fernández, Adrian; Insfran, Emilio; Abrahão, Silvia (2011). “Usability evaluation methods for the web: A systematic mapping study”. Information and software technology, v. 53, n. 8, pp. 789-817.

Fernández-Cavia, José; Rovira, Cristòfol; Díaz-Luque, Pablo; Cavaller, Víctor (2014). “Web Quality Index (WQI) for official tourist destination websites. Proposal for an assessment system”. Tourism management perspectives, v. 9, pp. 5-13.

Fogli, Daniela; Guida, Giovanni (2015). “A practical approach to the assessment of quality in use of corporate web sites”. Journal of systems and software, v. 99, pp. 52-65.

Garrett, Jesse-James (2011). The elements of user experience: User-centered design for the web and beyond. Indianapolis: New Riders. ISBN: 978 0 321 68368 7

González-Sánchez, José-Luis; Montero-Simarro, Francisco; Gutiérrez-Vela, Francisco-Luis (2012). “Evolución del concepto de usabilidad como indicador de calidad del software”. El profesional de la información, v. 21, n. 5, pp. 529-536.

Gordon, Joye; Berhow, Susan (2009). “University websites and dialogic features for building relationships with potential students”. Public relations review, v. 35, n. 2, pp. 150-152.

Gough, David; Oliver, Sandy; Thomas, James (2017). An introduction to systematic reviews. London: SAGE Publications Ltd. ISBN: 978 1 473929432

Grant, Maria J.; Booth, Andrew (2009). “A typology of reviews: an analysis of 14 review types and associated methodologies”. Health information & libraries journal, v. 26, n. 2, pp. 91-108.

Hasan, Layla; Abuelrub, Emad (2011). “Assessing the quality of web sites”. Applied computing and informatics, v. 9, n. 1, pp. 11-29.

Hassan-Montero, Yusef; Martín-Fernández, Francisco J. (2003). “Guía de evaluación heurística de sitios web”. No solo usabilidad, n. 2, 30 marzo.

Health On the Net Foundation (2017). “Principles: The HON code of conduct for medical and health web sites”. HONcode, p. 1.

Hernández, Blanca; Jiménez, Julio; Martín, María-José (2009). “Key website factors in e-business strategy”. International journal of information management, v. 29, n. 5, pp. 362-371.

Hu, Yifeng; Shyam-Sundar, S. (2010). “Effects of online health sources on credibility and behavioral intentions”. Communication research, v. 37, n. 1, pp. 105-132.

Huang, Zhao; Benyoucef, Morad (2014). “Usability and credibility of e-government websites”. Government information quarterly, v. 31, n. 4, pp. 584-595.

Huizingh, Eelko K. R. E. (2000). “The content and design of web sites: an empirical study”. Information & management, v. 37, n. 3, pp. 123-134.

Internet Live Stats (2020). Internet usage & social media statistics.

ISO (2008). ISO 9241-151:2008. Ergonomics of human-system interaction. Part 151: Guidance on World Wide Web user interfaces.

ISO (2015). ISO 9000:2015. Quality management systems - Fundamentals and vocabulary.

Ivory, Melody Y.; Hearst, Marti A. (2001). “The state of the art in automating usability evaluation of user interfaces”. ACM computing surveys, v. 33, n. 4, pp. 470-516.

Jayanthi, B.; Krishnakumari, P. (2016). “An intelligent method to assess webpage quality using extreme learning machine”. International journal of computer science and network security, v. 16, n. 9, pp. 81-85.

Jiménez, Cristhy; Rusu, Cristian; Roncagliolo, Silvana; Inostroza, Rodolfo; Rusu, Virginica (2012). “Evaluating a methodology to establish usability heuristics”. In: 2012 31st International Conference of the Chilean Computer Science Society. IEEE, pp. 51-59.

Jiménez-Iglesias, Lucía; Pérez-Montoro, Mario; Sánchez-Gómez, Lydia (2017). “Diseño de información digital: revisión y clasificación de indicadores heurísticos para contenidos web”. El profesional de la información, v. 26, n. 6, pp. 1029-1046.

Jowkar, Abdolrasoul; Didegah, Fereshteh (2010). “Evaluating Iranian newspapers’ web sites using correspondence analysis”. Library hi tech, v. 28, n. 1, pp. 119-130.

Karkin, Naci; Janssen, Marijn (2014). “Evaluating websites from a public value perspective: A review of Turkish local government websites”. International journal of information management, v. 34, n. 3, pp. 351-363.

Kaur, Satinder; Gupta, S. K. (2014). “Key aspects to evaluate the performance of a commercial website”. In: IJCA Proceedings on International Conference on Advances in Computer Engineering and Applications. Icacea, n. 1, pp. 7-11.

Kaushik, Avinash (2010). Web analytics 2.0: The art of online accountability & science of customer centricity. Wiley Publishing, Inc. ISBN: 978 0 470 52939 3

Kitchenham, Barbara; Charters, Stuart (2007). Guidelines for performing systematic literature reviews in software engineering.

Krug, Steve (2014). Don’t make me think, revisited: A common sense approach to web and mobile usability. Berkeley, California: Pearson Education. ISBN: 978 0 321965516

Lavrakas, Paul J. (2008). Encyclopedia of survey research methods. Thousand Oaks, CA: Sage Publications, Inc. ISBN: 978 1 412918084

Law, Rob; Qi, Shanshan; Buhalis, Dimitrios (2010). “Progress in tourism management: A review of website evaluation in tourism research”. Tourism management, v. 31, n. 3, pp. 297-313.

Leavitt, Michael O.; Shneiderman, Ben (2006). Research-based web design & usability guidelines. Washington, DC. ISBN: 0 16 076270 7

Liu, Chang; Arnett, Kirk P. (2000). “Exploring the factors associated with web site success in the context of electronic commerce”. Information & management, v. 38, n. 1, pp. 23-33.

Loiacono, Eleanor T.; Watson, Richard T.; Goodhue, Dale L. (2002). “WebQual: A measure of website quality”. In: American Marketing Association: Winter Marketing Educators’ Conference.

Loiacono, Eleanor T.; Watson, Richard T.; Goodhue, Dale L. (2007). “WebQual: an instrument for consumer evaluation of web sites”. International journal of electronic commerce, v. 11, n. 3, pp. 51-87.

Lopezosa, Carlos (2019). SEO, periodismo y comunicación: conceptos, herramientas y procesos para optimizar la visibilidad web de los medios digitales. Tesis doctoral, Universitat Pompeu Fabra.

Maia, Camila-Loiola-Brito; Furtado, Elizabeth-Sucupira (2016). “A systematic review about user experience evaluation”. In: Marcus, Aaron (ed.). Design, user experience, and usability: Design thinking and methods, pp. 445-455. ISBN: 978 3 319 40408 0

Mariage, Céline; Vanderdonckt, Jean; Pribeanu, Costin (2006). “State of the art of web usability guidelines”. In: Vu, Kim-Phuong L.; Proctor, Robert W. (eds.). Handbook of human factors in web design. Boca Raton: CRC Press Inc., pp. 688-700. ISBN: 978 1 4398 2594 5

Martín-Martín, Alberto; Orduña-Malea, Enrique; Thelwall, Mike; Delgado-López-Cózar, Emilio (2018). “Google Scholar, Web of Science, and Scopus: a systematic comparison of citations in 252 subject categories”. Journal of informetrics, v. 12, n. 4, pp. 1160-1177.

Matera, Maristella; Rizzo, Francesca; Carughi, Giovanni-Toffetti (2006). “Web usability: Principles and evaluation methods”. In: Mendes, Emilia; Mosley, Nile (eds.). Web engineering. Berlin Heeidelberg: Springer-Verlag, pp. 143-180. ISBN: 978 3 540 28196 2

Moher, David; Liberati, Alessandro; Tetzlaff, Jennifer; Altman, Douglas G.; The Prisma Group (2009). “Preferred reporting items for systematic reviews and meta-analyses: The Prisma statement”. PLoS medicine, v. 6, n. 7, p. e1000097.

Morville, Peter (2005). Ambient findability. Sebastopol, California: O’Reilly Media. ISBN: 0596007655

Morville, Peter; Rosenfeld, Louis (2006). Information architecture for the World Wide Web. Sebastopol (California): O’Reilly. ISBN: 978 0 596527341

Nielsen, Jakob (2000). Designing web usability. Indianapolis, Indiana: New Riders. ISBN: 156205810X

Nielsen, Jakob (2005). “10 usability heuristics for user interface design”. Nielsen Norman Group, 24 April.

Nielsen, Jakob; Loranger, Hoa (2006). Prioritizing web usability. Berkeley, California: New Riders. ISBN: 978 0 321350312

Norman, Donald A. (2002). The design of everyday things. New York: Basic Books. ISBN: 0465067107

Olsina, Luis; Covella, Guillermo; Rossi, Gustavo (2006). “Web quality”. In: Mendes, Emilia; Mosley, Nile (eds.). Web engineering. Berlin, Heidelberg: Springer-Verlag, pp. 109-142. ISBN: 978 3 540 28196 2

Orduña-Malea, Enrique; Aguillo, Isidro F. (2014). Cibermetría: midiendo el espacio red. Barcelona: Editorial UOC. ISBN: 978 84 9064 233 7

Palmer, Jonathan W. (2002). “Web site usability, design, and performance metrics”. Information systems research, v. 13, n. 2, pp. 151-167.

Park, Young A.; Gretzel, Ulrike (2007). “Success factors for destination marketing web sites: a qualitative meta-analysis”. Journal of travel research, v. 46, n. 1, pp. 46-63.

Pedraza-Jiménez, Rafael; Codina, Lluís; Guallar, Javier (2016). Calidad en sitios web: Método de análisis general, e-commerce, imágenes, hemerotecas y turismo, Barcelona: Editorial UOC. ISBN: 978 84 9064 487 4

Pérez-Montoro, Mario; Codina, Lluís (2017). Navigation design and SEO for content-intensive websites: a guide for an efficient digital communication. Cambridge, MA: Chandos Publishing - Elsevier. ISBN: 978 0 081006771

Pribeanu, Costin (2009). “A usability assistant for the heuristic evaluation of interactive systems”. Studies in informatics and control, v. 18, n. 4, pp. 355-362.

Quiñones, Daniela; Rusu, Cristian (2017). “How to develop usability heuristics: a systematic literature review”. Computer standards & interfaces, v. 53, pp. 89-122.

Ramler, Rudolf; Weippl, Edgar; Winterer, Mario; Schwinger, Wieland; Altmann, Josef (2002). “A quality-driven approach to web testing”. In: Proceedings of ICWE’02 Conference, pp. 81-95.

Rekik, Rim; Kallel, Ilhem; Alimi, Adel M. (2015). “Quality evaluation of web sites: a comparative study of some multiple criteria decision making methods”. In: 15th International Conference on Intelligent Systems Design and Applications (ISDA), pp. 585-590. ISBN: 978 1 4673 8709 5

Rekik, Rim; Kallel, Ilhem; Casillas, Jorge; Alimi, Adel M. (2018). “Assessing web sites quality: a systematic literature review by text and association rules mining”. International journal of information management, v. 38, n. 1, pp. 201-216.

Rocha, Álvaro (2012). “Framework for a global quality evaluation of a website”. Online information review, v. 36, n. 3, pp. 374-382.

Rodríguez-Martínez, Ruth; Codina, Lluís; Pedraza-Jiménez, Rafael (2012). “Indicadores para la evaluación de la calidad en cibermedios: análisis de la interacción y de la adopción de la web 2.0”. Revista española de documentación científica, v. 35, n. 1, pp. 61-93.

Rubin, Jeffrey; Chisnell, Dana (2008). Handbook of usability testing: How to plan, design, and conduct effective tests. Indianapolis, Ind.: Wiley. ISBN: 978 0 470185483

Sanabre-Vives, Carles; Pedraza-Jiménez, Rafael; Vinyals-Mirabent, Sara (2020). “Double-entry analysis system for comprehensive quality evaluation of websites: case study in the tourism sector”. Profesional de la información, v. 29, n. 4, e290432.

Sanabre-Vives, Carles; Pedraza-Jiménez, Rafael; Codina, Lluís (2018). “Website canvas model: propuesta de un modelo visual para la ideación estratégica de sitios web”. Revista española de documentación científica, v. 41, n. 4, p. 221.

Sauro, Jeff (2010). A practical guide to measuring usability: Quantifying the usability of websites and software. CreateSpace. ISBN: 978 1 453806562

Sauro, Jeff; Lewis, James R. (2012). Quantifying the user experience: practical statistics for user research. Waltham, MA: Elsevier / Morgan Kaufmann. ISBN: 978 0 123849687

Semerádová, Tereza; Weinlich, Petr (2020). “Looking for the definition of website quality”. In: Semerádová, Tereza; Weinlich, Petr. Website quality and shopping behavior: Quantitative and qualitative evidence. Springer Nature, pp. 5-27. ISBN: 978 3 030 44439 6

Shneiderman, Ben (2000). “Universal usability”. Communications of the ACM, v. 43, n. 5, pp. 84-91.

Shneiderman, Ben (2016). “The eight golden rules of interface design”. Department of Computer Science, University of Maryland.

Tapia-León, Mariela (2015). “Factores de calidad en sitios web de destinos turísticos: estado de la cuestión”., n. 13.

Thelwall, Mike (2002). “Conceptualizing documentation on the web: an evaluation of different heuristic-based models for counting links between university web sites”. Journal of the American Society for Information Science and Technology, v. 53, n. 12, pp. 995-1005.

Thelwall, Mike; Kousha, Kayvan (2015). “Web indicators for research evaluation. Part 1: Citations and links to academic articles from the web”. El profesional de la información, v. 24, n. 5, pp. 587-606.

Tidwell, Jenifer (2006). Designing interfaces: Patterns for effective interaction design. Cambridge, MA: O’Reilly. ISBN: 0596008031

Tognazzini, Bruce (2014). “First principles of interaction design (revised and expanded)”. Ask TOG: Interaction design solutions for the real world.

Tonkin, Sebastian; Whitmore, Caleb; Cutroni, Justin (2010). Performance marketing with Google Analytics: Strategies and techniques for maximizing online ROI. ISBN: 978 0 470578315

Tullis, Thomas; Albert, William (2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics. Morgan Kaufmann. ISBN: 978 0 123735584

Ugras, Tuba; Gülseçen, Sevinç; Çubukçu, Ceren; Erdoğmuş, İpek İli; Gashi, Vala; Bedir, Merve (2016). “Research trends in web site usability: A systematic review”. In: Marcus, Aaron (ed.). Design, user experience, and usability: Design thinking and methods. Springer, pp. 517-528. ISBN: 978 3 319 40409 7

Unesco (2018). Unesco’s internet universality indicators: A framework for assessing internet development. Paris: United Nations Educational, Scientific and Cultural Organization. ISBN: 978 92 3 100314 1

Wątróbski, Jarosław; Ziemba, Paweł; Jankowski, Jarosław; Wolski, Waldemar (2016). “Pequal. E-commerce websites quality evaluation methodology”. Annals of computer science and information systems, v. 8, pp. 1317-1327.

Webster, Jane; Watson, Richard T. (2002). “Analyzing the past to prepare for the future: Writing a literature review”. MIS quarterly, v. 26, n. 2, pp. xiii-xxiii.

Yin, Robert K. (2015). Qualitative research from start to finish. London: The Guilford Press. ISBN: 978 1 462517978




Cómo citar

Morales-Vargas, A., Pedraza-Jiménez, R., & Codina, L. (2020). Website quality: An analysis of scientific production. Profesional De La Información, 29(5).



Artículos de investigación / Research articles


La descarga de datos todavía no está disponible.