Toward the consolidation of a multi-metric-based journal ranking and categorization system for computer science subject areas



Palabras clave:

Journal rankings, Research evaluation, Indicators, Scientific journals, Metrics, Algorithms, Machine learning, Cluster analysis, Principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), Cross tabulation, ESA index


The evaluation of scientific journals poses challenges owing to the existence of various impact measures. This is because journal ranking is a multidimensional construct that may not be assessed effectively using a single metric such as an impact factor. A few studies have proposed an ensemble of metrics to prevent the bias induced by an individual metric. In this study, a multi-metric journal ranking method based on the standardized average index (SA index) was adopted to develop an extended standardized average index (ESA index). The ESA index utilizes six metrics: the CiteScore, Source Normalized Impact per Paper (SNIP), SCImago Journal Rank (SJR), Hirsh index (H-index), Eigenfactor Score, and Journal Impact Factor from three well-known databases (Scopus, SCImago Journal & Country Rank, and Web of Science). Experiments were conducted in two computer science subject areas: (1) artificial intelligence and (2) computer vision and pattern recognition. Comparing the results of the multi-metric-based journal ranking system with the SA index, it was demonstrated that the multi-metric ESA index exhibited high correlation with all other indicators and significantly outperformed the SA index. To further evaluate the performance of the model and determine the aggregate impact of bibliometric indices with the ESA index, we employed unsupervised machine learning techniques such as clustering coupled with principal component analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE). These techniques were utilized to measure the clustering impact of various bibliometric indicators on both the complete set of bibliometric features and the reduced set of features. Furthermore, the results of the ESA index were compared with those of other ranking systems, including the internationally recognized Scopus, SJR, and HEC Journal Recognition System (HJRS) used in Pakistan. These comparisons demonstrated that the multi-metric-based ESA index can serve as a valuable reference for publishers, journal editors, researchers, policymakers, librarians, and practitioners in journal selection, decision making, and professional assessment.


Los datos de descargas todavía no están disponibles.


Abbas, Naveed-Neem; Ahmed, Tanveer; Shah, Syed-Habib-Ullah; Omar, Muhammad; Park, Han Woo (2019). “Investigating the applications of artificial intelligence in cyber security”. Scientometrics, v. 121, n. 2, pp. 1189-1211.

Allen, Liz; Jones, Ceri; Dolby, Kevin; Lynn, David; Walport, Mark (2009). “Looking for landmarks: The role of expert review and bibliometric analysis in evaluating scientific publication outputs”. PLoS one, v. 4, n. 6.

Beliakov, Gleb; James, Simon (2011). “Citation-based journal ranks: The use of fuzzy measures”. Fuzzy sets and systems, v. 167, n. 1, pp. 101-119.

Bergstrom, Carl (2007). “Eigenfactor: Measuring the value and prestige of scholarly journals”. College & research libraries news, v. 68, n. 5, pp. 314-316.

Bollen, Johan; Van-de-Sompel, Herbert; Hagberg, Aric; Chute, Ryan (2009). “A principal component analysis of 39 scientific impact measures”. PLoS one, v. 4, n. 6.

Dellavalle, Robert P.; Schilling, Lisa M.; Rodriguez, Marko A.; Van-de-Sompel, Herbert; Bollen, Johan (2007). “Refining dermatology journal impact factors using PageRank”. Journal of the American Academy of Dermatology, v. 57, n. 1, pp. 116-119.

Duan, Lian; Nasri, Farrokh; Paknejad, Javad (2018). “A data-driven method for rating management information systems journals in the same scale of the Association of Business Schools Journal Guide”. Expert systems, v. 35, n. 6.

Ennas, Gianfranco; Biggio, Battista; Di-Guardo, Maria-Chiara (2015). “Data-driven journal meta-ranking in business and management”. Scientometrics, v. 105, n. 3, pp. 1911-1929.

Feng, Lin; Zhou, Jian; Liu, Sheng-Lan; Cai, Ning; Yang, Jie (2020). “Analysis of journal evaluation indicators : an experimental study based on unsupervised Laplacian score”. Scientometrics, v. 124, pp. 233-254.

Fernández-Cano, Antonio; Fernández-Guerrero, Inés M. (2017). “A multivariate model for evaluating emergency medicine journals”. Scientometrics, v. 110, n. 2, pp. 991-1003.

Ferrer-Sapena, Antonia; Sánchez-Pérez, Enrique A.; Peset, Fernanda; González, Luis-Millán; Aleixandre-Benavent, Rafael (2016). “The impact factor as a measuring tool of the prestige of the journals in research assessment in mathematics”. Research evaluation, v. 25, n. 3, pp. 306-314.

Fersht, Alan (2009). “The most influential journals: Impact factor and eigenfactor”. Proceedings of the National Academy of Sciences of the United States of America, v. 106, n. 17, pp. 6883-6884.

González-Pereira, Borja; Guerrero-Bote, Vicente P.; De-Moya-Anegón, Félix (2010). “A new approach to the metric of journals scientific prestige: The SJR indicator”. Journal of informetrics, v. 4, n. 3, pp. 379-391.

Halim, Zahid; Khan, Shafaq (2019). “A data science based framework to categorize academic journals”. Scientometrics, v. 119, n. 1.

Holmberg, Kim; Park, Han Woo (2018). “An altmetric investigation of the online visibility of South Korea-based scientific journals”. Scientometrics, v. 117, n. 1, pp. 603-613.

Hsu, Wen-Chin; Tsai, Chih-Fong; Li, Jia-Huan (2015). “A hybrid indicator for journal ranking: An example from the field of Health Care Sciences and Services”. Online information review, v. 39, n. 7, pp. 858-869.

James, Chris; Colledge, Lisa; Meester, Wim; Azoulay, Norman; Plume, Andrew (2018). “CiteScore metrics: Creating journal metrics from the Scopus citation index”. ArXiv, 16 pp.

Lacasse, Jeffrey R.; Hodge, David R.; Bean, Kristen F. (2011). “Evaluating the productivity of social work scholars using the h-index”. Research on social work practice, v. 21, n. 5, pp. 599-607.

Leydesdorff, Loet; Park, Han Woo (2017). Full and fractional counting in bibliometric networks. Journal of informetrics, 11, n. 1, pp. 117-120.

Mañana-Rodríguez, Jorge (2015). “A critical review of SCImago Journal & Country Rank”. Research evaluation, v. 24, n. 4, pp. 343-354.

Moed, Henk F. (2010). “Measuring contextual citation impact of scientific journals”. Journal of informetrics, v. 4, n. 3, pp. 265-277.

Morris, Huw; Harvey, Charles; Kelly, Aidan (2009). “Journal rankings and the ABS journal quality guide”. Management decision, v. 47, n. 9, pp. 1441-1451.

Mubarak, Muhammed; Seemee, Shabana (2021). “Journal prestige index: Expanding the horizons of assessment of research impact”. Journal of the College of Physicians and Surgeons Pakistan, v. 31, n. 11, pp. 1261-1262.

Perera, Upeksha; Wijewickrema, Manjula (2018). “Relationship between journal-ranking metrics for a multidisciplinary set of journals”. Portal: Libraries and the academy, v. 18, n. 1, pp. 35-58.

Saarela, Mirka; Kärkkäinen, Tommi (2020). “Can we automate expert-based journal rankings? Analysis of the Finnish publication indicator”. Journal of informetrics, v. 14, n. 2.

Saarela, Mirka; Kärkkäinen, Tommi; Lahtonen, Tommi; Rossi, Tuomo (2016). “Expert-based versus citation-based ranking of scholarly and scientific publication channels”. Journal of informetrics, 10, n. 3, pp. 693-718.

Setti, Gianluca (2013). “Bibliometric indicators: Why do we need more than one?”. IEEE Access, v. 1, pp. 232-246.

Tsai, Chih-Fong (2014). “Citation impact analysis of top ranked computer science journals and their rankings”. Journal of informetrics, v. 8, n. 2, pp. 318-328.

Tüselmann, Heinz; Sinkovics, Rudolf R.; Pishchulov, Grigory (2015). “Towards a consolidation of worldwide journal rankings - A classification using random forests and aggregate rating via data envelopment analysis”. Omega (United Kingdom), v. 51, pp. 11-23.

Walters, William H. (2017). “Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact?”. Journal of informetrics, v. 11, n. 3, pp. 730-744.

Wical, Stephanie H.; Kocken, Gregory J. (2017). “Open access and promotion and tenure evaluation plans at the University of Wisconsin-Eau Claire”. Serials review, v. 43, n. 2, pp. 111-119.

Wyatt, Sally; Milojevič, Stasa; Park, Han Woo; Leydesdorff, Loet (2017). Quantitative and qualitative STS: The intellectual and practical contributions of Scientometrics. Boston, MA: MIT Press.

Yuen, Jason (2018). “Comparison of Impact Factor, Eigenfactor Metrics, and SCImago Journal Rank Indicator and h-index for neurosurgical and spinal surgical journals”. World neurosurgery, v. 119, pp. e328-e337.

Zhu, Yu-Peng; Park, Han Woo (2022). “Profiling the most highly cited scholars from China: Who they are. To what extent they are interdisciplinary”. Profesional de la informacion, v. 31, n. 4, e310408.




Cómo citar

Hameed, A., Omar, M., Bilal, M., & Park, H. W. (2023). Toward the consolidation of a multi-metric-based journal ranking and categorization system for computer science subject areas. Profesional De La información Information Professional, 32(7).



Artí­culos de investigación / Research articles