Altmetrics can capture research evidence: an analysis across types of studies in COVID-19 literature
DOI:
https://doi.org/10.3145/epi.2023.mar.13Keywords:
COVID-19, Altmetrics, Social media metrics, Social media, Social networks, Twitter, News, Mendeley, Citations, Bibliometrics, Scientific population, Study type, PubMed, Altmetric.comAbstract
COVID-19 has greatly impacted science. It has become a global research front that constitutes a unique phenomenon of interest for the scientometric community. Accordingly, there has been a proliferation of descriptive studies on COVID-19 papers using altmetrics. Social media metrics serve to elucidate how research is shared and discussed, and one of the key points is to determine which factors are well-conditioned altmetric values. The main objective of this study is to analyze whether the altmetric mentions of COVID-19 medical studies are associated with the type of study and its level of evidence. Data were collected from the PubMed and Altmetric.com databases. A total of 16,672 publications by study types (e.g., case reports, clinical trials, or meta-analyses) that were published in the year 2021 and that had at least one altmetric mention were retrieved. The altmetric indicators considered were Altmetric Attention Score (AAS), news mentions, Twitter mentions, and Mendeley readers. Once the dataset of COVID-19 had been created, the first step was to carry out a descriptive study. Then, a normality hypothesis was evaluated by means of the Kolmogorov-Smirnov test, and since this was significant in all cases, the overall comparison of groups was performed using the nonparametric Kruskal-Wallis test. When this test rejected the null hypothesis, pairwise comparisons were performed with the Mann-Whitney U test, and the intensity of the possible association was measured using Cramer´s V coefficient. The results suggest that the data do not fit a normal distribution. The Mann-Whitney U test revealed coincidences in five groups of study types: The altmetric indicator with most coincidences was news mentions, and the study types with the most coincidences were the systematic reviews together with the meta-analyses, which coincided with four altmetric indicators. Likewise, between the study types and the altmetric indicators, a weak but significant association was observed through the chi-square and Cramer´s V. It can thus be concluded that the positive association between altmetrics and study types in medicine could reflect the level of the "pyramid" of scientific evidence.
Downloads
References
Arieta-Miranda, Jessica M.; Ruiz-Yasuda, Catherine C.; Pérez-Vargas, Luis-Fernando; Torres-Ricse, Dayhanne A.; Díaz, Solange-Pérez; Arieta, Yosseline-Chávez; Victorio, Daniel-José-Blanco; Ramos, Gilmer-Torres (2022). "New pyramid proposal for the levels of scientific evidence according to SIGN". Plastic and reconstructive surgery, v. 149, n. 4, pp. 841e-843e. https://doi.org/10.1097/PRS.0000000000008946
Aristovnik, Aleksander; Ravšelj, Dejan; Umek, Lan (2020). "A bibliometric analysis of COVID-19 across science and social science research landscape". Sustainability, v. 12, n. 21. https://doi.org/10.3390/su12219132
Arroyo-Machado, Wenceslao; Torres-Salinas, Daniel; Robinson-García, Nicolás (2021). "Identifying and characterizing social media communities: A socio-semantic network approach to altmetrics". Scientometrics, v. 126, n. 11, pp. 9267-9289. https://doi.org/10.1007/s11192-021-04167-8
Arsenault, Benoit J. (2022). "From the garden to the clinic: How Mendelian randomization is shaping up atherosclerotic cardiovascular disease prevention strategies". European heart journal, v. 43, n. 42, pp. 4447-4449. https://doi.org/10.1093/eurheartj/ehac394
Bhandari, Mohit; Montori, Victor M.; Devereaux, Philip J.; Wilczynski, Nancy L.; Morgan, Douglas; Haynes, R. Brian (2004). "Doubling the impact: Publication of systematic review articles in orthopaedic journals". JBJS, v. 86, n. 5. https://journals.lww.com/jbjsjournal/Fulltext/2004/05000/Doubling_the_Impact__Publication_of_Systematic.19.aspx
Brainard, Jeffrey (2021). "No revolution: COVID-19 boosted open access, but preprints are only a fraction of pandemic papers". Science, 8 Sept. https://doi.org/10.1126/science.acx9058
Chriscaden, Kimberly (2020). Impact of COVID-19 on people´s livelihoods, their health and our food systems. World Health Organization. https://www.who.int/news/item/13-10-2020-impact-of-COVID-19-on-people´s-livelihoods-their-health-and-our-food-systems
Colavizza, Giovanni (2020). "COVID-19 research in Wikipedia". Quantitative science studies, v. 1, n. 4, pp. 1349-1380. https://doi.org/10.1162/qss_a_00080
Colavizza, Giovanni; Costas, Rodrigo; Traag, Vincent A.; Van-Eck, Nees-Jan; Van-Leeuwen, Thed; Waltman, Ludo (2021). "A scientometric overview of CORD-19". PLoS one, v. 16, n. 1, e0244839. https://doi.org/10.1371/journal.pone.0244839
Fassin, Yves (2021). "Research on COVID-19: A disruptive phenomenon for bibliometrics". Scientometrics, v. 126, n. 6, pp. 5305-5319. https://doi.org/10.1007/s11192-021-03989-w
Fraumann, Grisha; Colavizza, Giovanni (2022). "The role of blogs and news sites in science communication during the COVID-19 pandemic". Frontiers in research metrics and analytics, v. 7. https://www.frontiersin.org/articles/10.3389/frma.2022.824538
Harris, Joshua D.; Quatman, Carmen E.; Manring, Maurice M.; Siston, Robert A.; Flanigan, David C. (2014). "How to write a systematic review". The American journal of sports medicine, v. 42, n. 11, pp. 2761-2768. https://doi.org/10.1177/0363546513497567
Haunschild, Robin; Bornmann, Lutz (2021). "Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers". Scientometrics, v. 126, n. 6, pp. 5181-5199. https://doi.org/10.1007/s11192-021-03962-7
Haustein, Stefanie; Costas, Rodrigo; Larivière, Vincent (2015). "Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns". PLoS one, v. 10, n. 3, e0120495. https://doi.org/10.1371/journal.pone.0120495
Hayawi, K.; Shahriar, S.; Serhani, M. A.; Taleb, I.; Mathew, S. S. (2022). "ANTi-Vax: A novel Twitter dataset for COVID-19 vaccine misinformation detection". Public health, v. 203, pp. 23-30. https://doi.org/10.1016/j.puhe.2021.11.022
Jung, Richard G.; Di-Santo, Pietro; Clifford, Cole; Prosperi-Porta, Graeme; Skanes, Stephanie; Hung, Annie; Parlow, Simon; Visintini, Sarah; Ramírez, F. Daniel; Simard, Trevor; Hibbert, Benjamin (2021). "Methodological quality of COVID-19 clinical research". Nature communications, v. 12, n. 1, pp. 943. https://doi.org/10.1038/s41467-021-21220-5
Kousha, Kayvan; Thelwall, Mike (2020). "COVID-19 publications: Database coverage, citations, readers, tweets, news, Facebook walls, Reddit posts". Quantitative science studies, v. 1, n. 3, pp. 1068-1091. https://doi.org/10.1162/qss_a_00066
Kowalczyk, Nina; Truluck, Christina (2013). "Literature reviews and systematic reviews: What is the difference?". Radiologic technology, v. 85, n. 2, pp. 219-222. http://www.radiologictechnology.org/content/85/2/219.extract
Lazcano-Ponce, Eduardo; Salazar-Martínez, Eduardo; Gutiérrez-Castrellón, Pedro; Ángeles-Llerenas, Angélica; Hernández-Garduño, Adolfo; Viramontes, José-Luis (2004). "Ensayos clínicos aleatorizados: Variantes, métodos de aleatorización, análisis, consideraciones éticas y regulación". Salud pública de México, v. 46, n. 6, pp. 559-584. https://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S0036-36342004000600012
Majumder, Maimuna S.; Mandl, Kenneth D. (2020). "Early in the epidemic: Impact of preprints on global discourse about COVID-19 transmissibility". The lancet global health, v. 8, n. 5, pp. e627-e630. https://doi.org/10.1016/S2214-109X(20)30113-3
Marcec, Robert; Likic, Robert (2022). "Using Twitter for sentiment analysis towards AstraZeneca/Oxford, Pfizer/BioNTech and Moderna COVID-19 vaccines". Postgraduate medical journal, v. 98, n. 1161, pp. 544-550. https://doi.org/10.1136/postgradmedj-2021-140685
Montori, Victor M.; Wilczynski, Nancy L.; Morgan, Douglas; Haynes, R. Brian; The Hedges Team (2003). "Systematic reviews: A cross-sectional study of location and citation counts". BMC medicine, v. 1, n. 1, article 2. https://doi.org/10.1186/1741-7015-1-2
Murad, M. Hassan; Asi, Noor; Alsawas, Mouaz; Alahdab, Fares (2016). "New evidence pyramid". Evidence based medicine, v. 21, pp. 125-127. https://doi.org/10.1136/ebmed-2016-110401
Nane, Gabriela F.; Robinson-García, Nicolás; Van-Schalkwyk, Franí§ois; Torres-Salinas, Daniel (2022). "COVID-19 and the scientific publishing system: Growth, open access and scientific fields". Scientometrics, v. 128, pp. 345-362. https://doi.org/10.1007/s11192-022-04536-x
Odone, Anna; Salvati, Stefano; Bellini, Lorenzo; Bucci, Daria; Capraro, Michele; Gaetti, Giovanni; Amerio, Andrea; Signorelli, Carlo (2020). "The runaway science: A bibliometric analysis of the COVID-19 scientific literature". Acta bio-medica: Atenei Parmensis, v. 91, n. 9-S, pp. 34-39. https://doi.org/10.23750/abm.v91i9-S.10121
Okike, Kanu; Kocher, Mininder S.; Torpey, Jennifer L.; Nwachukwu, Benedict U.; Mehlman, Charles T.; Bhandari, Mohit (2011). "Level of evidence and conflict of interest disclosure associated with higher citation rates in orthopedics". Journal of clinical epidemiology, v. 64, n. 3, pp. 331-338. https://doi.org/10.1016/j.jclinepi.2010.03.019
Patsopoulos, Nikolaos A.; Analatos, Apostolos A.; Ioannidis, John P. (2005). "Relative citation impact of various study designs in the health sciences". JAMA, v. 293, n. 19, pp. 2362. https://doi.org/10.1001/jama.293.19.2362
Pinho-Gomes, Ana-Catarina; Peters, Sanne; Thompson, Kelly; Hockham, Carinna; Ripullone, Katherine; Woodward, Mark; Carcel, Cheryl (2020). "Where are the women? Gender inequalities in COVID-19 research authorship". BMJ global health, v. 5, n. 7, e002922. https://doi.org/10.1136/bmjgh-2020-002922
Priem, Jason (2014). "Beyond bibliometrics: Harnessing multidimensional indicators of performance". In: Cronin, Blaise; Sugimoto, Cassidy R. (eds.). Altmetrics in the wild: Using social media to explore scholarly impact, pp. 263-287). MIT Press. https://arxiv.org/html/1203.4745
Rí¶hrig, Bernd; Du-Prel, Jean-Baptist; Wachtlin, Daniel; Blettner, Maria (2009). "Types of study in medical research: Part 3 of a series on evaluation of scientific publications". Deutsches Arzteblatt International, v. 106, n. 15, pp. 262-268. https://doi.org/10.3238/arztebl.2009.0262
Torres-Salinas, Daniel (2020). "Ritmo de crecimiento diario de la producción científica sobre COVID-19. Análisis en bases de datos y repositorios en acceso abierto". Profesional de la información, v. 29, n. 2, e290215. https://doi.org/10.3145/epi.2020.mar.15
Torres-Salinas, Daniel; Robinson-García, Nicolás; Castillo-Valdivieso, Pedro A. (2020). Open access and altmetrics in the pandemic age: Forecast analysis on COVID-19 literature. bioRxiv. https://doi.org/10.1101/2020.04.23.057307
Valderrama, Pilar; Baca, Pilar; Solana, Carmen; Ferrer-Luque, Carmen-María (2021). "Root canal disinfection articles with the highest relative citation ratios. A Bibliometric analysis from 1990 to 2019". Antibiotics, v. 10, n. 11, 1412. https://doi.org/10.3390/antibiotics10111412
Valderrama, Pilar; Torres-Salinas, Daniel (2022). "Does the type of study on COVID-19 influence the value of altmetrics?". In: Robinson-García, Nicolás; Torres-Salinas, Daniel; Arroyo-Machado, Wenceslao (eds.). STI 2022 Conference Proceedings. Zenodo. https://doi.org/10.5281/zenodo.6957471
Van-Schalkwyk, Franí§ois; Dudek, Jonathan (2022). "Reporting preprints in the media during the COVID-19 pandemic". Public understanding of science, v. 31, n. 5, pp. 608-616. https://doi.org/10.1177/09636625221077392
Van-Schalkwyk, Franí§ois; Dudek, Jonathan; Costas, Rodrigo (2020). "Communities of shared interests and cognitive bridges: The case of the anti-vaccination movement on Twitter". Scientometrics, v. 152, n. 2, pp. 1499-1516. https://doi.org/10.1007/s11192-020-03551-0
Zahedi, Zohreh; Costas, Rodrigo; Wouters, Paul (2014). "How well developed are altmetrics? A cross-disciplinary analysis of the presence of "˜alternative metrics´ in scientific publications". Scientometrics, v. 101, n. 2, pp. 1491-1513. https://doi.org/10.1007/s11192-014-1264-0
Zhang, Lin; Zhao, Wenjing; Sun, Beibei; Huang, Ying; Glí¤nzel, Wolfgang (2020). "How scientific research reacts to international public health emergencies: A global analysis of response patterns". Scientometrics, v. 124, n. 1, pp. 747-773. https://doi.org/10.1007/s11192-020-03531-4
Zhang, Yi; Cai, Xiaojing; Fry, Caroline V.; Wu, Mengjia; Wagner, Caroline S. (2021). "Topic evolution, disruption and resilience in early COVID-19 research". Scientometrics, v. 126, n. 5, pp. 4225-4253. https://doi.org/10.1007/s11192-021-03946-7
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Profesional de la información
This work is licensed under a Creative Commons Attribution 4.0 International License.
Dissemination conditions of the articles once they are published
Authors can freely disseminate their articles on websites, social networks and repositories
However, the following conditions must be respected:
- Only the editorial version should be made public. Please do not publish preprints, postprints or proofs.
- Along with this copy, a specific mention of the publication in which the text has appeared must be included, also adding a clickable link to the URL: http://www.profesionaldelainformacion.com
- Only the final editorial version should be made public. Please do not publish preprints, postprints or proofs.
- Along with that copy, a specific mention of the publication in which the text has appeared must be included, also adding a clickable link to the URL: http://revista.profesionaldelainformacion.com
Profesional de la información journal offers the articles in open access with a Creative Commons BY license.
