Measuring the readability of exhibit panels in museums
DOI:
https://doi.org/10.3145/epi.2018.may.10Keywords:
Readability, Legibility, Metrics, Museum, Informative panels, NLP, Word frequency lists.Abstract
The objective of this paper is the assessment of the comprehensibility in museum panels, considering that they are addressed to all audiences and written in standard language. The indicators proposed in the scientific bibliography on comprehensibility are compiled. Subsequently, their application to the museum panels is analyzed. A corpus is built with the texts of panels from six museums and the difficulty of the panels is evaluated by means of classical metrics and tests on users´ perception. The ability of these classic metrics to predict the difficulty for the user is computed with automatic learning methods. Linguistic and term familiarity indicators have been added to improve the accuracy of the assessment. The most effective way to predict the degree of comprehensibility is a hybrid model of classical, linguistic and term familiarity indicators.
Downloads
References
Aluísio, Sandra; Gasperin, Caroline (2010). "Fostering digital inclusion and accessibility: The PorSimples project for simplification of portuguese texts". En: Procs of the NAACL HLT 2010 Young investigarors workshop on computational approaches to languages of the Americas, Los Angeles, California, June, pp. 46-53. http://www.aclweb.org/anthology/W10-1607
Aluisio, Sandra; Specia, Lucia; Gasperin, Caroline; Scarton, Carolina (2010). "Readability assessment for text simplification". En: Fifth workshop on innovative use of NLP for building educational applications, pp. 1-9. https://goo.gl/S8L2WJ
Badarudeen, Sameer; Sabharwal, Sanjeev (2010). "Assessing readability of patient education materials: current role in orthopaedics". Clinical orthopaedics and related research, v. 468, n. 10, pp. 2572-2580. https://doi.org/10.1007/s11999-010-1380-y
Blanco, Roi; Lioma, Christina (2012). "Graph-based term weighting for information retrieval". Information retrieval, v. 15, n. 1, pp. 54-92. https://doi.org/10.1007/s10791-011-9172-x
Bott, Stefan; Rello, Luz; Drndarevic, Biljana; Saggion, Horacio (2012). "Can Spanish be simpler? LexSiS: Lexical simplification for Spanish". En: Proceedings of Coling 2012, pp. 357-374. https://aclweb.org/anthology/C/C12/C12-1023.pdf
Carbon, Claus-Christian (2017). "Art perception in the museum: How we spend time and space in art exhibitions". i-Perception, v. 8, n. 1. https://doi.org/10.1177/2041669517694184
Dale, Edgar; Chall, Jeanne S. (1948). "A formula for predicting readability". Educational research bulletin, v. 27, n. 1, pp. 11-28.
Elhadad, Noémie (2006). "Comprehending technical texts: Predicting and defining unfamiliar terms". En: AMIA Annual symposium proceedings, pp. 239-243. https://www.ncbi.nlm.nih.gov/pubmed/17238339
España (2002). "Ley 34/2002, de 11 de julio, de servicios de la sociedad de la información y de comercio electrónico". BOE, n. 166, 12 julio. https://www.boe.es/buscar/act.php?id=BOE-A-2002-13758
España (2013) "Ley 19/2013, de 9 de diciembre, de transparencia, acceso a la información pública y buen gobierno". BOE, n. 295, 10 diciembre. https://www.boe.es/buscar/doc.php?id=BOE-A-2013-12887
Feng, Lijun; Jansche, Martin; Huenerfauth, Matt; Elhadad, Noémie (2010). "A comparison of features for automatic readability assessment". En: Procs of the 23rd Intl conf on computational linguistics: Posters, pp. 276-284. http://dl.acm.org/citation.cfm?id=1944566.1944598
Ferrando-Belart, Vicky (2004). "La legibilidad: un factor fundamental para comprender un texto". Atención primaria, v. 34, n. 3, pp. 143-146. https://goo.gl/E3S4Ko
Fleiss, Joseph L. (1971). "Measuring nominal scale agreement among many raters". Psychological bulletin, v. 76, n. 5, pp. 378-382. https://doi.org/10.1037/h0031619
Franí§ois, Thomas; Fairon, Cédrick (2012). "An "˜AI readability´ formula for French as a foreign language". En: Joint conference on empirical methods in natural language processing and computational natural language learning, Jeju Island, Korea, 12-14 July, pp. 466-477. http://www.aclweb.org/anthology/D12-1043
Franí§ois, Thomas; Miltsakaki, Eleni (2012). "Do NLP and machine learning improve traditional readability formulas?". En: Proceedings of the First workshop on predicting and improving text readability for target reader populations, pp. 49-57. http://dl.acm.org/citation.cfm?id=2390916.2390925
Graesser, Arthur C.; McNamara, Danielle S.; Kulikowich, Jonna M. (2011). "Coh-Metrix: Providing multilevel analyses of text characteristics". Educational researcher, v. 40, n. 5, pp. 223-234. https://doi.org/10.2307/2529310
Graesser, Arthur C.; McNamara, Danielle S.; Louwerse, Max M.; Cai, Zhiqiang (2004). "Coh-Metrix: Analysis of text on cohesion and language". Behavior research methods, instruments, & computers, v. 36, n. 2, pp. 193-202. https://doi.org/10.3758/BF03195564
Instituto de Turismo de España (2014). Museo del Prado. Caracterización de los visitantes. Encuesta a los visitantes del Museo del Prado. https://goo.gl/k5c19g
Kauchak, David; Leroy, Gondy; Hogue, Alan (2017). "Measuring text difficulty using parse-tree frequency". Journal of the Asociation for Information Science and Technology, v. 68, n. 9, pp. 2088-2100. https://doi.org/10.1002/asi.23855
Keselman, Alla; Tse, Tony; Crowell, Jon; Browne, Allen; Ngo, Long; Zeng, Qing (2007). "Assessing consumer health vocabulary familiarity: An exploratory study". Journal of medical internet research, v. 9, n. 1, e5. https://doi.org/10.2196/jmir.9.1.e5
Landis, J. Richard; Koch, G. Gary (1977). "The measurement of observer agreement for categorical data". Biometrics, v. 33, n. 1, pp. 159-174. http://www.ncbi.nlm.nih.gov/pubmed/843571
Larsson, Patrik (2006). Classification into readability levels. Implementation and evaluation. Uppsala Universitet. Department of Linguistics and Philology. https://goo.gl/YuTSvq
Leroy, Gondy; Endicott, James E. (2011). "Term familiarity to indicate perceived and actual difficulty of text in medical digital libraries". En: Xing, Chunxiao; Crestani, Fabio; Rauber, A. (eds.). Digital libraries for cultural heritage, knowledge dissemination, and future creation: 13th Intl conf on Asia-Pacific digital libraries, Icadl 2011, Beijing, China, October 24-27, v. 7008, pp. 307-310. https://doi.org/10.1007/978-3-642-24826-9_38
Leroy, Gondy; Endicott, James E. (2012). "Combining NLP with evidence-based methods to find text metrics related to perceived and actual text difficulty". En: Proceedings of the 2nd ACM Sighit Intl health informatics symposium, pp. 749-754. https://doi.org/10.1145/2110363.2110452
Leroy, Gondy; Endicott, James E.; Kauchak, David; Mouradi, Obay; Just, Melissa (2013). "User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention". Journal of medical internet research, v. 15, n. 7, e144.
Ministerio de Cultura (2011). Conociendo a nuestros visitantes. Estudio de público en museos del Ministerio de Cultura. Resumen ejecutivo. Ministerio de Educación, Cultura y Deporte. https://goo.gl/Xhirta
Montanero-Fernández, Manuel (2004). "Cómo evaluar la comprensión lectora: alternativas y limitaciones". Revista de educación, v. 335, pp. 415-424. https://dialnet.unirioja.es/servlet/articulo?codigo=1066564
Morato, Jorge; Llorens, Juan; Génova, Gonzalo; Moreiro-González, José-Antonio (2003). "Experiments in discourse analysis impact on information classification and retrieval algorithms". Information processing & management, v. 39, n. 6, pp. 825-851. https://doi.org/10.1016/S0306-4573(02)00081-X
Morato, Jorge; Sánchez-Cuadrado, Sonia; Moreno, Valentín; Moreiro-González, José-Antonio (2013). "Evolución de los factores de posicionamiento web y adaptación de las herramientas de optimización". Revista española de documentación científica, v. 36, n. 3, e018. https://doi.org/10.3989/redc.2013.3.956
Newbold, Neil; McLaughlin, Harry; Gillam, Lee (2010). "Rank by readability: Document weighting for information retrieval". En: Cunningham, Hamish; Hanbury, Allan; Rí¼ger, Stefan. Advances in multidisciplinary retrieval. IRFC. Lecture notes in computer science, v. 6107, pp. 20-30. https://doi.org/10.1007/978-3-642-13084-7_3
Nomura, Misako; Nielsen, Gyda S.; Tronbacke, Bror (2010). Guidelines for easy-to-read materials. IFLA professional reports. https://www.ifla.org/files/assets/hq/publications/professional-report/120.pdf
OECD (2016). Skills matter. Further results from the survey of adult skills. OECD Publishing. ISBN: 978 92 64 25804 4 https://doi.org/10.1787/9789264258051-en
Padró, Lluís; Collado, Miquel; Reese, Samuel; Lloberes, Marina; Castellón, Irene (2010). "FreeLing 2.1: Five years of open-source language processing tools". En: Procs of the 7th Intl conf on language resources and evaluation (LREC´10), pp. 931-936. https://goo.gl/bpQb1P
Pitler, Emily; Nenkova, Ani (2008). "Revisiting readability: A unified framework for predicting text quality". In: Procs of the Conf on empirical methods in natural language processing, pp. 186-195. http://dl.acm.org/citation.cfm?id=1613715.1613742
Sigaud-Sellos, Pedro (2010). Aproximación a los conceptos de legibilidad y lecturabilidad: aplicación a la lectura de textos digitales. Pamplona: Universidad de Navarra. https://goo.gl/2uh4K3
Social Science Consulting (2013). TextQuest. http://www.textquest.de/tq-man42.pdf
Stenner, A. Jackson (1996). "Measuring reading comprehension with the lexile framework". En: 4th North American conf on adolescent/adult literacy. Washington, D.C. https://goo.gl/tVTSbC
Stenner, A. Jackson; Smith, Malbert; Burdick, Donald S. (1983). "Toward a theory of construct definition". Journal of educational measurement, v. 20, n. 4, pp. 305-316. https://doi.org/10.1111/j.1745-3984.1983.tb00209.x
Temnikova, Irina; Vieweg, Sarah; Castillo, Carlos (2015). "The case for readability of crisis communications in social media". In: Procs of the 24th Intl conf on world wide web, pp. 1245-1250. https://doi.org/10.1145/2740908.2741718
Tonelli, Sara; Tran-Manh, Ke; Pianta, Emanuele (2012). "Making readability indices readable". En: Naacl-HLT 2012 Workshop on predicting and improving text readability for target reader populations, pp. 40-48. http://www.aclweb.org/anthology/W12-2206
Van-Oosten, Philip; Tanghe, Dries; Hoste, Veronique (2010). "Towards an improved methodology for automated readability prediction". En: LREC 2010: Seventh conference on international language resources and evaluation, pp. 775-782. http://hdl.handle.net/1854/LU-1055826
Venturi, Giulia; Bellandi, Tommaso; Dell´Orletta, Felice; Montemagni, Simonetta (2015). "NLP-based readability assessment of health-related texts: A case study on Italian informed consent forms". En: Procs of the 6th intl workshop on health text mining and information analysis, pp. 131-141. http://www.aclweb.org/anthology/W15-2618
Witten, Ian H.; Frank, Eibe; Hall, Mark A. (2011). "Data mining: Practical machine learning tools and techniques" (3rd ed.). San Francisco (USA): Morgan Kaufmann Publishers. ISBN: 978 0 123748560
Zeng-Treitler, Qing; Kim, Hyeoneui; Goryachev, Sergey; Keselman, Alla; Slaughter, Laura; Smith, Catherine-Arnott (2007). "Text characteristics of clinical reports and their implications for the readability of personal health records". Studies in health technology and informatics, v. 129, pp. 1117-1121. http://ebooks.iospress.nl/publication/11151
Downloads
Published
How to Cite
Issue
Section
License
Dissemination conditions of the articles once they are published
Authors can freely disseminate their articles on websites, social networks and repositories
However, the following conditions must be respected:
- Only the editorial version should be made public. Please do not publish preprints, postprints or proofs.
- Along with this copy, a specific mention of the publication in which the text has appeared must be included, also adding a clickable link to the URL: http://www.profesionaldelainformacion.com
- Only the final editorial version should be made public. Please do not publish preprints, postprints or proofs.
- Along with that copy, a specific mention of the publication in which the text has appeared must be included, also adding a clickable link to the URL: http://revista.profesionaldelainformacion.com
Profesional de la información journal offers the articles in open access with a Creative Commons BY license.
