Toward non-human-centered design: designing an academic article with ChatGPT
DOI:
https://doi.org/10.3145/epi.2023.sep.12Keywords:
Artificial intelligence, AI, Academic texts, Scientific articles, AI implementation, AI perspectives, Article design, Artificial intelligence in education, Assessment, ChatGPT, Conversational AI, GPT-3.5, Human-AI collaboration, Machine learning, Natural language processing (NLP), Learning, TeachingAbstract
Non-human-centered design tools, such as ChatGPT, have shown potential as effective aids in academic article design. This study conducts a comparative evaluation of ChatGPT-3.5 and ChatGPT-4, examining their capabilities and limitations in supporting the academic article design process. The study aims to demonstrate the utility of ChatGPT as a writing tool and investigate its applicability and efficacy in the context of academic paper design. The author interacted with both versions of ChatGPT, providing prompts and analyzing the generated responses. In addition, a different expert academic was consulted to assess the appropriateness of the ChatGPT responses. The findings suggest that ChatGPT, despite its limitations, could serve as a useful tool for academic writing, particularly in the design of academic articles. Despite the limitations of both GPT-3.5 and GPT-4, GPT-3.5 offers a broader perspective, whereas GPT-4 provides a more in-depth and detailed approach to the design of articles. ChatGPT exhibits capabilities in aiding the design process, generating ideas aligned with the overall purpose and focus of the paper, producing consistent and contextually relevant responses to various natural language inputs, partially assisting in literature reviews, supporting paper design in terms of both content and format, and providing reasonable editing and proofreading for articles. However, limitations were identified, including reduced critical thinking, potential for plagiarism, risk of misinformation, lack of originality and innovation, and limited access to literature.
Downloads
References
AlAfnan, Mohammad-Awad; Mohdzuki, Siti-Fatimah (2023). "Do artificial intelligence chatbots have a writing style? An investigation into the stylistic features of ChatGPT-4". Journal of Artificial intelligence and technology, v. 3, n. 3. https://doi.org/10.37965/jait.2023.0267
Aljanabi, Mohammad (2023). "ChatGPT: Future directions and open possibilities". Mesopotamian journal of cybersecurity, pp. 16-17. https://doi.org/10.58496/MJCS/2023/003
Aljanabi, Mohammad; Ghazi, Mohanad; Ali, Ahmed-Hussein; Abed, Saad-Abas; ChatGPT (2023). "ChatGpt: Open possibilities". Iraqi journal for computer science and mathematics, v. 4, n. 1, pp. 62-64. https://doi.org/10.52866/ijcsm.2023.01.01.0018
Alkaissi, Hussam; Mcfarlane, Samy I. (2023). "Artificial hallucinations in ChatGPT: implications in scientific writing". Cureus, v. 15, n. 2. https://doi.org/10.7759/cureus.35179
Alshater, Muneer M. (2022). "Exploring the role of artificial intelligence in enhancing academic performance: A case study of ChatGPT". SSRN. https://doi.org/10.2139/ssrn.4312358
Altman, Sam (2022). "ChatGPT launched on Wednesday. Today it crossed 1 million users!". Twitter, December 5. [@sama]. https://twitter.com/sama/status/1599668808285028353
Amodei, Dario; Ananthanarayanan, Sundaram; Anubhai, Rishita; Bai, Jingliang; Battenberg, Eric; Case, Carl; "¦ ; Zhu, Zhenyao (2016). "Deep speech 2: End-to-end speech recognition in English and mandarin". In: Proceedings of the 33rd International conference on machine learning, pp. 173-182. http://proceedings.mlr.press/v48/amodei16.html?ref=https://codemonkey.link
Anderson, Nash; Belavy, Daniel L.; Perle, Stephen M.; Hendricks, Sharief; Hespanhol, Luiz; Verhagen, Evert; Memon, Aamir R. (2023). "AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation". BMJ open sport & exercise medicine, v. 9, n. 1, e001568. https://doi.org/10.1136/bmjsem-2023-001568
Ariyaratne, Sisith; Iyengar, Karthikeyan P.; Nischal, Neha; Babu, Naparla-Chitti; Botchu, Rajesh (2023). "A comparison of ChatGPT-generated articles with human-written articles". Skeletal radiology, n. 52. https://doi.org/10.1007/s00256-023-04340-5
Aydın, í–mer; Karaarslan, Enis (2022). "OpenAI ChatGPT generated literature review: Digital twin in healthcare". In: Aydın, í–mer (ed.). Emerging computer technologies 2. Ä°zmir Akademi Dernegi, pp. 22-31. https://doi.org/10.2139/ssrn.4308687
Baeza-Yates, Ricardo (2022). "Ethical challenges in AI". In: Proceedings of the fifteenth ACM international conference on web search and data mining, v. 15. https://doi.org/10.1145/3488560.3498370
Bang, Yejin; Cahyawijaya, Samuel; Lee, Nayeon; Dai, Wenliang; Su, Dan; Wilie, Bryan; Lovenia, Holy; Ji, Ziwei; Yu, Tiezheng; Chung, Willy; Do, Quyet V.; Xu, Yan; Fung, Pascale (2023). "A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity". ArXiv preprint. ArXiv:2302.04023. https://doi.org/10.48550/arXiv.2302.04023
Basic, Zeljana; Banovac, Ana; Kruzic, Ivana; Jerkovic, Ivan (2023). "Better by you, better than me, Chatgpt3 as writing assistance in students essays". ArXiv preprint. ArXiv:2302.04536. https://doi.org/10.48550/arXiv.2302.04536
Bishop, Lea (2023). "A computer wrote this paper: What ChatGPT means for education, research, and writing". Research, and writing, January 26. https://doi.org/10.2139/ssrn.4338981
Borji, Ali (2023). "A categorical archive of ChatGPT failures". ArXiv preprint. ArXiv:2302.03494. https://doi.org/10.48550/arXiv.2302.03494
Caira, Celine; Russo, Lucia; Aranda, Luis (2023). "Artificially inequitable? AI and closing the gender gap". OECD.AI. Policy observatory, March 8. https://oecd.ai/en/wonk/closing-the-gender-gap
Cao, Zhi (2023). "Evaluating the efficacy of ChatGPT-4 in providing scientific references across diverse disciplines". ArXiv preprint. ArXiv:2306.09914. https://doi.org/10.48550/arXiv.2306.09914
Castillo-González, William (2022). "ChatGPT and the future of scientific communication". Metaverse basic and applied research, v. 1, n. 8. https://doi.org/10.56294/mr20228
Chen, Lingjiao; Zaharia, Matei; Zou, James (2023). "How is ChatGPT´s behavior changing over time?". ArXiv preprint. ArXiv:2307.09009. https://doi.org/10.48550/arXiv.2307.09009
Cotton, Debby R. E.; Cotton, Peter A.; Shipway, J. Reuben (2023). "Chatting and cheating. Ensuring academic integrity in the era of ChatGPT". https://edarxiv.org/mrz8h?trk=public_post_main-feed-card_reshare-text
Crowe, Sarah; Cresswell, Kathrin; Robertson, Ann; Huby, Guro; Avery, Anthony; Sheikh, Aziz (2011). "The case study approach". BMC medical research methodology, v. 11, n. 1. https://doi.org/10.1186/1471-2288-11-100
Curtis, Nigel; ChatGPT (2023). "To ChatGPT or not to ChatGPT? The impact of artificial intelligence on academic publishing". The pediatric infectious disease journal, v. 42, n. 4, p. 275. https://doi.org/10.1097/INF.0000000000003852
De-Farias, Salomí£o-Alencar (2023). "Pí¢nico na Academia! Inteligíªncia artificial na construí§í£o de textos científicos com o uso do ChatGPT". Revista interdisciplinar de marketing, v. 13, n. 1, pp. 79-83.
Dellermann, Dominik; Calma, Adrian; Lipusch, Nikolaus; Weber, Thorsten; Weigel, Sascha; Ebel, Philipp (2021). "The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems". ArXiv preprint. ArXiv:2105.03354. https://doi.org/10.48550/arXiv.2105.03354
Dutton, Jonathan J. (2023). "Artificial intelligence and the future of computer-assisted medical research and writing". Ophthalmic plastic and reconstructive surgery, v. 39, n. 3, pp. 203-205. https://doi.org/10.1097/IOP.0000000000002420
Elsevier (2023). "The use of generative AI and AI-assisted technologies in scientific writing". Publishing ethics. https://www.elsevier.com/about/policies/publishing-ethics#
Fidel, Raya (1984). "The case study method: A case study". Library and information science research, v. 6, n. 3, pp. 273-288. http://faculty.washington.edu/fidelr/RayaPubs/TheCaseStudyMethod.pdf
Flanagan, Johansson (1999). "Public participation in the design of educational programmes for cancer nurses: a case report". European journal of cancer care, v. 8, n. 2, pp. 107-112. https://doi.org/10.1046/j.1365-2354.1999.00141.x
Flanagin, Annette; Bibbins-Domingo, Kirsten; Berkwits, Michael; Christiansen, Stacy L. (2023). "Nonhuman "˜authors´ and implications for the integrity of scientific publication and medical knowledge". JAMA, v. 329, n. 8, pp. 637-639. https://doi.org/10.1001/jama.2023.1344
Gao, Catherine A.; Howard, Frederick M.; Markov, Nikolay S.; Dyer, Emma C.; Ramesh, Siddhi; Luo, Yuan; Pearson, Alexander T. (2022). "Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers". bioRxiv, 2022-12. https://doi.org/10.1101/2022.12.23.521610
Gilat, Ron; Cole, Brian J. (2023). "How will artificial intelligence affect scientific writing, reviewing and editing? The future is here". Arthroscopy, v. 39, n. 5, pp. 1119-1120. https://doi.org/10.1016/j.arthro.2023.01.014
Gilson, Lucy (2012). "The case study approach". In: Gilson, Lucy (ed.). Health policy and systems research. A methodology reader. Geneva: Alliance for Health Policy and Systems Research, World Health Organisation, pp. 161-165. ISBN: 978 92 4 150313 6 https://assets.publishing.service.gov.uk/media/57a08a93ed915d622c0007db/Alliancehpsr_Reader.pdf
Golan, Roei; Reddy, Rohit; Muthigi, Akhil; Ramasamy, Ranjith (2023). "Artificial intelligence in academic writing: a paradigm-shifting technological advance". Nature reviews urology, 24 February. https://doi.org/10.1038/s41585-023-00746-x
Grimaldi, Gianluca; Ehrler, Bruno (2023). "AI et al.: Machines are about to change scientific publishing forever". ACS energy letters, v. 8, n. 1, pp. 878-880. https://doi.org/10.1021/acsenergylett.2c02828
Gustafsson, Johanna (2017). Single case studies vs. multiple case studies: a comparative study. http://www.diva-portal.org/smash/get/diva2:1064378/FULLTEXT01.pdf
Hammad, Mohamed (2023). "The impact of artificial intelligence (AI) programs on writing scientific research". Annals of biomedical engineering, v. 51, pp. 459-460. https://doi.org/10.1007/s10439-023-03140-1
Huang, Jingshan; Tan, Ming (2023). "The role of ChatGPT in scientific communication: writing better scientific review articles". American journal of cancer research, v. 13, n. 4, pp. 1148-1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801
Hughes, Alex (2023). "ChatGPT: Everything you need to know about OpenAI´s GPT-4 tool". Science focus, September 1. https://www.sciencefocus.com/future-technology/gpt-3
Hutson, Matthew (2022). "Could AI help you to write your next paper?". Nature, v. 611, n. 7934, pp. 192-193. https://doi.org/10.1038/d41586-022-03479-w
Kitamura, Felipe C. (2023). "ChatGPT is shaping the future of medical writing but still requires human judgment". Radiology, v. 307, n. 2, e230171. https://doi.org/10.1148/radiol.230171
Liedtka, Jeanne (2015). "Perspective: Linking design thinking with innovation outcomes through cognitive bias reduction". Journal of product innovation management, v. 32, n. 6, pp. 925-938. https://doi.org/10.1111/jpim.12163
Lin, Zhicheng (2023). "Why and how to embrace AI such as ChatGPT in your academic life". PsyArXiv, Preprint. https://psyarxiv.com/sdx3j
Lock, Samantha (2022). "What is ai chatbot phenomenon ChatGPT and could it replace humans?". The Guardian, December 5. https://www.theguardian.com/technology/2022/dec/05/what-is-ai-chatbot-phenomenon-chatgpt-and-could-it-replace-humans
Lund, Brady D.; Wang, Ting (2023). "Chatting about ChatGPT: How may AI and GPT impact academia and libraries?". Library hi tech news, v. 40, n. 3, pp. 26-29. https://doi.org/10.1108/LHTN-01-2023-0009
Lund, Brady D.; Wang, Ting; Mannuru, Nishith-Reddy; Nie, Bing; Shimray, Somipam; Wang, Ziang (2023). "ChatGPT and a new academic reality: Artificial intelligence"written research papers and the ethics of the large language models in scholarly publishing". Journal of the Association for Information Science and Technology, v. 74, n. 5, pp. 570-581. https://doi.org/10.1002/asi.24750
Ma, Yongqiang; Liu, Jiawei; Yi, Fan; Cheng, Qikai; Huang, Yong; Lu, Wei; Liu, Xiaozhong (2023). "Is this abstract generated by AI? A research for the gap between AI-generated scientific text and human-written scientific text". arXiv preprint arXiv:2301.10416. https://doi.org/10.48550/arXiv.2301.10416
Metz, Cade; Collins, Keith (2023). "10 ways GPT-4 is impressive but still flawed". The New York Times, March 14. https://www.nytimes.com/2023/03/14/technology/openai-new-gpt4.html
Mijwil, Maad; Aljanabi, Mohammad; ChatGPT (2023). "Towards artificial intelligence-based cybersecurity: The practices and ChatGPT generated ways to combat cybercrime". Iraqi journal for computer science and mathematics, v. 4, n. 1, pp. 65-70. https://doi.org/10.52866/ijcsm.2023.01.01.0019
Mijwil, Maad; Filali, Youssef; Aljanabi, Mohammad; Bounabi, Mariem; Al-Shahwani, Humam; ChatGPT (2023). "The purpose of cybersecurity governance in the digital transformation of public services and protecting the digital environment". Mesopotamian journal of cybersecurity. https://doi.org/10.58496/MJCS/2023/001
Nguyen, Minh-Hoang (2023). Academic writing and AI: Day-2 experiment with Bayesian mindsponge framework. Center for Open Science. https://econpapers.repec.org/paper/osfosfxxx/kr29c.htm
Nori, Harsha; King, Nicholas; McKinney, S. Mayer; Carignan, Dean; Horvitz, Eric (2023). "Capabilities of GPT-4 on medical challenge problems". ArXiv. Preprint. ArXiv:2303.13375. https://doi.org/10.48550/arXiv.2303.13375
O´Connor, Siobhan; ChatGPT (2022). "Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?". Nurse education in practice, v. 66. https://doi.org/10.1016/j.nepr.2022.103537
OpenAI (2022). ChatGPT-3: Optimizing language models for dialogue. https://chat.openai.com/chat
OpenAI (2023). ChatGPT 4. https://openai.com/gpt-4
Ouyang, Fan; Zheng, Luyi; Jiao, Pengcheng (2022). "Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020". Education and information technologies, v. 27, n. 6, pp. 7893-7925. https://doi.org/10.1007/s10639-022-10925-9
Pavlik, John V. (2023). "Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education". Journalism & mass communication educator, v. 78, n. 1, pp. 84-93. https://doi.org/10.1177/10776958221149577
Perkins, Mike (2023). "Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond". Journal of university teaching & learning practice, v. 20, n. 2. https://doi.org/10.53761/1.20.02.07
Polonsky, Michael-Jay; Rotman, Jeffrey D. (2023). "Should artificial intelligent (AI) agents be your coauthor? Arguments in favour, informed by ChatGPT". SSRN, preprint. https://doi.org/10.2139/ssrn.4349524
Radford, Alec; Wu, Jeffrey; Child, Rewon; Luan, David; Amodei, Dario; Sutskever, Ilya (2019). "Language models are unsupervised multitask learners". OpenAI blog, v. 1, n. 8. https://paperswithcode.com/paper/language-models-are-unsupervised-multitask
Rahaman, M. Saidur; Ahsan, M. M. Tahmid; Anjum, Nishath; Terano, Harold. J. R.; Rahman, M. Mizanur (2023). "From ChatGPT-3 to GPT-4: a significant advancement in ai-driven NLP tools". Journal of engineering and emerging technologies, v. 2, n. 1. https://doi.org/10.52631/jeet.v1i1.188
Rettberg, Jill-Walker (2022). "ChatGPT is multilingual but monocultural, and it´s learning your values". Jill/txt, December 6. https://jilltxt.net/right-now-chatgpt-is-multilingual-but-monocultural-but-its-learning-your-values
Roose, Kevin (2022). "The brilliance and weirdness of ChatGPT". The New York Times. https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html
Rudolph, Jí¼rgen; Tan, Samson; Tan, Shannon (2023). "ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?". Journal of applied learning and teaching, v. 6, n. 1. https://doi.org/10.37074/jalt.2023.6.1.9
Sabzalieva, Emma; Valentini, Arianna (2023). ChatGPT and artificial intelligence in higher education: quick start guide. France: Unesco. https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf
Srivastava, Mashrin (2023). "A day in the life of ChatGPT as a researcher: Sustainable and efficient machine learning-a review of sparsity techniques and future research directions". OSF Preprints. https://doi.org/10.13140/RG.2.2.34470.60480
Stokel-Walker, Chris (2023). "ChatGPT listed as author on research papers: many scientists disapprove". Nature, v. 613, n. 7945, pp. 620-621. https://doi.org/10.1038/d41586-023-00107-z
Sullivan, Miriam; Kelly, Andrew; McLaughlan, Paul (2023). "ChatGPT in higher education: Considerations for academic integrity and student learning". Journal of applied learning and teaching, v. 6, n. 1. https://doi.org/10.37074/jalt.2023.6.1.17
Susnjak, Teo (2022). "ChatGPT: The end of online exam integrity?". ArXiv preprint. ArXiv:2212.09292. https://doi.org/10.48550/arXiv.2212.09292
Teixeira-da-Silva, Jaime A. (2021). "Outsourced English revision, editing, publication consultation and integrity services should be acknowledged in an academic paper". Journal of nanoparticle research, v. 23, n. 4, 81. https://doi.org/10.1007/s11051-021-05199-0
Teixeira-da-Silva, Jaime A. (2023). "Is ChatGPT a valid author?". Nurse education in practice, v. 68, 103600. https://doi.org/10.1016/J.NEPR.2023.103600
Thorp, H. Holden (2023). "ChatGPT is fun, but not an author". Science, v. 379, n. 6630, pp. 313-313. https://doi.org/10.1126/science.adg7879
Túñez-López, José-Miguel; Fieiras-Ceide, César; Vaz-Álvarez, Martín (2021). "Impact of artificial intelligence on journalism: transformations in the company, products, contents, and professional profile". Communication & society, v. 34, n. 1, pp. 177-193. https://doi.org/10.15581/003.34.1.177-193
Verganti, Roberto; Vendraminelli, Luca; Iansiti, Marco (2020). "Innovation and design in the age of artificial intelligence". Journal of product innovation management, v. 37, n. 3, pp. 212-227. https://doi.org/10.1111/jpim.12523
Wager, Elizabeth; Kleinert, Sabine (2011). "Responsible research publication: international standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22-24, 2010". In: Mayer, T.; Steneck, N. (eds.). Promoting research integrity in a global environment. Singapore: Imperial College Press/World Science Publishing, pp. 309-316. https://publicationethics.org/files/International%20standards_authors_for%20website_11_Nov_2011.pdf
Wang, Dakuo; Churchill, Elizabeth; Maes, Pattie; Fan, Xiangmin; Shneiderman, Ben; Shi, Yuanchun; Wang, Qianying (2020). "From human-human collaboration to human-AI collaboration: Designing AI systems that can work together with people". In: Extended abstracts of the 2020 CHI conference on human factors in computing systems. https://doi.org/10.1145/3334480.3381069
Williamson, Ben; Macgilchrist, Felicitas; Potter, John (2023). "Re-examining AI, automation and datafication in education". Learning, media and technology, v. 48, n. 1. https://doi.org/10.1080/17439884.2023.2167830
Yin, Robert (2009). Case study research. Design and methods. London: SAGE.
Zainal, Zaidah (2007). "Case study as a research method". Jurnal Kemanusiaan, v. 5, n. 1. http://psyking.net/htmlobj-3837/case_study_as_a_research_method.pdf
Zhang, Yunhao; Gosline, Renee (2023). "Human favoritism, not AI aversion: People´s perceptions (and bias) toward creative content generated by Ai (ChatGPT-4), human experts, and human-AI collaboration". SSRN. https://doi.org/10.2139/ssrn.4453958
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Profesional de la información
This work is licensed under a Creative Commons Attribution 4.0 International License.
Dissemination conditions of the articles once they are published
Authors can freely disseminate their articles on websites, social networks and repositories
However, the following conditions must be respected:
- Only the editorial version should be made public. Please do not publish preprints, postprints or proofs.
- Along with this copy, a specific mention of the publication in which the text has appeared must be included, also adding a clickable link to the URL: http://www.profesionaldelainformacion.com
- Only the final editorial version should be made public. Please do not publish preprints, postprints or proofs.
- Along with that copy, a specific mention of the publication in which the text has appeared must be included, also adding a clickable link to the URL: http://revista.profesionaldelainformacion.com
Profesional de la información journal offers the articles in open access with a Creative Commons BY license.
