Toward non-human-centered design: designing an academic article with ChatGPT

Authors

  • Tuba Livberber Akdeniz University

DOI:

https://doi.org/10.3145/epi.2023.sep.12

Keywords:

Artificial intelligence, AI, Academic texts, Scientific articles, AI implementation, AI perspectives, Article design, Artificial intelligence in education, Assessment, ChatGPT, Conversational AI, GPT-3.5, Human-AI collaboration, Machine learning, Natural language processing (NLP), Learning, Teaching

Abstract

Non-human-centered design tools, such as ChatGPT, have shown potential as effective aids in academic article design. This study conducts a comparative evaluation of ChatGPT-3.5 and ChatGPT-4, examining their capabilities and limitations in supporting the academic article design process. The study aims to demonstrate the utility of ChatGPT as a writing tool and investigate its applicability and efficacy in the context of academic paper design. The author interacted with both versions of ChatGPT, providing prompts and analyzing the generated responses. In addition, a different expert academic was consulted to assess the appropriateness of the ChatGPT responses. The findings suggest that ChatGPT, despite its limitations, could serve as a useful tool for academic writing, particularly in the design of academic articles. Despite the limitations of both GPT-3.5 and GPT-4, GPT-3.5 offers a broader perspective, whereas GPT-4 provides a more in-depth and detailed approach to the design of articles. ChatGPT exhibits capabilities in aiding the design process, generating ideas aligned with the overall purpose and focus of the paper, producing consistent and contextually relevant responses to various natural language inputs, partially assisting in literature reviews, supporting paper design in terms of both content and format, and providing reasonable editing and proofreading for articles. However, limitations were identified, including reduced critical thinking, potential for plagiarism, risk of misinformation, lack of originality and innovation, and limited access to literature.

Downloads

Download data is not yet available.

References

AlAfnan, Mohammad-Awad; Mohdzuki, Siti-Fatimah (2023). "Do artificial intelligence chatbots have a writing style? An investigation into the stylistic features of ChatGPT-4". Journal of Artificial intelligence and technology, v. 3, n. 3. https://doi.org/10.37965/jait.2023.0267

Aljanabi, Mohammad (2023). "ChatGPT: Future directions and open possibilities". Mesopotamian journal of cybersecurity, pp. 16-17. https://doi.org/10.58496/MJCS/2023/003

Aljanabi, Mohammad; Ghazi, Mohanad; Ali, Ahmed-Hussein; Abed, Saad-Abas; ChatGPT (2023). "ChatGpt: Open possibilities". Iraqi journal for computer science and mathematics, v. 4, n. 1, pp. 62-64. https://doi.org/10.52866/ijcsm.2023.01.01.0018

Alkaissi, Hussam; Mcfarlane, Samy I. (2023). "Artificial hallucinations in ChatGPT: implications in scientific writing". Cureus, v. 15, n. 2. https://doi.org/10.7759/cureus.35179

Alshater, Muneer M. (2022). "Exploring the role of artificial intelligence in enhancing academic performance: A case study of ChatGPT". SSRN. https://doi.org/10.2139/ssrn.4312358

Altman, Sam (2022). "ChatGPT launched on Wednesday. Today it crossed 1 million users!". Twitter, December 5. [@sama]. https://twitter.com/sama/status/1599668808285028353

Amodei, Dario; Ananthanarayanan, Sundaram; Anubhai, Rishita; Bai, Jingliang; Battenberg, Eric; Case, Carl; "¦ ; Zhu, Zhenyao (2016). "Deep speech 2: End-to-end speech recognition in English and mandarin". In: Proceedings of the 33rd International conference on machine learning, pp. 173-182. http://proceedings.mlr.press/v48/amodei16.html?ref=https://codemonkey.link

Anderson, Nash; Belavy, Daniel L.; Perle, Stephen M.; Hendricks, Sharief; Hespanhol, Luiz; Verhagen, Evert; Memon, Aamir R. (2023). "AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation". BMJ open sport & exercise medicine, v. 9, n. 1, e001568. https://doi.org/10.1136/bmjsem-2023-001568

Ariyaratne, Sisith; Iyengar, Karthikeyan P.; Nischal, Neha; Babu, Naparla-Chitti; Botchu, Rajesh (2023). "A comparison of ChatGPT-generated articles with human-written articles". Skeletal radiology, n. 52. https://doi.org/10.1007/s00256-023-04340-5

Aydın, í–mer; Karaarslan, Enis (2022). "OpenAI ChatGPT generated literature review: Digital twin in healthcare". In: Aydın, í–mer (ed.). Emerging computer technologies 2. Ä°zmir Akademi Dernegi, pp. 22-31. https://doi.org/10.2139/ssrn.4308687

Baeza-Yates, Ricardo (2022). "Ethical challenges in AI". In: Proceedings of the fifteenth ACM international conference on web search and data mining, v. 15. https://doi.org/10.1145/3488560.3498370

Bang, Yejin; Cahyawijaya, Samuel; Lee, Nayeon; Dai, Wenliang; Su, Dan; Wilie, Bryan; Lovenia, Holy; Ji, Ziwei; Yu, Tiezheng; Chung, Willy; Do, Quyet V.; Xu, Yan; Fung, Pascale (2023). "A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity". ArXiv preprint. ArXiv:2302.04023. https://doi.org/10.48550/arXiv.2302.04023

Basic, Zeljana; Banovac, Ana; Kruzic, Ivana; Jerkovic, Ivan (2023). "Better by you, better than me, Chatgpt3 as writing assistance in students essays". ArXiv preprint. ArXiv:2302.04536. https://doi.org/10.48550/arXiv.2302.04536

Bishop, Lea (2023). "A computer wrote this paper: What ChatGPT means for education, research, and writing". Research, and writing, January 26. https://doi.org/10.2139/ssrn.4338981

Borji, Ali (2023). "A categorical archive of ChatGPT failures". ArXiv preprint. ArXiv:2302.03494. https://doi.org/10.48550/arXiv.2302.03494

Caira, Celine; Russo, Lucia; Aranda, Luis (2023). "Artificially inequitable? AI and closing the gender gap". OECD.AI. Policy observatory, March 8. https://oecd.ai/en/wonk/closing-the-gender-gap

Cao, Zhi (2023). "Evaluating the efficacy of ChatGPT-4 in providing scientific references across diverse disciplines". ArXiv preprint. ArXiv:2306.09914. https://doi.org/10.48550/arXiv.2306.09914

Castillo-González, William (2022). "ChatGPT and the future of scientific communication". Metaverse basic and applied research, v. 1, n. 8. https://doi.org/10.56294/mr20228

Chen, Lingjiao; Zaharia, Matei; Zou, James (2023). "How is ChatGPT´s behavior changing over time?". ArXiv preprint. ArXiv:2307.09009. https://doi.org/10.48550/arXiv.2307.09009

Cotton, Debby R. E.; Cotton, Peter A.; Shipway, J. Reuben (2023). "Chatting and cheating. Ensuring academic integrity in the era of ChatGPT". https://edarxiv.org/mrz8h?trk=public_post_main-feed-card_reshare-text

Crowe, Sarah; Cresswell, Kathrin; Robertson, Ann; Huby, Guro; Avery, Anthony; Sheikh, Aziz (2011). "The case study approach". BMC medical research methodology, v. 11, n. 1. https://doi.org/10.1186/1471-2288-11-100

Curtis, Nigel; ChatGPT (2023). "To ChatGPT or not to ChatGPT? The impact of artificial intelligence on academic publishing". The pediatric infectious disease journal, v. 42, n. 4, p. 275. https://doi.org/10.1097/INF.0000000000003852

De-Farias, Salomí£o-Alencar (2023). "Pí¢nico na Academia! Inteligíªncia artificial na construí§í£o de textos cientí­ficos com o uso do ChatGPT". Revista interdisciplinar de marketing, v. 13, n. 1, pp. 79-83.

Dellermann, Dominik; Calma, Adrian; Lipusch, Nikolaus; Weber, Thorsten; Weigel, Sascha; Ebel, Philipp (2021). "The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems". ArXiv preprint. ArXiv:2105.03354. https://doi.org/10.48550/arXiv.2105.03354

Dutton, Jonathan J. (2023). "Artificial intelligence and the future of computer-assisted medical research and writing". Ophthalmic plastic and reconstructive surgery, v. 39, n. 3, pp. 203-205. https://doi.org/10.1097/IOP.0000000000002420

Elsevier (2023). "The use of generative AI and AI-assisted technologies in scientific writing". Publishing ethics. https://www.elsevier.com/about/policies/publishing-ethics#

Fidel, Raya (1984). "The case study method: A case study". Library and information science research, v. 6, n. 3, pp. 273-288. http://faculty.washington.edu/fidelr/RayaPubs/TheCaseStudyMethod.pdf

Flanagan, Johansson (1999). "Public participation in the design of educational programmes for cancer nurses: a case report". European journal of cancer care, v. 8, n. 2, pp. 107-112. https://doi.org/10.1046/j.1365-2354.1999.00141.x

Flanagin, Annette; Bibbins-Domingo, Kirsten; Berkwits, Michael; Christiansen, Stacy L. (2023). "Nonhuman "˜authors´ and implications for the integrity of scientific publication and medical knowledge". JAMA, v. 329, n. 8, pp. 637-639. https://doi.org/10.1001/jama.2023.1344

Gao, Catherine A.; Howard, Frederick M.; Markov, Nikolay S.; Dyer, Emma C.; Ramesh, Siddhi; Luo, Yuan; Pearson, Alexander T. (2022). "Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers". bioRxiv, 2022-12. https://doi.org/10.1101/2022.12.23.521610

Gilat, Ron; Cole, Brian J. (2023). "How will artificial intelligence affect scientific writing, reviewing and editing? The future is here". Arthroscopy, v. 39, n. 5, pp. 1119-1120. https://doi.org/10.1016/j.arthro.2023.01.014

Gilson, Lucy (2012). "The case study approach". In: Gilson, Lucy (ed.). Health policy and systems research. A methodology reader. Geneva: Alliance for Health Policy and Systems Research, World Health Organisation, pp. 161-165. ISBN: 978 92 4 150313 6 https://assets.publishing.service.gov.uk/media/57a08a93ed915d622c0007db/Alliancehpsr_Reader.pdf

Golan, Roei; Reddy, Rohit; Muthigi, Akhil; Ramasamy, Ranjith (2023). "Artificial intelligence in academic writing: a paradigm-shifting technological advance". Nature reviews urology, 24 February. https://doi.org/10.1038/s41585-023-00746-x

Grimaldi, Gianluca; Ehrler, Bruno (2023). "AI et al.: Machines are about to change scientific publishing forever". ACS energy letters, v. 8, n. 1, pp. 878-880. https://doi.org/10.1021/acsenergylett.2c02828

Gustafsson, Johanna (2017). Single case studies vs. multiple case studies: a comparative study. http://www.diva-portal.org/smash/get/diva2:1064378/FULLTEXT01.pdf

Hammad, Mohamed (2023). "The impact of artificial intelligence (AI) programs on writing scientific research". Annals of biomedical engineering, v. 51, pp. 459-460. https://doi.org/10.1007/s10439-023-03140-1

Huang, Jingshan; Tan, Ming (2023). "The role of ChatGPT in scientific communication: writing better scientific review articles". American journal of cancer research, v. 13, n. 4, pp. 1148-1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801

Hughes, Alex (2023). "ChatGPT: Everything you need to know about OpenAI´s GPT-4 tool". Science focus, September 1. https://www.sciencefocus.com/future-technology/gpt-3

Hutson, Matthew (2022). "Could AI help you to write your next paper?". Nature, v. 611, n. 7934, pp. 192-193. https://doi.org/10.1038/d41586-022-03479-w

Kitamura, Felipe C. (2023). "ChatGPT is shaping the future of medical writing but still requires human judgment". Radiology, v. 307, n. 2, e230171. https://doi.org/10.1148/radiol.230171

Liedtka, Jeanne (2015). "Perspective: Linking design thinking with innovation outcomes through cognitive bias reduction". Journal of product innovation management, v. 32, n. 6, pp. 925-938. https://doi.org/10.1111/jpim.12163

Lin, Zhicheng (2023). "Why and how to embrace AI such as ChatGPT in your academic life". PsyArXiv, Preprint. https://psyarxiv.com/sdx3j

Lock, Samantha (2022). "What is ai chatbot phenomenon ChatGPT and could it replace humans?". The Guardian, December 5. https://www.theguardian.com/technology/2022/dec/05/what-is-ai-chatbot-phenomenon-chatgpt-and-could-it-replace-humans

Lund, Brady D.; Wang, Ting (2023). "Chatting about ChatGPT: How may AI and GPT impact academia and libraries?". Library hi tech news, v. 40, n. 3, pp. 26-29. https://doi.org/10.1108/LHTN-01-2023-0009

Lund, Brady D.; Wang, Ting; Mannuru, Nishith-Reddy; Nie, Bing; Shimray, Somipam; Wang, Ziang (2023). "ChatGPT and a new academic reality: Artificial intelligence"written research papers and the ethics of the large language models in scholarly publishing". Journal of the Association for Information Science and Technology, v. 74, n. 5, pp. 570-581. https://doi.org/10.1002/asi.24750

Ma, Yongqiang; Liu, Jiawei; Yi, Fan; Cheng, Qikai; Huang, Yong; Lu, Wei; Liu, Xiaozhong (2023). "Is this abstract generated by AI? A research for the gap between AI-generated scientific text and human-written scientific text". arXiv preprint arXiv:2301.10416. https://doi.org/10.48550/arXiv.2301.10416

Metz, Cade; Collins, Keith (2023). "10 ways GPT-4 is impressive but still flawed". The New York Times, March 14. https://www.nytimes.com/2023/03/14/technology/openai-new-gpt4.html

Mijwil, Maad; Aljanabi, Mohammad; ChatGPT (2023). "Towards artificial intelligence-based cybersecurity: The practices and ChatGPT generated ways to combat cybercrime". Iraqi journal for computer science and mathematics, v. 4, n. 1, pp. 65-70. https://doi.org/10.52866/ijcsm.2023.01.01.0019

Mijwil, Maad; Filali, Youssef; Aljanabi, Mohammad; Bounabi, Mariem; Al-Shahwani, Humam; ChatGPT (2023). "The purpose of cybersecurity governance in the digital transformation of public services and protecting the digital environment". Mesopotamian journal of cybersecurity. https://doi.org/10.58496/MJCS/2023/001

Nguyen, Minh-Hoang (2023). Academic writing and AI: Day-2 experiment with Bayesian mindsponge framework. Center for Open Science. https://econpapers.repec.org/paper/osfosfxxx/kr29c.htm

Nori, Harsha; King, Nicholas; McKinney, S. Mayer; Carignan, Dean; Horvitz, Eric (2023). "Capabilities of GPT-4 on medical challenge problems". ArXiv. Preprint. ArXiv:2303.13375. https://doi.org/10.48550/arXiv.2303.13375

O´Connor, Siobhan; ChatGPT (2022). "Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?". Nurse education in practice, v. 66. https://doi.org/10.1016/j.nepr.2022.103537

OpenAI (2022). ChatGPT-3: Optimizing language models for dialogue. https://chat.openai.com/chat

OpenAI (2023). ChatGPT 4. https://openai.com/gpt-4

Ouyang, Fan; Zheng, Luyi; Jiao, Pengcheng (2022). "Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020". Education and information technologies, v. 27, n. 6, pp. 7893-7925. https://doi.org/10.1007/s10639-022-10925-9

Pavlik, John V. (2023). "Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education". Journalism & mass communication educator, v. 78, n. 1, pp. 84-93. https://doi.org/10.1177/10776958221149577

Perkins, Mike (2023). "Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond". Journal of university teaching & learning practice, v. 20, n. 2. https://doi.org/10.53761/1.20.02.07

Polonsky, Michael-Jay; Rotman, Jeffrey D. (2023). "Should artificial intelligent (AI) agents be your coauthor? Arguments in favour, informed by ChatGPT". SSRN, preprint. https://doi.org/10.2139/ssrn.4349524

Radford, Alec; Wu, Jeffrey; Child, Rewon; Luan, David; Amodei, Dario; Sutskever, Ilya (2019). "Language models are unsupervised multitask learners". OpenAI blog, v. 1, n. 8. https://paperswithcode.com/paper/language-models-are-unsupervised-multitask

Rahaman, M. Saidur; Ahsan, M. M. Tahmid; Anjum, Nishath; Terano, Harold. J. R.; Rahman, M. Mizanur (2023). "From ChatGPT-3 to GPT-4: a significant advancement in ai-driven NLP tools". Journal of engineering and emerging technologies, v. 2, n. 1. https://doi.org/10.52631/jeet.v1i1.188

Rettberg, Jill-Walker (2022). "ChatGPT is multilingual but monocultural, and it´s learning your values". Jill/txt, December 6. https://jilltxt.net/right-now-chatgpt-is-multilingual-but-monocultural-but-its-learning-your-values

Roose, Kevin (2022). "The brilliance and weirdness of ChatGPT". The New York Times. https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html

Rudolph, Jí¼rgen; Tan, Samson; Tan, Shannon (2023). "ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?". Journal of applied learning and teaching, v. 6, n. 1. https://doi.org/10.37074/jalt.2023.6.1.9

Sabzalieva, Emma; Valentini, Arianna (2023). ChatGPT and artificial intelligence in higher education: quick start guide. France: Unesco. https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf

Srivastava, Mashrin (2023). "A day in the life of ChatGPT as a researcher: Sustainable and efficient machine learning-a review of sparsity techniques and future research directions". OSF Preprints. https://doi.org/10.13140/RG.2.2.34470.60480

Stokel-Walker, Chris (2023). "ChatGPT listed as author on research papers: many scientists disapprove". Nature, v. 613, n. 7945, pp. 620-621. https://doi.org/10.1038/d41586-023-00107-z

Sullivan, Miriam; Kelly, Andrew; McLaughlan, Paul (2023). "ChatGPT in higher education: Considerations for academic integrity and student learning". Journal of applied learning and teaching, v. 6, n. 1. https://doi.org/10.37074/jalt.2023.6.1.17

Susnjak, Teo (2022). "ChatGPT: The end of online exam integrity?". ArXiv preprint. ArXiv:2212.09292. https://doi.org/10.48550/arXiv.2212.09292

Teixeira-da-Silva, Jaime A. (2021). "Outsourced English revision, editing, publication consultation and integrity services should be acknowledged in an academic paper". Journal of nanoparticle research, v. 23, n. 4, 81. https://doi.org/10.1007/s11051-021-05199-0

Teixeira-da-Silva, Jaime A. (2023). "Is ChatGPT a valid author?". Nurse education in practice, v. 68, 103600. https://doi.org/10.1016/J.NEPR.2023.103600

Thorp, H. Holden (2023). "ChatGPT is fun, but not an author". Science, v. 379, n. 6630, pp. 313-313. https://doi.org/10.1126/science.adg7879

Túñez-López, José-Miguel; Fieiras-Ceide, César; Vaz-Álvarez, Martí­n (2021). "Impact of artificial intelligence on journalism: transformations in the company, products, contents, and professional profile". Communication & society, v. 34, n. 1, pp. 177-193. https://doi.org/10.15581/003.34.1.177-193

Verganti, Roberto; Vendraminelli, Luca; Iansiti, Marco (2020). "Innovation and design in the age of artificial intelligence". Journal of product innovation management, v. 37, n. 3, pp. 212-227. https://doi.org/10.1111/jpim.12523

Wager, Elizabeth; Kleinert, Sabine (2011). "Responsible research publication: international standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22-24, 2010". In: Mayer, T.; Steneck, N. (eds.). Promoting research integrity in a global environment. Singapore: Imperial College Press/World Science Publishing, pp. 309-316. https://publicationethics.org/files/International%20standards_authors_for%20website_11_Nov_2011.pdf

Wang, Dakuo; Churchill, Elizabeth; Maes, Pattie; Fan, Xiangmin; Shneiderman, Ben; Shi, Yuanchun; Wang, Qianying (2020). "From human-human collaboration to human-AI collaboration: Designing AI systems that can work together with people". In: Extended abstracts of the 2020 CHI conference on human factors in computing systems. https://doi.org/10.1145/3334480.3381069

Williamson, Ben; Macgilchrist, Felicitas; Potter, John (2023). "Re-examining AI, automation and datafication in education". Learning, media and technology, v. 48, n. 1. https://doi.org/10.1080/17439884.2023.2167830

Yin, Robert (2009). Case study research. Design and methods. London: SAGE.

Zainal, Zaidah (2007). "Case study as a research method". Jurnal Kemanusiaan, v. 5, n. 1. http://psyking.net/htmlobj-3837/case_study_as_a_research_method.pdf

Zhang, Yunhao; Gosline, Renee (2023). "Human favoritism, not AI aversion: People´s perceptions (and bias) toward creative content generated by Ai (ChatGPT-4), human experts, and human-AI collaboration". SSRN. https://doi.org/10.2139/ssrn.4453958

Published

2023-09-26

How to Cite

Livberber, T. (2023). Toward non-human-centered design: designing an academic article with ChatGPT. Profesional De La información, 32(5). https://doi.org/10.3145/epi.2023.sep.12

Issue

Section

Artificial Intelligence