Quick review of pedagogical experiences using GPT-3 in education

  1. Joel Manuel Prieto Andreu 1
  2. Antonio Labisa Palmeira 2
  1. 1 Universidad Internacional de La Rioja
    info

    Universidad Internacional de La Rioja

    Logroño, España

    ROR https://ror.org/029gnnp81

  2. 2 Universidade Lusófona de Humanidades e Tecnologias (Portugal)
Revue:
JOTSE

ISSN: 2013-6374

Année de publication: 2024

Volumen: 14

Número: 2

Pages: 633-647

Type: Article

DOI: 10.3926/JOTSE.2111 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

D'autres publications dans: JOTSE

Résumé

GPT-3 is a neuronal language model that performs tasks such as classification, question-answering and text summarization. Although chatbots like BlenderBot-3 work well in a conversational sense, and GPT-3 can assist experts in evaluating questions, they are quantifiably worse than real teachers in several pedagogical dimensions. We present the first systematic literature review that analyzes the main contributions and uses of GPT-3 in the field of education. The protocols suggested in the PRISMA 2020 statement were followed for the drafting of the review. According to the results, 34 significant productions were identified through a systematic search in ISI Web of Science, SCOPUS and Google Scholar. GPT-3 has been considered in the academic, ethical and medical fields, in humanities and in computer science, in the formulation of questions and answers, and through cooperative educational dialogs. GPT-3 has been proven to have valuable applications in education, such as the automation of routine tasks, in making quick diagnoses of the students’ weaknesses and in the automatic generation of questions, but it still faces challenges and limitations that require additional investigation. We discuss the educational possibilities and the limitations to the use of GPT-3.

Références bibliographiques

  • Abdelghani, R., Wang, Y.H., Yuan, X., Wang, T., Sauzéon, H., & Oudeyer, P.Y. (2022). GPT-3-driven pedagogical agents for training children’s curious question-asking skills. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00340-7
  • Agustin, Y.H. (2022). Implementation of Generative Pre-Trained Transformer 3 Classify-Text in Determining Thesis Supervisor. Sinkron: Jurnal dan Penelitian Teknik Informatika, 7(4), 2415-2420. https://doi.org/10.33395/sinkron.v7i4.11757
  • Alawi, F. (2023). Artificial intelligence: The future might already be here. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 135(3), 313-315. https://doi.org/10.1016/j.oooo.2023.01.002
  • Aljanabi, M., Ghazi, M., Ali, A.H., & Abed, S.A. (2023). ChatGpt: Open Possibilities. Iraqi Journal for Computer Science and Mathematics, 4(1), 62-64. https://doi.org/10.52866/20ijcsm.2023.01.01.0018
  • Alvarez-Carmona, M.A., Aranda, R., Diaz-Pacheco, A., & Ceballos-Mejıa, J.J. (2022). Generador automático de resúmenes científicos en investigación turística. SciELO Preprints. https://doi.org/10.1590/scielopreprints.4194
  • Aydın, N., & Erdem, O.A. (2022). A Research On The New Generation Artificial Intelligence Technology Generative Pretraining Transformer 3. In 3rd International Informatics and Software Engineering Conference (IISEC) (1-6). IEEE. https://doi.org/10.1109/iisec56263.2022.9998298
  • Bhat, S., Nguyen, H.A., Moore, S., Stamper, J., Sakr, M., & Nyberg, E. (2022). Towards Automated Generation and Evaluation of Questions in Educational Domains. In Proceedings of the 15th International Conference on Educational Data Mining (701-704). Durham, United Kingdom.
  • Chan, A. (2022). GPT-3 and Instruct GPT: Technological dystopianism, utopianism, and “Contextual” perspectives in AI ethics and industry. AI Ethics, 3, 53-64. https://doi.org/10.1007/s43681-022-00148-6
  • Franganillo, J. (2022). Contenido generado por inteligencia artificial: oportunidades y amenazas. Anuario ThinkEPI, 16. https://doi.org/10.3145/thinkepi.2022.e16a24
  • Gaikwad, A., Rambhia, P., & Pawar, S. (2022). An Extensive Analysis Between Different Language Models: GPT-3, BERT and MACAW. Research Square. https://doi.org/10.21203/rs.3.rs-2155616/v1
  • Garritty, C., Stevens, A., Gartlehner, G., King, V., & Kamel, C. (2016). Cochrane Rapid Reviews Methods Group to play a leading role in guiding the production of informed high-quality, timely research evidence syntheses. Systematic Reviews, 5(1),184. https://doi.org/s13643-016-0360-z
  • Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A research synthesis. National Comprehensive Center for Teacher Quality.
  • Goyal, T., Li, J.J., & Durrett, G. (2022). News Summarization and Evaluation in the Era of GPT-3. arXiv, arXiv:2209.12356. https://doi.org/10.48550/arXiv.2209.12356
  • Hamel, C., Michaud, A., Thuku, M., Skidmore, B., Stevens, A., Nussbaumer-Streit, B. et al. (2021). Defining rapid reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews. Journal of Clinical Epidemiology, 129, 74-85. https://doi.org/10.1016/j.jclinepi.2020.09.041
  • Jonsson, M., & Tholander, J. (2022). Cracking the code: Co-coding with AI in creative programming education. In Proceedings of the 14th Conference on Creativity and Cognition (5-14). https://doi.org/10.1145/3527927.3532801
  • Korngiebel, D.M., & Mooney, S.D. (2021). Considering the possibilities and pitfalls of Generative Pre-trained Transformer 3 (GPT-3) in healthcare delivery. NPJ Digital Medicine, 4(1), 1-3. https://doi.org/10.1038/s41746-021-00464-x
  • Lammerse, M., Hassan, S.Z., Sabet, S.S., Riegler, M.A., & Halvorsen, P. (2022). Human vs. GPT-3: The challenges of extracting emotions from child responses. In 14th International Conference on Quality of Multimedia Experience (QoMEX) (1-4). IEEE. https://doi.org/10.1109/qomex55416.2022.9900885
  • Lee, Y., Kim, T.S., Chang, M., & Kim, J. (2022). Interactive Children’s Story Rewriting Through Parent- Children Interaction. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing) (62-71). https://doi.org/10.18653/v1/2022.in2writing-1.9
  • MacNeil, S., Tran, A., Mogil, D., Bernstein, S., Ross, E., & Huang, Z. (2022). Generating diverse code explanations using the GPT-3 large language model. In Proceedings of the 2022 ACM Conference on International Computing Education Research (2, 37-39). https://doi.org/10.1145/3501709.3544280
  • Moore, S., Nguyen, H.A., Bier, N., Domadia, T., & Stamper, J. (2022). Assessing the Quality of Student- Generated Short Answer Questions Using GPT-3. In European Conference on Technology Enhanced Learning (243-257). Springer, Cham. https://doi.org/10.1007/978-3-031-16290-9_18
  • Naseri, H., Kafi, K., Skamene, S., Tolba, M., Faye, M.D., Ramia, P. et al. (2021). Development of a generalizable natural language processing pipeline to extract physician-reported pain from clinical reports: Generated using publicly-available datasets and tested on institutional clinical reports for cancer patients with bone metastases. Journal of Biomedical Informatics, 120, 103864. https://doi.org/10.1016/j.jbi.2021.103864
  • Nath, S., Marie, A., Ellershaw, S., Korot, E., & Keane, P.A. (2022). New meaning for NLP: the trials and tribulations of natural language processing with GPT-3 in ophthalmology. British Journal of Ophthalmology, 106(7). https://doi.org/10.1136/bjophthalmol-2022-321141
  • Nguyen, H.A., Bhat, S., Moore, S., Bier, N., & Stamper, J. (2022). Towards Generalized Methods for Automatic Question Generation in Educational Domains. In European Conference on Technology Enhanced Learning (272-284). Springer, Cham. https://doi.org/10.1007/978-3-031-16290-9_20
  • Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D. et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906. https://doi.org/10.31222/osf.io/jb4dx
  • Phillips, T., Saleh, A., Glazewski, K.D., Hmelo-Silver, C.E., Mott, B., & Lester, J.C. (2022). Exploring the use of GPT-3 as a tool for evaluating text-based collaborative discourse. Companion Proceedings of the 12th International Conference on Learning Analytics & Knowledge (LAK22), (1-3).
  • Ramnarain-Seetohul, V., Bassoo, V., & Rosunally, Y. (2022). Work-in-Progress: Computing Sentence Similarity for Short Texts using Transformer models. In IEEE Global Engineering Education Conference (EDUCON) (1765-1768). IEEE. https://doi.org/10.1109/educon52537.2022.9766649
  • Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022). Automatic generation of programming exercises and code explanations using large language models. In Proceedings of the 2022 ACM Conference on International Computing Education Research (1, 27-43). https://doi.org/10.1145/3501385.3543957
  • Stammbach, D., Antoniak, M., & Ash, E. (2022). Heroes, Villains, and Victims, and GPT-3: Automated Extraction of Character Roles Without Training Data. Proceedings of the 4th Workshop of Narrative Understanding (WNU2022) (47-56). https://doi.org/10.18653/v1/2022.wnu-1.6
  • Tack, A., & Piech, C. (2022). The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues. arXiv, arXiv:2205.07540. https://doi.org/10.48550/arXiv.2205.07540
  • Thiergart, J., Huber, S., & Übellacker, T. (2021). Understanding emails and drafting responses–an approach using GPT-3. ArXiv, arXiv:2102.03062. https://doi.org/10.48550/arXiv.2102.03062
  • Villarroel, J.J.G. (2021). Implicancia de la inteligencia artificial en las aulas virtuales para la educación superior. Orbis Tertius-UPAL, 5(10), 31-52.
  • Zapata, V.J.V., & Hoyos, D.M.R. (2023). Correspondencia artificial: exploraciones del ChatGPT y sus implicaciones en el quehacer académico. Folios, revista de la Facultad de Comunicaciones y Filología, 49, 36-41.
  • Zong, M., & Krishnamachari, B. (2022). Solving math word problems concerning systems of equations with GPT-3. Proceedings of the AAAI Conference on Artificial Intelligence, 37(13), 15972-15979. https://doi.org/10.1609/aaai.v37i13.26896
La source de données: Dialnet