Content Accuracy and Learning Impact of Generative Artificial Intelligence as a Virtual Tutor for Organic Chemistry among Science Education Students in Tertiary Institutions in Minna Metropolis

Abstract

This study investigated the content accuracy and learning impact of Generative Artificial Intelligence (GenAI) as a virtual tutor for Organic Chemistry among Science Education students in tertiary institutions in Minna Metropolis, Niger State. The study was motivated by the increasing use of GenAI tools for academic learning and the need to determine the accuracy of AI-generated Chemistry content and its effectiveness in improving students' learning outcomes. A quasi-experimental research design was adopted for the study. The population comprised Chemistry Education students of the Federal University of Technology, Minna, and the Niger State College of Education, Minna, made up of 65 Level 2 Chemistry Education students of the Federal University of Technology, Minna, and 53 NCE II Chemistry Education students of the Niger State College of Education, Minna, giving a total population of 118 students, all of whom were studied. Data were collected using an Organic Chemistry Achievement Test and a researcher-developed instrument for assessing the content accuracy of GenAI-generated responses. To establish the reliability of the instruments, a pilot test was conducted among 32 students offering Chemistry at the Niger State College of Health Technology, Minna, who were excluded from the main study. The instruments using Cronbach's Alpha, yielded a reliability coefficient of 0.74, indicating satisfactory internal consistency. Data collected were analysed using mean and standard deviation to answer the research questions, while appropriate inferential statistics were used to test the null hypotheses at the 0.05 level of significance. The findings revealed that GenAI demonstrated a high level of content accuracy, with an overall mean score of 3.48, indicating that the generated responses were generally consistent with accepted Organic Chemistry knowledge. The findings also showed that students taught with GenAI as a virtual tutor achieved a higher post-test mean score of 72.64 compared with 61.38 for students taught using the conventional instructional approach, representing a mean difference of 11.26 points. The difference was statistically significant at the 0.05 level of significance. The study concluded that GenAI has considerable potential as a virtual tutoring tool for supporting the teaching and learning of Organic Chemistry, although its outputs require appropriate academic verification. The study recommended guided and responsible integration of GenAI into Chemistry Education to complement conventional instructional practices.