Evaluation of the Effectiveness of Artificial Intelligence in Enhancing Students' Listening, Grammar, Vocabulary, Motivation, and Self-Regulation Skills Compared to Human Interaction

Abstract

The use of modern technologies in education, particularly artificial intelligence (AI), has become a key focus in educational research. This study aims to evaluate the effectiveness of AI in enhancing students' listening, grammar, vocabulary, motivation, and self-regulation skills compared to human interaction. In this study, 40 students aged 14 to 18 were purposefully divided into two groups of 20. The first group received instruction through AI (Duolingo), while the second group interacted with a human partner. The training program lasted for twelve weeks. Listening, grammar, and vocabulary skills were assessed using pre-tests and post-tests, with data analyzed through the Wilcoxon test. A nonparametric covariance test was used to compare the two groups. Motivation and self-regulation were assessed via a questionnaire after the intervention, with differences between the groups analyzed using the Mann-Whitney test. The results indicated that both groups showed significant improvement in listening, grammar, and vocabulary skills compared to their baseline performance (P-value = 0). However, no statistically significant difference was observed between the two groups (P-value = 0). Additionally, students who interacted with a human partner demonstrated higher motivation levels than those trained with AI (P-value = 0). Conversely, students who received AI-based instruction exhibited greater self-regulation skills (P-value = 0). This study suggests that AI can be as effective as human interaction in improving listening, grammar, and vocabulary skills. However, its impact on motivation is lower than human interaction, whereas it is more effective in enhancing students' self-regulation abilities. These findings can be valuable in designing technology-based educational programs.

Keywords: Artificial Intelligence, Human Interaction, Listening Skills, Grammar and Vocabulary Skills, Motivation, Self-Regulation

DOI: 10.7176/JEP/16-9-16

Publication date:August 31^st 2025