Effect of Using Assist Devices on Exam Completion Questions Among Thai College Students

Abstract

Action research was undertaken to begin to explore the possibility of improving second-language Thai college student performance on completion questions by using bolded and underscored words in test item stems, called ‘assist devices’. This intervention was designed to focus student attention on key terms. Twenty-one students, in an intact class, were exposed to two course exams or testing trials, which included a total of 30 completion question, 15 of which were randomly selected for treatment with assist devices, and thus used as the experimental condition. The 15 remaining, randomly selected questions, were not treated and thus used as the control condition. The results of the first trial were invalidated due to an unforeseen error in the online exam. Thus, only the results of the second trial were analyzed using One-way Repeated Measures ANOVA, which indicated a significant treatment effect. While the results were encouraging, any substantive conclusions on the use of assist devices on completion test items with second-language Thai college students will have to wait for further longitudinal testing to establish a possible pattern of correlation. While a strong correlation was evident, a valid cause and effect relationship could not be established due to the measurement error introduced by unknown item discrimination data, which was not explored in this pilot testing phase. The use of randomly selected, treated and untreated questions in the same trial, however, was thought to be an innovative design strategy for avoiding the need of random assignment of subjects into experimental and control groups, thereby making a valuable contribution to the action research paradigm.

Keywords: Learner-centered assessment, Completion questions, Thai college students, Second-language students