Detection of Sex-Related Differential Item Functioning in Raven’s Standard Progressive Matrices Test Using the Mantel-Haenszel Method

Abstract

This study aimed to determine the presence of sex-related differential item functioning (DIF) in Raven’s Standard Progressive Matrices (SPM) Test. The research design was a comparative study, where boys formed the focal group and girls formed the reference group. The software used was SPSS-v26, and binary data of focal and reference groups were analyzed using jMetrik software to detect DIF according to the Mantel-Haenszel method. A total of 1032 students (49.6% boys and 50.4% girls), 570 from intermediate school and 462 from secondary school, were selected from 24 schools using a stratified random-sampling procedure. The statistical analyses showed that the Raven’s Standard Progressive Matrices (SPM) Test was one-dimensional. The results showed that, of five moderate DIF items, four items favored the focal group (boys), and one item favored the reference group (girls). The results also showed two large DIF items; the direction of the one item favored the reference group (girls), and the direction of the second item favor the focal group (boys). The findings showed that there were several unbiased items, but some were clearly biased against female performance. According to these findings, we recommend reanalyzing the data using methods depending on the item response theory, such as the logistic regression, simultaneous item bias test (SIB) or IRT-likelihood ratio (IRT-LR) methods to confirm the results seen here.

Keywords: Differential Item Functioning (DIF), Bias Item, Reference Group, Focal Group

DOI: 10.7176/JEP/11-29-10

Publication date:October 31^st 2020