Classification Accuracy Effects of Q-Matrix Validation and Sample Size in DINA and G-DINA Models

Abstract

This article studies the extend of change in latent classes, relating to students, which were calculated using DINA and Generalized-DINA(G-DINA) Models under different distributions and sub-sample sizes which were calculated using DINA and Generalized-DINA(G-DINA) Models. Main focus of this study is the results of practical application rather than statistical structure of Cognitive Diagnostic Models (CDM). The attribute the individuals master that take the test in CDM are determined categorically. For this reason, both the fit of Q matrix with data and the effect of sample size are searched in modelling the students’ category. In the case of low model data fit and inadequate sample size, the findings of this research will be a guide in  how the decisions change about which attribute a student master or not. To this end, a mathematic test consisted of 18 multiple choice questions taken by a group of 1000 examinee was employed. Analyses were carried out using 5 different Q-Matrices, for which relations between test items and attributes were determined by experts, and latent classes determined by both DINA and G-DINA models were compared. Comparisons were made with a view to accuracy of values between classes associated with examinees in different sample sizes drawn from the same population and values obtained for population. Thus, for both models, whether they lead to independent results from the samples was tested for sample sizes of 30, 50, 100, 200 and 400 and effects of Q matrix- data fit on analysis results were determined. Results of analysis showed Q-matrix – data fit had significant impact on decisions about students for both models.

Keywords: Cognitive Diagnostic Models, DINA model, G-DINA model, Q Matrix