International Journal of Language Testing

International Journal of Language Testing

Retrofitting Non-diagnostic Grammar Assessment: Application of the G-DINA Model to a High Stakes Grammar Test

Document Type : Original Research Article

Authors
1 Faculty of Foreign Languages and Literatures, University of Tehran.
2 Professor of Applied Linguistics, Faculty of Foreign Languages and Literatures, University of Tehran. and Sub-President for Education and Research, Aras International Campus, University of Tehran
3 Associate Professor of Psychometrics and Behavioral and Educational Data Science at the Faculty of Psychology and Education, Kharazmi University, Tehran
4 English Department, Faculty of Foreign Languages, University of Science and Technology
Abstract
The purpose of this study was to compare the functioning of five Cognitive Diagnostic Models (CDMs) to identify the best-fitting CDM that can better explain the interaction underlying the attributes of the grammar section of the University of Tehran English Proficiency Test (UTEPT). To this end, a Q-matrix representing the key cognitive abilities required for the grammar section was developed. Expert input identified six essential grammar attributes: Agreements, Clauses, Lexical Knowledge, Connectors, Tense Recognition, and Voice Awareness. Data from 810 examinees (268 males and 542 females) aged 24 to 48, including those who took the test in 2022, were analyzed. The five CDMs were initially compared in terms of relative and absolute fit statistics at the test and item level to choose the best model. It was found that the Generalized Deterministic Inputs Noisy “And” gate (G-DINA) model outperformed the restrictive models; thus, it was chosen for the second phase of the study. Regarding the second purpose of the study, the G-DINA model was used to identify the strengths and weaknesses of the test takers. The results revealed that understanding Tense Recognition was the most challenging attribute for the examinees, while Lexical Knowledge was the easiest attribute. These findings highlight the need for instructional strategies focusing on enhancing Tense Recognition and integrating teaching approaches that address the interdependent nature of grammar attributes.
Keywords