The Construction and Validation of a Q-matrix for Cognitive Diagnostic Analysis: The Case of the Reading Comprehension Section of the IAUEPT

Document Type : Special Issue

Authors

1 English Department, Islamic Azad University, Mashhad Branch, Iran.

2 Ferdowsi University of Mashhad

3 Department of English Language and Literature, Ferdowsi University of Mashhad, Mashhad.

4 English Department, Islamic Azad University of Mashhad, Mashhad, Iran.

Abstract

Cognitive diagnostic models (CDMs) have received sustained attention in educational settings because they can be used to operationalize formative assessment to provide diagnostic feedback and inform instruction. A large number of CDMs have been developed over the past few years. An important component of all CDMs is a Q-matrix that specifies a particular hypothesis about the relationship between each test item and its required attributes. The purpose of this study was to construct and validate a Q-matrix for the reading comprehension section of the Islamic Azad University English Proficiency Test (IAUEPT), as an advanced English placement test designed to measure language ability of Ph.D. candidates who tend to pursue their studies in the IAU. To achieve this, using item responses of 1152 candidates to twenty items of the reading section of the test, an initial Q-matrix was constructed based on theories and models of second/foreign language (L2) reading comprehension, previous applications of CDMs on L2 reading comprehension, and brainstorming and consensus of five content experts. Then, the initial Q-matrix was empirically validated using the method proposed by de la Torre and Chiu (2016) and checking mesa plots, and heatmap plot. Five attributes were derived for the reading comprehension section: vocabulary, grammar, making an inference, understanding specific information, and identifying explicit information. Finally, the analysis of the Generalized Deterministic Inputs, Noisy “and” Gate (GDINA) regarding absolute fit at item- and test-level as well as three residual-based statistics showed the accuracy of the Q-matrix and a perfect model-data fit.

Keywords

International Journal of Language Testing
Volume 13, Special Issue
March 2023
Pages 31-53

Files

Share

How to cite

Statistics