Distractor Analysis in Multiple-Choice Items Using the Rasch Model

Bakytbekovich, Omarov Nazarbek; Mohammed, Aisha; Alghurabi, Ammar Muhi Khleel; Alallo, Hajir Mahmood Ibrahim; Ali, Yusra Mohammed; Hassan, Aalaa Yaseen; Demeuova, Lyazat; Viktorovna, Shvedova Irina; Nazym, Bekenova; Afif, Al Khateeb Nashaat Sultan

doi:10.22034/ijlt.2023.387942.1236

Distractor Analysis in Multiple-Choice Items Using the Rasch Model

Document Type : Special Issue

Authors

Omarov Nazarbek Bakytbekovich ¹

Aisha Mohammed ²

Ammar Muhi Khleel Alghurabi ³

Hajir Mahmood Ibrahim Alallo ⁴

Yusra Mohammed Ali ⁵

Aalaa Yaseen Hassan ⁶

Lyazat Demeuova ⁷

Shvedova Irina Viktorovna ⁸

Bekenova Nazym ⁹

Al Khateeb Nashaat Sultan Afif ⁸

¹ Non-profit Joint Stock Company Semey Medical University, Department of Hospital Surgery, 071400-Semey City, Abai Street, 103, Republic of the Kazakhstan.

² College of Education, Al-Farahidi University, Baghdad, Iraq

³ College of Education, The Islamic University in Najaf, Iraq

⁴ English Department, Ahl-Al-Bayt University, Kerbala, Iraq

⁵ Department of Medical Laboratory Technics, Al-Zahrawi University College, Karbala, Iraq

⁶ Al-Nisour University College, Baghdad, Iraq

⁷ Institute of Natural Sciences and Geography, Abai Kazakh National Pedagogical University, Almaty, Kazakhstan

⁸ People’s Friendship University of Russia, Moscow, Russia

⁹ Candidate of Biological Sciences, Docent, Institute of Natural Sciences and Geography, Abai Kazakh National Pedagogical University, Almaty, Kazakhstan.

10.22034/ijlt.2023.387942.1236

Abstract

Multiple-choice (MC) item format is commonly used in educational assessments due to its economy and effectiveness across a variety of content domains. However, numerous studies have examined the quality of MC items in high-stakes and higher education assessments and found many flawed items, especially in terms of distractors. These faulty items lead to misleading insights about the performance of students and the final decisions. The analysis of distractors is typically conducted in educational assessments with multiple-choice items to ensure high quality items are used as the basis of inference. Item response theory (IRT) and Rasch models have received little attention for analyzing distractors. For that reason, the purpose of the present study was to apply the Rasch model, to a grammar test to analyze items’ distractors of the test. To achieve this, the current study investigated the quality of 10 instructor-written MC grammar items used in an undergraduate final exam, using the items responses of 310 English as a foreign language (EFL) students who had taken part in an advanced grammar course. The results showed the acceptable fit to the Rasch model and high reliability. Malfunctioning distractors were identified.

Keywords

Distractor analysis

Item response theory

Multiple-choice items

Rasch model