How to grade items for a question bank and rank tests based on student performance

Thomas, Puthiaparampil and Md Mizanur, Rahman and Henry Rantai, Gudum and Brohi, Imam Bux and Fong, Isabel Lim and Rosalia, Saimon (2020) How to grade items for a question bank and rank tests based on student performance. MedEdPublish, 2020, Volume 9, Issue 1, Volume (Issue1). pp. 1-7. ISSN ISBN 2312_7996

PDF Thomas.pdf Download (132kB)

Abstract

Introduction: Not utilising post-test psychometric analyses of questions, and not maintaining a question bank seemed to adversely affect the quality of tests and increase the workload of academicians, as they are required to write fresh questions for all examinations. The literature review did not reveal any gold standard method for creating a question bank. This study has formulated criteria for recruiting multiple choice questions for a question bank and introduced a formula to rank whole tests. Methods: We collected used question papers of multiple true-false questions (MCQ) and one best answer questions (BAQ) and got two experienced academicians to scrutinize them and identify items with flaws. The flawless items were counted in each test and their test performance index (TPi) determined. The psychometric item analysis reports of the tests were analysed to enlist bankable items. The TPi of the tests were also calculated by this method. The TPi derived by expert opinions were compared with those obtained by objective criteria using the Pearson Correlation Coefficient and the Spearman’s rho. Results: Judgements by two experts showed a positive correlation, so also expert judgements against the objective formulae. Omission rates in MTF items showed a highly significant negative correlation with difficulty index, falling short of a perfect -1, which supported including omission index in their triple formula. The mean number of functioning distractors (FD) per BAQ item was1.87 (SD 1.14), which supported ≥2 FD per item in their triple formula. Conclusion: Expert judgments in question vetting is essential. However, objective post-test scrutiny of items using difficulty and discrimination indexes enhanced with omission index and distractor efficiency to recruit items for question banks are required. Test Performance Index will be a useful metric to rank the tests.

Actions (For repository members only: login required)

View Item