Home/Journal of Computer Adaptive Testing in Africa/Article Record
Peer-Reviewed Article September 7, 2026

An Integrated Multidimensional Item Response Theory Validation Framework for a Chemistry Achievement Test

Are you an author of this work?Link to your sovereign AfriScholar profile
Published in: Journal of Computer Adaptive Testing in Africa
Publisher: Association for Computer Adaptive Testing in Africa • South Africa • 2790-4393 • 2790-4407
DOI: https://doi.org/10.71291/rwykgq70

Abstract

The validity of chemistry achievement assessments remains a concern due to the multidimensional nature of the construct and the continued reliance on fragmented validation approaches. This study addresses this gap by adopting an integrated psychometric validation framework grounded in multidimensional Item Response Theory (MIRT). A 150-item dichotomously scored Chemistry Achievement Test (CAT) was administered to 741 senior secondary school students. Dimensionality was examined using Stout’s DETECT procedure, revealing weak multidimensionality characterised by a dominant general factor and three secondary dimensions, thereby supporting a four-dimensional structure. Competing MIRT models (M1PL, M2PL, M3PL, and M4PL) were evaluated using likelihood-ratio tests and information criteria. The multidimensional three-parameter logistic (M3PL) model demonstrated superior fit and parsimony, underscoring the importance of modelling both item discrimination and guessing in multiple-choice contexts. Local item independence, assessed via Yen’s Q3 statistic, showed no evidence of problematic residual correlations. Item parameter screening indicated that 84% of items met established criteria, leading to the removal of poorly functioning items. Measurement invariance was evaluated through IRT likelihood-ratio Differential Item Functioning (DIF) analysis across sex and school type, resulting in a final set of 112 invariant items. The findings provide strong evidence for the structural validity, reliability, and fairness of the CAT. Beyond its empirical contribution, the study advances an integrated validation framework that systematically combines dimensionality assessment, model evaluation, local independence diagnostics, and invariance testing, offering a robust template for developing equitable and psychometrically sound science assessments.

Download Full Text PDFCrossref RecordCrossref XML

Verified Citation Velocity

Recorded scholarly citations in African and international peer-reviewed literature

0
Total Citations

Citations are continuously tracked and harvested across continental OJS endpoints and OpenAlex feeds.