Harnessing IRT and CAT for Next-Gen Educational Assessment in the Fifth Industrial Revolution
DOI:
https://doi.org/10.71291/jocatia.v2i.27Keywords:
Computer adaptive testing, item response theory, educational assessment, fifth industrial revolution, adaptive learning technologies, four parameter logistic modelAbstract
The Fifth Industrial Revolution (5IR) emphasises the harmonious collaboration between technology
and human-centred approaches, revolutionising educational assessment. Traditional standardised tests
often lack adaptability, leading to inefficiencies and biases. This study examines the effectiveness of
Item Response Theory (IRT) and Computer Adaptive Testing (CAT) in optimising assessment
processes by improving efficiency, precision, and reliability. Despite growing interest in adaptive
testing, research on its large-scale applicability in education remains limited, highlighting a critical gap
this study addresses. A simulation-based quantitative methodology was employed, utilising Monte
Carlo techniques to generate 1,000 examinee responses modelled through a four-parameter logistic
(4PL) IRT model. Two test conditions—fixed-length CAT and variable-length CAT—were
implemented to compare their effectiveness. Item selection followed the Maximum Fisher
Information (MFI) criterion, while Bayesian Maximum A Posteriori (MAP) was used for ability
estimation. The results reveal that variable-length CAT significantly reduces test length by
approximately 30% while maintaining high measurement precision. Adaptive testing demonstrated
lower estimation errors and higher reliability than fixed-length assessments, confirming its
effectiveness in modern educational evaluation. Additionally, item parameter analysis provided
insights into test design optimisation. These results underscore the advantages of integrating CAT in
large-scale assessments, particularly in enhancing fairness, personalisation, and engagement. The study
concludes that CAT is a viable alternative to traditional testing methods, aligning with the 5IR’s
emphasis on technological and human synergy in education. Future research should explore AI-driven
CAT enhancements to further refine assessment accuracy and accessibility.
Downloads
References
Abduraxmonov, A., & Ismailov, Y. N. (2022). Assessments in Education. https://doi.org/10.48550/arxiv.2208.05826
Adarkwah, M. A. (2024). New Trends of Digital and Intelligent Technology, In book: Envisioning the Future of Education Through Design, 1. DOI: 10.1007/978-981-97-0076-9_1
Aryu Networks. (2020). Technological innovations in education. Retrieved from https://aryunetworks.com/what-will-the-5th-industrial-revolution-look-like/
Ayanwale, M. A., & Ndlovu, M. (2022). Transition from computer-based testing of national benchmark tests to adaptive testing: Robust application of fourth industrial revolution tools. Cypriot Journal of Educational Sciences, 17(9), 3327-3343. DOI: 10.18844/cjes.v17i9.7124
Ayanwale, M. A., & Ndlovu, M. (2024). The feasibility of computerized adaptive testing of the national benchmark test: A simulation study. Journal of Pedagogical Research, 8(2), 95-112. DOI: 10.33902/JPR.202425210
Ayanwale, M. A., Chere-Masopha, J., Mochekele, M., & Morena, M. C. (2024). Implementing Computer Adaptive Testing for High-Stakes Assessment: A Shift for Examinations Council of Lesotho. International Journal of New Education, (14). DOI: 10.24310/ijne.14.2024.20487
Babcock, B., & Weiss, D. J. (2014). Termination criteria in computerized adaptive tests: Do variable-length CATs provide efficient and effective measurement? Journal of Computerized Adaptive Testing, 1(1–5), 1–18.
Bartlett, S. J., Duncan, T., Ruffing, V., Bingham, C. O., Deleon, E., Orbai, A.-M., & Clegg-Smith, K. (2015). Reliability and Validity of Selected PROMIS Measures in People with Rheumatoid Arthritis. PLOS ONE, 10(9), e0138543. https://doi.org/10.1371/journal.pone.0138543
Chalmers, R. P. (2016). Generating Adaptive and Non-Adaptive Test Interfaces for Multidimensional Item Response Theory Applications. Journal of Statistical Software, 71(5). https://doi.org/10.18637/jss.v071.i05
Díaz-Parra, O., Fuentes-Penna, A., Barrera-Cámara, R. A., Trejo-Macotela, F. R. ., Ramos-Fernández, J. C. ., Ruiz-Vanoye, J. A. ., Ochoa Zezzatti, A. ., & Rodríguez-Flores, J. . (2022). Smart Education and future trends. International Journal of Combinatorial Optimization Problems and Informatics, 13(1), 65–74. Retrieved from https://ijcopi.org/ojs/article/view/294
Dunya, B. A., & Wind, S. (2025). Exploring small item pools in CAT. Educational Measurement: Issues and Practice, 44(1), 20-36. https://doi.org/xxxx
Edwin Welch, R., & Frick, T. W. (1993). Computerized adaptive testing in instructional settings. Educational Technology Research and Development, 41(3), 47–62. https://doi.org/10.1007/bf02297357
Eggen, T. J., & Verschoor, A. J. (2006). Optimal testing with easy or difficult items in computerized adaptive testing. Applied Psychological Measurement, 30(5), 379–393. https://doi.org/10.1177/0146621606288890
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates.
Engelbrecht, A., Yitmen, I., & Almusaed, A. (2024). Industry 4.0 Transformation Towards Industry 5.0 Paradigm-Challenges, Opportunities and Practices: Challenges, Opportunities and Practices, 1-178. Doi:10.5772/intechopen.1001746
Fayers, P. M. (2007). Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment. Quality of Life Research, 16(S1), 187–194. https://doi.org/10.1007/s11136-007-9197-1
Gauri, P., & Van Eerden, J. (2019). What the Fifth Industrial Revolution is and why it matters. Europeansting. com, 16. Retrieved from https://europeansting.com/2019/05/16/what-the-fifth-industrial-revolution-is-and-why-it-matters/
Giugliano, G., Laudante, E., Formati, F., & Buono, M. (2023). Approaches and Technologies for the Human-Centered Industry 5.0. Proyecta 56, 3. https://doi.org/10.25267/p56-idj.2023.i3.05
Hambleton, R. K., & Swaminathan, H. (1985). Estimation of item and ability parameters. In Item response theory: Principles and applications (pp. 125–150). Springer. https://link.springer.com/book/10.1007/978-94-017-1988-9
Huang, Y.-M., Lin, Y.-T., & Cheng, S.-C. (2008). An adaptive testing system for supporting versatile educational assessment. Computers & Education, 52(1), 53–67. https://doi.org/10.1016/j.compedu.2008.06.007
Huda, A., Firdaus, F., Irfan, D., Hendriyani, Y., Almasri, A., & Sukmawati, M. (2024). Optimizing Educational Assessment: The Practicality of Computer Adaptive Testing (CAT) with an Item Response Theory (IRT) Approach. JOIV: International Journal on Informatics Visualization, 8(1), 473-480.
Kustiyahningsih, Y., & Cahyani, A. D. (2013). Computerized adaptive test based on item response theory in e-learning system. International Journal of Computer Applications, 81(6).
Moore, T. M., Reise, S. P., Scott, J. C., Ruparel, K., Gur, R. E., Jackson, C. T., Gur, R. C., Savitt, A. P., & Port, A. M. (2015). Development of an abbreviated form of the Penn Line Orientation Test using large samples and computerized adaptive test simulation. Psychological Assessment, 27(3), 955–964. https://doi.org/10.1037/pas0000102
Msayer, M., Aoula, E. S., & Bouihi, B. (2024). Artificial intelligence in computerized adaptive testing to assess the cognitive performance of students: A Systematic Review. In 2024 International Conference on Intelligent Systems and Computer Vision (ISCV) (pp. 1-8). IEEE.
Noble, S. M., Mende, M., Grewal, D., & Parasuraman, A. (2022). The Fifth Industrial Revolution: How harmonious human–machine collaboration is triggering a retail and service [R]evolution. Journal of Retailing, 98(2), 199–208. https://doi.org/10.1016/j.jretai.2022.04.003
Petersen, M. A., Helbostad, J., Chie, W.-C., Groenvold, M., Singer, S., Holzner, B., Velikova, G., Kaasa, S., Fayers, P., Costantini, A., Aaronson, N. K., Conroy, T., & Young, T. (2010). Development of computerized adaptive testing (CAT) for the EORTC QLQ-C30 physical functioning dimension. Quality of Life Research, 20(4), 479–490. https://doi.org/10.1007/s11136-010-9770-x
Priya, B., Sharma, V., Awotunde, J. B., & Adeniyi, A. E. (2025). Artificial Intelligence in Industry 5.0: Transforming Manufacturing through Machine Learning and Robotics in Collaborative Age. Computational Intelligence in Industry 4.0 and 5.0 Applications, 61-100. https://doi.org/10.1201/9781003581963
Ridwan, W., Wiranto, I., & Dako, R. D. R. D. (2021). Computerized Adaptive Test based on Sugeno Fuzzy Inference System. 1098(3), 032077. https://doi.org/10.1088/1757-899X/1098/3/032077
Siswi, N. R. T., Purwanto, M. G., Kusumastuti, M. N., & Sanjaya, Y. (2023). Innovation for measuring students’ metacognitive abilities through project-based learning. Jurnal Inovasi Dan Teknologi Pembelajaran, 10(1), 71. https://doi.org/10.17977/um031v10i12023p071
Thompson, G. (2017). Computer adaptive testing, big data and algorithmic approaches to education. British journal of sociology of education, 38(6), 827-840.
Thompson, N. A., & Weiss, D. J. (2011). A framework for the development of computerized adaptive tests. Practical Assessment, Research and Evaluation, 16(1), 1–9.
Tóth, A.-P., Nagy, L., Kennedy, R., Abonyi, J., & Ruppert, T. (2023). The human-centric Industry 5.0 collaboration architecture. MethodsX, 11, 102260. https://doi.org/10.1016/j.mex.2023.102260
Weiss, D. J. (1985). Adaptive testing by computer. Journal of Consulting and Clinical Psychology, 53(6), 774–789. https://doi.org/10.1037//0022-006X.53.6.774
Weiss, D. J. (2011). Better data from better measurements using computerized adaptive testing. Journal of Methods and Measurement in the Social Sciences, 2(1), 1–27. https://doi.org/10.2458/jmm.v2i1.12351
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Kunle Ayanwale
This work is licensed under a Creative Commons Attribution 4.0 International License.
