A Transformer-Based Cognitive Classification Framework for Integrating Bloom’s Taxonomy into Adaptive Testing Systems

Abstract

This study proposes a transformer-based machine learning framework for automated classification of test items according to Bloom’s taxonomy of cognitive processes, aiming to enhance cognitive alignment in adaptive testing systems. Manual classification of assessment items is often time-intensive and susceptible to subjectivity, posing challenges for large-scale and real-time adaptive testing environments. To address this limitation, the study employs Natural Language Processing techniques, specifically fine-tuning a BERT-based model, to classify Economics test items into six cognitive levels. A balanced dataset comprising 3,000 expert-annotated items (Cohen’s κ = 0.87) was used, with stratified training, validation, and testing procedures to ensure robustness. Model performance was evaluated using accuracy, precision, recall, and F1-score, alongside stratified 5-fold cross-validation and comparative analysis with Logistic Regression and Support Vector Machine baselines. The proposed model achieved an accuracy of 97.7% on the held-out test set and a cross-validated mean accuracy of 96.7% (SD = 0.5), substantially outperforming baseline models. While classification performance was consistently high across most categories, relatively lower recall for the “Understand” level suggests inherent semantic overlap between adjacent cognitive processes. The findings demonstrate the potential of transformer-based models to capture nuanced cognitive demand in assessment items. Importantly, the study conceptualises this classification framework as a complementary layer to Item Response Theory-based adaptive testing systems, enabling item selection that accounts for both psychometric properties and cognitive complexity. The study contributes a scalable approach to improving the consistency and pedagogical alignment of item classification, while highlighting the need for further validation using diverse and imbalanced datasets and for integration into operational adaptive testing environments.