Application of Generalizability Theory in Estimating Variance Components in Senior Secondary School Certificate Examination Physics Essay Questions

Abstract

This study applied Generalizability Theory to estimate variance components in Senior Secondary School Certificate Examination (SSSCE) Physics essay questions using a two-facet fully crossed G-study and D-study design. The population comprised 1,767 Senior Secondary Three (SS3) students in Obio/Akpor, Rivers State. A multistage sampling technique selected 10 out of 25 schools, and 150 students were sampled using the Taro Yamane formula. The 2019 WASSCE Physics essay questions, rated on a 10-point marking scheme, were administered. Data analysis using SPSS under the Variance Components MINQUE method revealed that students contributed the most to measurement error (∂²si = 1.030), followed by items (∂²i = 0.851) and raters (∂²r = 0.016). Interaction effects were highest for student-item (∂²si = 2.957), while student-item-rater (∂²sir = -2.868) was negligible. Reliability estimates indicated a universe score variance of 1.030, relative error variance of 2.981, absolute error variance of 4.878, a G-study coefficient of 0.26, and an index of dependability of 0.17, suggesting question difficulty. The findings highlight inconsistencies in student responses across items, which affect reliability. It is recommended that the West African Examinations Council (WAEC) establish a psychometric unit to validate essay questions, ensuring higher reliability and dependability before administration.