Application of Generalizability Theory in Estimating Variance Components in Senior Secondary School Certificate Examination Physics Essay Questions

Authors

  • Nwuchegbuo Iheanyichukwu University of Education Port Harcourt
  • Richard Iheanyichukwu Ogunka University of Port Harcourt

DOI:

https://doi.org/10.71291/jocatia.v1i.3

Keywords:

generalizability theory, essay questions, variance components

Abstract

This study applied Generalizability Theory to estimate variance components in Senior Secondary School Certificate Examination (SSSCE) Physics essay questions using a two-facet fully crossed G-study and D-study design. The population comprised 1,767 Senior Secondary Three (SS3) students in Obio/Akpor, Rivers State. A multistage sampling technique selected 10 out of 25 schools, and 150 students were sampled using the Taro Yamane formula. The 2019 WASSCE Physics essay questions, rated on a 10-point marking scheme, were administered. Data analysis using SPSS under the Variance Components MINQUE method revealed that students contributed the most to measurement error (?²si = 1.030), followed by items (?²i = 0.851) and raters (?²r = 0.016). Interaction effects were highest for student-item (?²si = 2.957), while student-item-rater (?²sir = -2.868) was negligible. Reliability estimates indicated a universe score variance of 1.030, relative error variance of 2.981, absolute error variance of 4.878, a G-study coefficient of 0.26, and an index of dependability of 0.17, suggesting question difficulty. The findings highlight inconsistencies in student responses across items, which affect reliability. It is recommended that the West African Examinations Council (WAEC) establish a psychometric unit to validate essay questions, ensuring higher reliability and dependability before administration.

Downloads

Download data is not yet available.

References

Adesoji, F. A. (2008). Students Ability Levels and Effectiveness of Problem—Solving Instructional Strategy. Journal of Social Physics, 17, 5-8.

Akinbobola, A. O., & Afolabi, F. (2010). Analysis of Science process skills in West African senior secondary school certificate Physics practical examinations in Nigeria. American-Eurasian Journal of Scientific Research, 5, 234-240.

Anatol, T. and Hariharan, S. (2009).Reliability of evaluation of student’s awareness to essay-type questions.West India medical journal, 58 (1), 13-16.

Asuru, V. A. (2015). Measurement and evaluation in education and psychology (2nd ed). Port Harcourt: Pearl Publishers International Ltd.

Atilla, Y (2015). Estimation of generalizability coefficient: an application with different program.

Atilla, Y. (2012). Dependability of job performance ratings according to generalizability theory. Journal of Education and Science, 163(37), 157 – 348.

Brennan, R.L (2001) Generalizability Theory: Statistics for Social Science and Public Policy. Springer- Verlag Berlin Heidlberg. New York.

Cannings, R., Hawthrone, K., Hood, K. and Houston, H., (2005). Putting double marking to the test.A framework to assess if it worth the trouble.Medical education, 39 (3) 299-308.

Faleye, B. A. (2019). Generalizability theory: A workshop presented at the annual conference Zuba, Abuja 2018. Association of educational researchers and evaluators of Nigeria (ASEREN).

Heitman, R.J; Kovaleski, J.E & Pugh, S.F. (2009).Application of generalizability theory in estimating the Reliability of Ankle- Complex Laxity Measurement.Journal of Athletical Teaching. https://www.ncbi,nih.gov

Ikeh, E.F. &Madu, B.C. (2018).Application of generalizability theory in estimating multiple sources of variation in economics essay test. A paper presented at National Annual Conference (Abuja 2018). Association of Educational Researcher and Evaluators of Nigeria.

Johnson, S. Dulanay, C. & Bank, K. (2000).Measurement error. Retrieved from http:/www.wcpss.net/evaluation research/reports/2000/mment_error.pdf

Kim, S. C & Wilson, M. (2009). A comparative analysis of the ratings in performance assessment using generalizability theory. Journal of Allied Measurement, 10(4) 408 – 423.

Kpolovie, P. J (2010).Advanced Research Methods, Owerri Springfield Publisher ltd.

Linn, R. L., Miller, M. D. and Ground, N. E., (2005). Measurement and assessment in teaching (9thed) upper saddle river N. J: Pearson education, inc.

Ogunka, R. I. and Orluwene, G. W. (2020). Application of generalizability theory in estimating variance components in national examinations council problem solving questions in mathematics.

Oladele, J. I. (2022). Estimating Measurement Error in University Students’ Work Experience Programme (SWEP) Assessment using Generalizability Theory: Implications for Transdisciplinarity in Engineering Training. Transdisciplinary Journal of Engineering & Science, 13. https://doi.org/10.22545/2022/00185

Olatoye, R. A., &Afuwape, M. O. (2003). Test anxiety as a determinant of examination misdemeanor among some Nigerian secondary school students. Ibadan Journal of Educational Studies, 3(182), 32-39.

Orluwene, G. W., (2012). Introduction to test theory and development process. Port Harcourt: Chris- Ron integrated services.

Preuss, R.A. (2003). Using generalizability theory to Develop Clinical Assessment Protocols. Physical Therapy, vol. 93 issue 4, pp 562-Examples of reference entries:

Downloads

Published

2022-06-29

How to Cite

Iheanyichukwu, N., & Ogunka, R. I. (2022). Application of Generalizability Theory in Estimating Variance Components in Senior Secondary School Certificate Examination Physics Essay Questions. Journal of Computer Adaptive Testing in Africa, 1, 55–66. https://doi.org/10.71291/jocatia.v1i.3