VALIDITY AND RELIABILITY ANALYSIS OF A CRITICAL THIKING SKILLS TEST ON ELECTRODYNAMICS USING THE RASCH MODEL AMONG INDONESIAN PHYSICS EDUCATION STUDENTS

M. Furqon, Parlindungan Sinaga, Liliasari Liliasari, Lala Septem Riza, Dadi Rusdiana

Abstract


This research aims to empirically validate an 8-item essay instrument designed to measure students' critical thinking skills in electrodynamics using the Rasch measurement model. While critical thinking is essential in physics, precise diagnostic tools for complex topics like alternating current and magnetic induction remain scarce. The study employed a quantitative descriptive method, administering the test to 30 university physics education students. The instrument, developed based on Ennis’s critical thinking indicators, utilized a strict dichotomous (0-1) scoring rubric to eliminate subjective rater bias. Data were analyzed using Winsteps software to determine item fit and reliability. Results indicated excellent construct validity; all eight items successfully met the Rasch criteria for Outfit MNSQ, Outfit Z-Std, and Point Measure Correlation. Furthermore, the instrument demonstrated robust reliability, with an Item Reliability of 0.85 and a Person Reliability of 0.84. The analysis also revealed strong strata separation, proving the test's capability to accurately categorize students into three distinct cognitive levels. In conclusion, this validated diagnostic tool is reliable, sensitive, and perfectly suited for integration into electrodynamics learning.

Keywords


validity; reliability; critical thinking; electrodynamics; Rasch model

Full Text:

PDF

References


Adams, W. K., & Wieman, C. E. (2011). Development and validation of instruments to measure learning of expert-like thinking. International Journal of Science Education, 33(9), 1289–1312. https://doi.org/10.1080/09500693.2010.512369

Bao, L., & Koenig, K. (2019). Physics education research for 21st century learning. Disciplinary and Interdisciplinary Science Education Research, 1(1), 1–12. https://doi.org/10.1186/s43031-019-0007-8

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge. https://doi.org/10.4324/9781315814698

Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4), rm4. https://doi.org/10.1187/cbe.16-04-0148

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4

Care, E., Griffin, P., & Wilson, M. (Eds.). (2018). Assessment and teaching of 21st century skills: Research and applications. Springer. https://doi.org/10.1007/978-3-319-65368-6

Chabay, R., & Sherwood, B. (2015). Matter and interactions (4th ed.). John Wiley & Sons.

Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Cengage Learning.

Ding, L., & Beichner, R. (2009). Approaches to data analysis of multiple-choice questions. Physical Review Special Topics-Physics Education Research, 5(2), 020103. https://doi.org/10.1103/PhysRevSTPER.5.020103

Docktor, J. L., & Mestre, J. P. (2014). Synthesis of discipline-based education research in physics. Physical Review Special Topics-Physics Education Research, 10(2), 020119. https://doi.org/10.1103/PhysRevSTPER.10.020119

Dori, Y. J., & Belcher, J. (2005). How does technology-enabled active learning affect undergraduate students' understanding of electromagnetism concepts? Journal of the Learning Sciences, 14(2), 243–279. https://doi.org/10.1207/s15327809jls1402_3

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates.

Engelhard, G. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. Routledge. https://doi.org/10.4324/9780203073636

Ennis, R. H. (2011). The nature of critical thinking: An outline of critical thinking dispositions and abilities. Inquiry: Critical Thinking Across the Disciplines, 26(1), 4–18. https://doi.org/10.5840/inquiryctnews20112613

Facione, P. A. (2011). Critical thinking: What it is and why it counts. Insight Assessment.

Fadhilah, N., Wati, M., & Annur, S. (2020). Development of physics test instruments to measure critical thinking skills on electricity and magnetism. Journal of Physics: Conference Series, 1491(1), 012046. https://doi.org/10.1088/1742-6596/1491/1/012046

Finkelstein, N. D., Adams, W. K., Keller, C. J., Kohl, P. B., Perkins, K. K., Podolefsky, N. S., & Reid, S. (2005). When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics-Physics Education Research, 1(1), 010103. https://doi.org/10.1103/PhysRevSTPER.1.010103

Forawi, S. A. (2016). Standard-based science education and critical thinking. Thinking Skills and Creativity, 20, 52–62. https://doi.org/10.1016/j.tsc.2016.02.005

Griffin, P., McGaw, B., & Care, E. (Eds.). (2012). Assessment and teaching of 21st century skills. Springer. https://doi.org/10.1007/978-94-007-2324-5

Guisasola, J., Almudí, J. M., & Zubimendi, J. L. (2004). Difficulties in learning the introductory magnetic field theory in the first years of university. Science Education, 88(3), 443–464. https://doi.org/10.1002/sce.10119

Gunawan, G., Harjono, A., Sahidu, H., & Nisrina, N. (2017). Improving students' critical thinking skills using physics virtual laboratories. Jurnal Pendidikan Fisika Indonesia, 13(2), 99–105. https://doi.org/10.15294/jpfi.v13i2.8105

Halpern, D. F. (2014). Thought and knowledge: An introduction to critical thinking (5th ed.). Psychology Press. https://doi.org/10.4324/9781315805672

Hambleton, R. K., & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38–47. https://doi.org/10.1111/j.1745-3992.1993.tb00543.x

Khasanah, N., Subali, B., & Habibi. (2017). The profile of students’ critical thinking skills on electricity and magnetism. Journal of Physics: Conference Series, 909, 012030. https://doi.org/10.1088/1742-6596/909/1/012030

Kuo, E., Hull, M. M., Gupta, A., & Elby, A. (2013). How students blend conceptual and formal mathematical reasoning in solving physics problems. Science Education, 97(1), 32–57. https://doi.org/10.1002/sce.21043

Linacre, J. M. (2012). Winsteps® (Version 3.75.0) [Computer Software]. Winsteps.com.

Madsen, A., McKagan, S. B., & Sayre, E. C. (2017). Resource letter RBAI-1: Research-based assessment instruments in physics and astronomy. American Journal of Physics, 85(4), 245–264. https://doi.org/10.1119/1.4977416

Maloney, D. P., O'Kuma, T. L., Hieggelke, C. J., & Van Heuvelen, A. (2001). Surveying students' conceptual knowledge of electricity and magnetism. American Journal of Physics, 69(S1), S12–S23. https://doi.org/10.1119/1.1371296

National Research Council (NRC). (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. The National Academies Press. https://doi.org/10.17226/13165

Partnership for 21st Century Learning. (2019). Framework for 21st century learning. Battelle for Kids.

Planinic, M., Ivanjek, L., & Susac, A. (2010). Rasch model based analysis of the Force Concept Inventory. Physical Review Special Topics-Physics Education Research, 6(1), 010103. https://doi.org/10.1103/PhysRevSTPER.6.010103

Raduta, C. (2019). Conceptual understanding of electromagnetism: A literature review. European Journal of Physics Education, 10(2), 23–38.

Redish, E. F. (2014). Oersted Lecture 2013: How should we think about how our students think? American Journal of Physics, 82(6), 537–551. https://doi.org/10.1119/1.4874260

Saarelainen, M., Laaksonen, A., & Hirvonen, P. E. (2007). Students' understanding of the magnetic field and the electromagnetic induction. European Journal of Physics, 28(1), 51–62. https://doi.org/10.1088/0143-0807/28/1/006

Sumintono, B., & Widhiarso, W. (2014). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial. Trim Komunikata.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Trim Komunikata.

Tiruneh, D. T., Verburgh, A., & Elen, J. (2017). Effectiveness of critical thinking instruction in higher education: A systematic review of intervention studies. Higher Education Studies, 4(1), 1–17. https://doi.org/10.5539/hes.v4n1p1

Wei, S., Liu, X., & Jia, Y. (2012). Using Rasch measurement to validate the instrument of students' understanding of models and modeling in science. International Journal of Science and Mathematics Education, 10(5), 1091–1112. https://doi.org/10.1007/s10763-011-9316-2

Wilcox, B. R., & Lewandowski, H. J. (2016). Open-ended versus multiple-choice questions in an assessment of students’ beliefs about experimental physics. Physical Review Physics Education Research, 12(2), 020132. https://doi.org/10.1103/PhysRevPhysEducRes.12.020132

Zohar, A., & Barzilai, S. (2013). A review of research on metacognition in science education: Current and future directions. Studies in Science Education, 49(2), 121–169. https://doi.org/10.1080/03057267.2013.847261

Zuza, K., De Cock, M., van Kampen, P., Bollen, L., & Guisasola, J. (2018). University students' understanding of the electromotive force concept in the context of electromagnetic induction. European Journal of Physics, 39(5), 055702. https://doi.org/10.1361-6404/aacc70




DOI: https://doi.org/10.29100/eduproxima.v8i3.10638

Refbacks

  • There are currently no refbacks.