Assessing the Effectiveness of CAT and MCAT in Measuring Students’ Understanding in Technology Education

Muslem Daud; Bor-Chen Kuo; Jamaluddin Jamaluddin; Muhammad Duskri; Cut Morina Zubainur; Jalaluddin Jalaluddin

doi:10.21111/at-tadib.v19i1.64

المؤلفون

Muslem Daud Universitas Serambi Mekkah, Banda Aceh, Indonesia
Bor-Chen Kuo National Taichung University of Education, Taiwan
Jamaluddin Jamaluddin Universitas Islam Negeri Ar-Raniry, Banda Aceh, Indonesia
Muhammad Duskri Universitas Islam Negeri Ar-Raniry, Banda Aceh, Indonesia
Cut Morina Zubainur Universitas Syiah Kuala, Banda Aceh, Indonesia
Jalaluddin Jalaluddin Universitas Serambi Mekkah, Banda Aceh, Indonesia

DOI:

https://doi.org/10.21111/at-tadib.v19i1.64

الكلمات المفتاحية:

CAT, MCAT, assesment, effective

الملخص

This research aims to do two things. First, it aims to make a computer adaption test (CAT) for Islamic subjects for the santri in ABC Pesantren. Second, it aims to see the results of the CAT test for Islamic subjects in the production of Learning Medical Records. The records is for diagnosing students’ abilities. This research consists of two parts using the research and development (R&D) approach: 1). We collected the data by observing, documenting, and interviewing. We analyzed the data using the concept of reliability among the raters/respondents. We used the generalizability coefficient in Genova 3.1. We analyzed the results using numbers. They describe data from RMP technology education. The research shows that: (1) the CAT can be made based on user needs. It’s Internet-based, user-friendly, interactive, highly secure, and easily accessible. (2) The CAT can recognize three different users: school management, teachers, and students. (3) The expert evaluation’s quantitative data has high consistency: 0.76. It comes from single, small group, and large group outcomes. (4) The CAT can accurately estimate students’ abilities and record learning records. The software uses the triangle decision tree method. It can pick test items and measure ability well. This is clear from the correlation (r) between ability and the software’s CAT scores.

المراجع

Amirzan. (2021). Implementing Discipline In The Covid-19 Period In Improving Student Achievement, . Jurnal Serambi Ilmu, Vol 22, No 2.

Ashabul Kahfi. (2020). Tantangan Dan Harapan Pembelajaran Jarak Jauh Di Masa Pandemi Covid 19, . Dirasah, Vol. 03 No.2.

Assidiqi, M. H. , & Sumarni, W. (2020). Pemanfaatan Platform Digital dalam Pembelajaran Daring di masa Pandemi Covid-190. Prosiding Seminar Nasional Pascasarjana UNNES.

Balasheva, I. V., & Pretrova, N. F. (2016). Psychological stability of a personality and capability of tolerant interaction as divers manifestations of tolerance, International . Journal of Environmental and Science Education, Vol. 11 (10).

Basar, A. M. (2021). Problematika Pembelajaran Jarak Jauh Pada Masa Pandemi Covid-19. Edunesia : Jurnal Ilmiah Pendidikan, 2(1), 208–218. https://doi.org/10.51276/edu.v2i1.112

Bennett, Neville, Walter R. Borg, dan Meredith D. Gall. “Educational Research: An Introduction.” British Journal of Educational Studies 32, no. 3 (Oktober 1984): 274. https://doi. org/10.2307/3121583.

Daud, M. , Mok M.C.M, & Kuo, B. C. ,. (2015). The Construction of an online curriculum-based multidimensional computerized adaptive testing, I. International Multidisciplinary Conference (Pp.322-328).

Daud, M. , Mok M.C.M., & Kuo, B. C. ,. (2015). The Construction of an online curriculum-based multidimensional computerized adaptive testing, In International Multidisciplinary Conference (pp.322-328). . Banda Aceh. Universitas Serambi Mekkah.

Daud, M. , Mok, M. C. M., Yusrizal, Y. , & Kuo, B. C. ,. (2018). Computerised Adaptive Testing: Application and Implications for Teaching and Learning in Indonesia, in Routledge International Handbook of Schools and Schooling in Asia, . Routledge Inc.

Daud, M. , Mutia, R., Imran, Israwati, Yusrizal, & Sariakin. (2020). An Evaluation of Indonesian Students’ Self-Confidence in Learning Mathematics, . Proceeding ICMR .

Fitria, S., & Surya, D. (2021). The Impact of Covid-19 Pandemic on Students’ Sedentary Behavior in Indonesia. JURNAL SERAMBI ILMU, 22(1), 88–97. https://doi.org/10.32672/si.v22i1.2732

Fliege, Herbert, Janine Becker, Otto B. Walter, Matthias Rose, Jakob B. Bjorner, dan Burghard F. Klapp. “Evaluation of a Computer-adaptive Test for the Assessment of Depression (D-CAT) in Clinical Application.” International Journal of Methods in Psychiatric Research 18, no. 1 (Maret 2009): 23–36. https:// doi.org/10.1002/mpr.274.

Kuo, Bor-Chen, Muslem Daud, dan Chih-WeiYang. “Multidimensional Computerized Adaptive Testing for Indonesia Junior High School Biology.” EURASIA Journal of Mathematics, Science and Technology Education 11, no. 5 (25 Agustus 2015). https:// doi.org/10.12973/eurasia.2015.1384a.

Lord, F. M. Applications of Item Response Theory To Practical Testing Problems. 0 ed. Routledge, 2012. https://doi. org/10.4324/9780203056615.

Mok, M. M. C., & Zhu, J. (2014). Cluster analysis of attitudes towards feedback and their mathematics achievement: a study of Hong Kong primary students. . Asia Pacific Journal of Educational Development, 3(2), 1-13.

Phankokkruad, M., dan K. Woraratpanya. “An Automated Decision System for Computer Adaptive Testing Using Genetic Algorithms.” Dalam 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 655–60. Phuket, Thailand: IEEE, 2008. https://doi.org/10.1109/ SNPD.2008.118.

Prandika, D. D., & Amrullah, M. (2021). The Effect of the Covid 19 Pandemic on the Online Learning Process. Proceedings of The ICECRS, 10, 63–71. https://doi.org/10.21070/icecrs20211058

Rafiq, R., & Shah, P. A. ,. (2015). Teachers’role and stress relationship: A Review, The International Journal of Human Resource Management and Industrial Research, Vol 2, (1), 8-17.

Riskiana Rahma, & Nursiwi Nugraheni. (2020). Strategi Pembelajaran Guru Sekolah Dasar Di Era Pandemi Covid-19,. Prosiding Webinar Nasional IAHN-TP Palangka Raya 2020.

Sarah, S. (2022). High Order Thinking Skills (Hots) Of Ibtidaiyah Madrasah Teacher Education Program (Pgmi) Study Program Students During The Covid-19 Pandemic, . Jurnal Serambi Ilmu, Vol 23, No 1 (2022).

Thu Nguyet, P. T., & Daud, M. (2021). Computer Adaptive Test Development To Assess Students’ Psychology. JURNAL SERAMBI ILMU, 22(1), 139–149. https://doi.org/10.32672/si.v22i1.2760

Ulfatun Wahidatun Nisa. “ISLAMIZATION OF KNOWLEDGE AND ITS CHALLENGE.” Dalam PROCEEDING OF INTERNATIONAL CONFERENCE ON EDUCATION, SOCIETY AND HUMANITY. Probolinggo: UNUJA, 2023.

Van Der Linden, Wim J., dan Peter J. Pashley. “Item Selection and Ability Estimation in Adaptive Testing.” Dalam Computerized Adaptive Testing: Theory and Practice, disunting oleh Wim

J. Van Der Linden dan Gees A.W. Glas, 1–25. Dordrecht: Springer Netherlands, 2000. https://doi.org/10.1007/0-306-47531-6_1.

Walter, O. B., Becker, J., Bjorner, J. B., Fliege, H., Klapp, B. F., & Rose, M. (2007). Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT). Quality of Life Research, 16(S1), 143–155. https://doi.org/10.1007/s11136-007-9191-7

Weiss, David J. “Computerized Adaptive Testing for Effective and Efficient Measurement in Counseling and Education.” Measurement and Evaluation in Counseling and Development 37, no. 2 (Juli 2004): 70–84. https://doi.org/10.1080/07481756.2004.11909751.

Zeigler-Hill, Virgil, dan Marion T. Wallace. “Self-Esteem Instability and Psychological Adjustment.” Self and Identity 11, no. 3 (Juli 2012): 317–42. https://doi.org/10.1080/15298868.2011.567763.