Assessing the Effectiveness of CAT and MCAT in Measuring Students' Understanding in Technology Education

Authors

  • Muslem Daud Lecturer in Economics Education Study Program, FKIP Universitas Serambi Mekkah, Banda Aceh, Indonesia
  • Bor-Chen Kuo Postgraduate Lecturer in Educational Information and Measurement, National Taichung University of Education, Taiwan
  • Jamaluddin Jamaluddin Lecturer at Physics Education Study Program, FTK Universitas Islam Negeri Ar-Raniry, Banda Aceh, Indonesia
  • Muhammad Duskri Lecturer in Mathematics Education Study Program, FKT Universitas Islam Negeri Ar-Raniry, Banda Aceh, Indonesia
  • Cut Morina Zubainur Lecturer in Mathematics Education Study Program, FKIP Universitas Syiah Kuala, Banda Aceh, Indonesia
  • Jalaluddin Jalaluddin Lecturer in the Physical Education Health and Recreation Study Program, FKIP Universitas Serambi Mekkah, Banda Aceh, Indonesia

DOI:

https://doi.org/10.21111/attadib.v19i1.11970

Keywords:

MCAT, assesment, effective, CAT

Abstract

This research aims to do two things. First, it aims to make a computer adaption test (CAT) for Islamic subjects for the santri in ABC Pesantren. Second, it aims to see the results of the CAT test for Islamic subjects in the production of Learning Medical Records. The records is for diagnosing students' abilities. This research consists of two parts using the research and development (R&D) approach: 1). We collected the data by observing, documenting, and interviewing. We analyzed the data using the concept of reliability among the raters/respondents. We used the generalizability coefficient in Genova 3.1. We analyzed the results using numbers. They describe data from RMP technology education. The research shows that: (1) the CAT can be made based on user needs. It's Internet-based, user-friendly, interactive, highly secure, and easily accessible. (2) The CAT can recognize three different users: school management, teachers, and students. (3) The expert evaluation's quantitative data has high consistency: 0.76. It comes from single, small group, and large group outcomes. (4) The CAT can accurately estimate students' abilities and record learning records. The software uses the triangle decision tree method. It can pick test items and measure ability well. This is clear from the correlation (r) between ability and the software's CAT scores.

References

Amirzan. (2021). Implementing Discipline In The Covid-19 Period In Improving Student Achievement, . Jurnal Serambi Ilmu, Vol 22, No 2.

Ashabul Kahfi. (2020). Tantangan Dan Harapan Pembelajaran Jarak Jauh Di Masa Pandemi Covid 19, . Dirasah, Vol. 03 No.2.

Assidiqi, M. H. , & Sumarni, W. (2020). Pemanfaatan Platform Digital dalam Pembelajaran Daring di masa Pandemi Covid-19. . Prosiding Seminar Nasional Pascasarjana UNNES.

Balasheva, I. V., & Pretrova, N. F. (2016). Psychological stability of a personality and capability of tolerant interaction as divers manifestations of tolerance, International . Journal of Environmental and Science Education, Vol. 11 (10).

Basar, A. M. (2021). Problematika Pembelajaran Jarak Jauh Pada Masa Pandemi Covid-19. Edunesia : Jurnal Ilmiah Pendidikan, 2(1), 208–218. https://doi.org/10.51276/edu.v2i1.112

Bennett, N., Borg, W. R., & Gall, M. D. (1984). Educational Research: An Introduction. British Journal of Educational Studies, 32(3), 274. https://doi.org/10.2307/3121583

Daud, M. , Mok M.C.M, & Kuo, B. C. ,. (2015). The Construction of an online curriculum-based multidimensional computerized adaptive testing, I. International Multidisciplinary Conference (Pp.322-328).

Daud, M. , Mok M.C.M., & Kuo, B. C. ,. (2015). The Construction of an online curriculum-based multidimensional computerized adaptive testing, In International Multidisciplinary Conference (pp.322-328). . Banda Aceh. Universitas Serambi Mekkah.

Daud, M. , Mok, M. C. M., Yusrizal, Y. , & Kuo, B. C. ,. (2018). Computerised Adaptive Testing: Application and Implications for Teaching and Learning in Indonesia, in Routledge International Handbook of Schools and Schooling in Asia, . Routledge Inc.

Daud, M. , Mutia, R., Imran, Israwati, Yusrizal, & Sariakin. (2020). An Evaluation of Indonesian Students’ Self-Confidence in Learning Mathematics, . Proceeding ICMR .

Fitria, S., & Surya, D. (2021). The Impact of Covid-19 Pandemic on Students’ Sedentary Behavior in Indonesia. JURNAL SERAMBI ILMU, 22(1), 88–97. https://doi.org/10.32672/si.v22i1.2732

Fliege, H., Becker, J., Walter, O. B., Rose, M., Bjorner, J. B., & Klapp, B. F. (2009). Evaluation of a computer‐adaptive test for the assessment of depression (D‐CAT) in clinical application. International Journal of Methods in Psychiatric Research, 18(1), 23–36. https://doi.org/10.1002/mpr.274

Kuo, B.-C., Daud, M., & Yang, C.-W. (2015a). Multidimensional Computerized Adaptive Testing for Indonesia Junior High School Biology. EURASIA Journal of Mathematics, Science and Technology Education, 11(5). https://doi.org/10.12973/eurasia.2015.1384a

Kuo, B.-C., Daud, M., & Yang, C.-W. (2015b). Multidimensional Computerized Adaptive Testing for Indonesia Junior High School Biology. EURASIA Journal of Mathematics, Science and Technology Education, 11(5). https://doi.org/10.12973/eurasia.2015.1384a

Lord, F. M. (2012). Applications of Item Response Theory To Practical Testing Problems. Routledge. https://doi.org/10.4324/9780203056615

Mok, M. M. C., & Zhu, J. (2014). Cluster analysis of attitudes towards feedback and their mathematics achievement: a study of Hong Kong primary students. . Asia Pacific Journal of Educational Development, 3(2), 1-13.

Phankokkruad, M., & Woraratpanya, K. (2008). An Automated Decision System for Computer Adaptive Testing Using Genetic Algorithms. 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 655–660. https://doi.org/10.1109/SNPD.2008.118

Prandika, D. D., & Amrullah, M. (2021). The Effect of the Covid 19 Pandemic on the Online Learning Process. Proceedings of The ICECRS, 10, 63–71. https://doi.org/10.21070/icecrs20211058

Rafiq, R., & Shah, P. A. ,. (2015). Teachers’ role and stress relationship: A Review, The International Journal of Human Resource Management and Industrial Research, Vol 2, (1), 8-17.

Riskiana Rahma, & Nursiwi Nugraheni. (2020). Strategi Pembelajaran Guru Sekolah Dasar Di Era Pandemi Covid-19,. Prosiding Webinar Nasional IAHN-TP Palangka Raya 2020.

Sarah, S. (2022). High Order Thinking Skills (Hots) Of Ibtidaiyah Madrasah Teacher Education Program (Pgmi) Study Program Students During The Covid-19 Pandemic, . Jurnal Serambi Ilmu, Vol 23, No 1 (2022).

Thu Nguyet, P. T., & Daud, M. (2021). Computer Adaptive Test Development To Assess Students’ Psychology. JURNAL SERAMBI ILMU, 22(1), 139–149. https://doi.org/10.32672/si.v22i1.2760

van der Linden, W. J., & Pashley, P. J. (2000). Item Selection and Ability Estimation in Adaptive Testing. In Computerized Adaptive Testing: Theory and Practice (pp. 1–25). Springer Netherlands. https://doi.org/10.1007/0-306-47531-6_1

Walter, O. B., Becker, J., Bjorner, J. B., Fliege, H., Klapp, B. F., & Rose, M. (2007). Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT). Quality of Life Research, 16(S1), 143–155. https://doi.org/10.1007/s11136-007-9191-7

Weiss, D. J. (2004). Computerized Adaptive Testing for Effective and Efficient Measurement in Counseling and Education. Measurement and Evaluation in Counseling and Development, 37(2), 70–84. https://doi.org/10.1080/07481756.2004.11909751

Zeigler-Hill, V., & Wallace, M. T. (2012). Self-esteem Instability and Psychological Adjustment. Self and Identity, 11(3), 317–342. https://doi.org/10.1080/15298868.2011.567763

Downloads

Published

2024-06-26

How to Cite

Daud, M., Kuo, B.-C., Jamaluddin, J., Duskri, M., Zubainur, C. M., & Jalaluddin, J. (2024). Assessing the Effectiveness of CAT and MCAT in Measuring Students’ Understanding in Technology Education. At-Ta’dib, 19(1), 117–132. https://doi.org/10.21111/attadib.v19i1.11970

Issue

Vol. 19 No. 1 (2024): Islamic Educational Institutions and Their Dynamics in Facing the Times.

Section

Articles

License

Copyright (c) 2024 Muslem Daud, Bor-Chen Kuo, Jamaluddin, Muhammad Duskri, Cut Morina Zubainur, Jalaluddin

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

1. PROPOSED POLICY FOR JOURNALS THAT OFFER OPEN ACCESS

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).

2. PROPOSED POLICY FOR JOURNALS THAT OFFER DELAYED OPEN ACCESS


Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication, with the work [SPECIFY PERIOD OF TIME] after publication simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).