Online Assessment Literacy of In-Service EFL Teachers: A Cross-Sectional Study of Competency Levels and Demographic Factors

Ziddan Marsa Utama; A. A. Gede Yudha Paramartha; Luh Gede Eka Wahyuni

doi:10.36663/joes.v6i2.1158

Authors

Ziddan Marsa Utama Universitas Pendidikan Ganesha
A. A. Gede Yudha Paramartha Universitas Pendidikan Ganesha
Luh Gede Eka Wahyuni Universitas Pendidikan Ganesha

DOI:

https://doi.org/10.36663/joes.v6i2.1158

Keywords:

Online Assessment Literacy, EFL Teachers, Digital Assessment, Assessment Literacy, Technology-Enhanced Learning

Abstract

Online assessment literacy (OAL) has become increasingly important as teachers are expected to design, implement, and evaluate assessments in technology-enhanced learning environments. However, limited evidence is available regarding teachers’ OAL levels and the influence of demographic characteristics on these competencies, particularly in the Indonesian context. This study aimed to investigate the online assessment literacy of in-service English as a Foreign Language (EFL) teachers in junior high schools and examine whether demographic factors influence their OAL scores. A quantitative cross-sectional survey design was employed. Data were collected from 48 in-service EFL teachers in Jembrana Regency, Bali, Indonesia, using an adapted version of the Assessment Literacy Inventory. The instrument consisted of 28 objective items covering seven assessment competency domains. Descriptive statistics were used to determine teachers’ OAL levels, while independent-samples t-tests were conducted to examine differences across demographic variables, including gender, educational background, teaching experience, school type, and certification status. The findings revealed that the overall level of OAL was categorized as fair (M = 15.42, SD = 2.89), with 56.25% of participants classified in the fair category and 35.42% in the good category. Among the assessed dimensions, teachers demonstrated stronger competencies in using assessment results for instructional decision-making and administering online assessments, while lower performance was observed in developing assessment instruments and recognizing ethical issues in online assessment. Furthermore, no statistically significant differences were found across any demographic variables (p > .05), indicating that gender, educational background, teaching experience, school type, and certification status did not significantly influence OAL scores. The study concludes that although teachers possess a basic understanding of online assessment practices, further professional development is needed to strengthen competencies related to assessment design and ethical assessment practices. The findings highlight the importance of continuous training and institutional support in enhancing teachers’ online assessment literacy in technology-enhanced learning environments.

Downloads

Download data is not yet available.

References

Abduh, M. (2021). Full-time online assessment during COVID-19 lockdown: EFL teachers’ perceptions. Asian EFL Journal Research, 28(1).

Al-Bahlani, S. M., & Ecke, P. (2021). Assessment competence and practices including digital assessment literacy of postsecondary English language teachers in Oman. Cogent Education, 10(2), 1–18.

Anam, S., & Putri, N. V. W. (2021). How literate am I about assessment: Evidence from Indonesian EFL pre-service and in-service teachers. English Review: Journal of English Education, 9(2), 109–119.

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.

Ashraf, H., & Zolfaghari, S. (2018). EFL teachers’ assessment literacy and their reflective teaching. International Journal of Instruction, 11(1), 425–436. https://doi.org/10.12973/iji.2018.11129a

Basuki, Y., & Hidayati, Y. (2019). Kahoot! or Quizizz: the Students’ Perspectives. Proceedings of the 3rd English Language and Literature International Conference, ELLiC, 27th April 2019, Semarang, Indonesia. https://doi.org/10.4108/eai.27-4-2019.2285331

Berry, V., Sheehan, S., & Munro, S. (2019). What does language assessment literacy mean to teachers? ELT Journal, 73(2), 113–123.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102

Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x

DeLuca, C., Chapman-Chin, A., LaPointe-McEwan, D., & Klinger, D. A. (2018). Student perspectives on assessment for learning. Curriculum Journal, 29(1), 77–94.

Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255–284.

Eyal, L. (2012). Digital assessment literacy: The core role of the teacher in a digital environment. Educational Technology & Society, 15(2).

GENÇ, E., ÇALIŞKAN, H., & YÜKSEL, D. (2020). Language assessment literacy level of EFL teachers: A focus on writing and speaking assessment knowledge of the teachers. Sakarya University Journal of Education, 10(2), 274–291. https://doi.org/10.19126/suje.626156

Hargittai, E., & Shafer, S. (2006). Differences in digital skills and their impact on educational practices. New Media & Society, 8(2), 211–233.

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Liyanage, I., & Bartlett, B. J. (2013). Digital literacy and the impact of gender and age: Findings from a systematic review. Computers & Education, 67, 255–266.

Mertler, C. A., & Campbell, C. (2005). Measuring teachers’ knowledge and application of classroom assessment concepts: Development of the Assessment Literacy Inventory. Paper presented at the Annual Meeting of the American Educational Research Association.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.

Paramartha, A. A. G. Y., Dewi, N. L. P. E. S., & Wahyuni, L. G. E. (2021). Readability of English Teacher Assessment Literacy Instrument in Indonesian Context. Proceedings of the 4th International Conference on Innovative Research Across Disciplines (ICIRAD 2021), 221–226. https://doi.org/10.2991/assehr.k.211222.035

Permana, I. G. Y., Rismadewi, N. W. M., & Putra, N. C. A. (2024). Gamification in EFL Classroom: Results of Implementation and Teachers’ Point of View. Journal of Ultimate Research and Trends in Education, 6(1), 1–11. https://doi.org/10.31849/utamax.v6i1.19297

Popham, W. J. (2010). Classroom assessment: What teachers need to know (6th ed.). Pearson Education.

Pradnyadewi, D. A. M., & Kristiani, P. E. (2021). Use of Quizizz in improving students’ reading skill. The Art of Teaching English as a Foreign Language (TATEFL), 2(1), 1–6. https://doi.org/10.36663/tatefl.v2i1.93

Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1–6.

Putra, I. K. K. D., Paramartha, A. A. G. Y., & Mahayanti, N. W. S. (2024). Online assessment literacy level of EFL teachers in Denpasar. Measurement in Educational Research, 4(1), 49–56. https://doi.org/10.33292/meter.v4i1.321

Redecker, C. (2013). The use of ICT for the assessment of key competences. Joint Research Centre of the European Commission.

Redecker, C., & Johannessen, Ø. (2013). Changing assessment—Towards a new assessment paradigm using ICT. European Journal of Education, 48(1), 79–96. https://doi.org/10.1111/ejed.12018

Shepard, L. A. (2020). Classroom assessment principles to support teaching and learning. University of Colorado Boulder.

Spector, P. E. (2019). Do not cross me: Optimizing the use of cross-sectional designs. Journal of Business and Psychology, 34(2), 125–137. https://doi.org/10.1007/s10869-018-09613-8

Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), 591–604. https://doi.org/10.1080/02602938.2011.557496

Sun, H., & Zhang, J. (2022). Assessment literacy of college EFL teachers in China: Status quo and mediating factors. Studies in Educational Evaluation, 74, 101157. https://doi.org/https://doi.org/10.1016/j.stueduc.2022.101157

Sultana, N. (2019). Language assessment literacy: An uncharted area for the English language teachers in Bangladesh. Language Testing in Asia, 9(1), 1–14. https://doi.org/10.1186/s40468-019-0077-8

Taylor, L. (2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21–36. https://doi.org/10.1017/S0267190509090035

Tondeur, J., van Braak, J., Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2017). Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: A systematic review of qualitative evidence. Educational Technology Research and Development, 65(3), 555–575. https://doi.org/10.1007/s11423-016-9481-2

Xu, Y., & Brown, G. T. L. (2016). Teacher assessment literacy in practice: A reconceptualization. Teaching and Teacher Education, 58, 149–162. https://doi.org/10.1016/j.tate.2016.05.010

Zulaiha, S., & Mulyono, H. (2020). Exploring junior high school EFL teachers’ training needs of assessment literacy. Cogent Education, 7(1), 1–13. https://doi.org/10.1080/2331186X.2020.1772943