The Role of Generative AI in Automating English Writing Assessment: Benefits and Challenges
DOI:
https://doi.org/10.63324/lec.2v.1i.63Keywords:
Artificial Intelligence, Education Technology, English Writing, Generative AI, Writing FeedbackAbstract
The integration of generative Artificial Intelligence (AI) in English writing assessment has gained considerable attention for its potential to enhance the efficiency and accuracy of evaluations. However, while AI tools like Grammarly and GPT-based platforms have been employed for basic writing assessments, their effectiveness in evaluating more complex writing aspects remains underexplored. This study aims to fill this gap by examining the role of generative AI in automating English writing assessment, focusing on its benefits, challenges, and implications for both students and educators. The research design is a mixed-methods approach, incorporating both quantitative and qualitative data through surveys and semi-structured interviews. The study involved 100 students and 10 educators from the Faculty of Sharia at UIN Raden Intan Lampung, who had experience using AI-based writing tools. Data was collected through structured surveys and in-depth interviews, as well as by comparing AI-generated writing assessments with human grading. The findings revealed that while AI tools are highly effective in evaluating grammar and structure, they struggle to assess higher-order writing skills, such as content coherence and critical thinking. Both students and educators acknowledged the time-saving benefits of AI tools, but also highlighted the limitations of AI in understanding nuanced writing aspects. The study implies that AI tools can complement traditional assessment methods but should not fully replace human judgment, especially in complex writing tasks. Future research should focus on enhancing AI algorithms to better assess the depth of writing and address ethical concerns regarding data privacy and biases in AI models.
References
Algaraady, J., & Mahyoob, M. (2023). ChatGPT’s capabilities in spotting and analyzing writing errors experienced by EFL learners. Arab World English Journal, 9, 3–17. https://doi.org/10.24093/awej/call9.1
Aravantinos, S., Lavidas, K., Voulgari, I., Papadakis, S., Karalis, T., & Komis, V. (2024). Educational Approaches with AΙ in Primary School Settings: A Systematic Review of the Literature Available in Scopus. Education Sciences, 14(7), 744. https://doi.org/10.3390/educsci14070744
Braun, V., Clarker, V., & Rance, N. (2014). How to use thematic analysis with interview data. In A. Vossler & N. Moller (Eds.), The Counselling & Psychotherapy Research Handbook, (pp. 183–197). Sage.
Creswell, J. W. (2012). Educational research: Planning, conducting and evaluating quantitative and qualitative research. Pearson Education.
Hastomo, T., Mandasari, B., & Widiati, U. (2024). Scrutinizing Indonesian pre-service teachers’ technological knowledge in utilizing AI-powered tools. Journal of Education and Learning (EduLearn), 18(4), 1572–1581. https://doi.org/10.11591/edulearn.v18i4.21644
Hastomo, T., Sari, A. S., Widiati, U., Ivone, F. M., Zen, E. L., & Andianto, A. (2025). Exploring EFL Teachers’ Strategies in Employing AI Chatbots in Writing Instruction to Enhance Student Engagement. World Journal of English Language, 15(7), 93–102. https://doi.org/10.5430/wjel.v15n7p93
Hopfenbeck, T. N. (2023). The Future of Educational Assessment: Self-assessment, Grit and ChatGPT? Assessment in Education: Principles, Policy & Practice, 30(2), 99–103. https://doi.org/10.1080/0969594X.2023.2212192
Kohnke, L. (2024). Exploring EAP students’ perceptions of GenAI and traditional grammar-checking tools for language learning. Computers and Education: Artificial Intelligence, 7, 100279. https://doi.org/10.1016/j.caeai.2024.100279
Lavidas, K., Voulgari, I., Papadakis, S., Athanassopoulos, S., Anastasiou, A., Filippidi, A., Komis, V., & Karacapilidis, N. (2024). Determinants of Humanities and Social Sciences Students’ Intentions to Use Artificial Intelligence Applications for Academic Purposes. Information, 15(6), 314. https://doi.org/10.3390/info15060314
Mandasari, B., Basthomi, Y., Hastomo, T., Afrianto, Hamzah, I., & Aminatun, D. (2025). The Snapshots of Indonesian Pre-Service English Teachers’ Perspectives on Integrating Technology-Based Tools to Rural Schools. Voices of English Language Education Society, 9(1), 42–57. https://doi.org/10.29408/veles.v9i1.27965
Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10(2), 1–17. https://doi.org/10.1080/2331186X.2023.2236469
Nikolic, S., Sandison, C., Haque, R., Daniel, S., Grundy, S., Belkina, M., Lyden, S., Hassan, G. M., & Neal, P. (2024). ChatGPT, Copilot, Gemini, SciSpace and Wolfram versus higher education assessments: an updated multi-institutional study of the academic integrity impacts of Generative Artificial Intelligence (GenAI) on assessment, teaching and learning in engineering. Australasian Journal of Engineering Education, 1–28. https://doi.org/10.1080/22054952.2024.2372154
Nurchurifiani, E., Maximilian, A., Ajeng, G. D., Wiratno, P., Hastomo, T., & Wicaksono, A. (2025). Leveraging AI-Powered Tools in Academic Writing and Research: Insights from English Faculty Members in Indonesia. International Journal of Information and Education Technology, 15(2), 312–322. https://doi.org/10.18178/ijiet.2025.15.2.2244
Oktarin, I. B., Saputri, M. E. E., Magdalena, B., Hastomo, T., & Maximilian, A. (2024). Leveraging ChatGPT to enhance students’ writing skills, engagement, and feedback literacy. Edelweiss Applied Science and Technology, 8(4), 2306–2319. https://doi.org/10.55214/25768484.v8i4.1600
Ono, K., & Morita, A. (2024). Evaluating Large Language Models: ChatGPT-4, Mistral 8x7B, and Google Gemini Benchmarked Against MMLU. COMPUTING AND PROCESSING, 8(1), 1–7. https://doi.org/10.36227/techrxiv.170956672.21573677/v1
Papadakis, S., Kiv, A. E., Kravtsov, H. M., Osadchyi, V. V., Marienko, M. V., Pinchuk, O. P., ... & Striuk, A. M. (2023). Unlocking the power of synergy: the joint force of cloud technologies and augmented reality in education. In Joint Proceedings of the 10th Workshop on Cloud Technologies in Education (CTE 2021) and 5th International Workshop on Augmented Reality in Education (AREdu 2022), Kryvyi Rih, Ukraine, May 23, 2022. CEUR Workshop Proceedings.
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), 342–363. https://doi.org/10.37074/jalt.2023.6.1.9
Uğraş, H., Uğraş, M., Papadakis, S., & Kalogiannakis, M. (2024). ChatGPT-Supported Education in Primary Schools: The Potential of ChatGPT for Sustainable Practices. Sustainability, 16(22), 9855. https://doi.org/10.3390/su16229855
Wang, K., Ruan, Q., Zhang, X., Fu, C., & Duan, B. (2024). Pre-service teachers’ GenAI anxiety, technology self-efficacy, and TPACK: Their structural relations with behavioral intention to design GenAI-assisted teaching. Behavioral Sciences, 14(5), 373. https://doi.org/10.3390/bs14050373
Waziana, W., Andewi, W., Hastomo, T., & Hasbi, M. (2024). Students’ perceptions about the impact of AI chatbots on their vocabulary and grammar in EFL writing. Register Journal, 17(2), 328–362. https://doi.org/https://doi.org/10.18326/register.v17i2.352-382
Wulyani, A. N., Widiati, U., Muniroh, S., Rachmadhany, C. D., Nurlaila, N., Hanifiyah, L., & Sharif, T. I. S. T. (2024). Patterns of utilizing AI–assisted tools among EFL students: Need surveys for assessment model development. LLT Journal: A Journal on Language and Language Teaching, 27(1), 157–173. https://doi.org/10.24071/llt.v27i1.7966
Zulianti, H., Hastuti, H., Nurchurifiani, E., Hastomo, T., Maximilian, A., & Ajeng, G. D. (2024). Enhancing Novice EFL Teachers’ Competency in AI-Powered Tools Through a TPACK-Based Professional Development Program. World Journal of English Language, 15(3), 117. https://doi.org/10.5430/wjel.v15n3p117
