International Journal of Education and Development using ICT

International Journal of Education and Development using ICT > Vol. 19, No. 3 (2023)

Student-Written Versus ChatGPT-Generated Discursive Essays: A Comparative Coh-Metrix Analysis of Lexical Diversity, Syntactic Complexity, and Referential Cohesion

Tlatso Nkhobo, University of South Africa
Chaka Chaka, University of South Africa

Abstract
This article reports on a comparative analysis of two sets of essays, student-discursive essays (SDEs) and ChatGPT-generated discursive essays (ChatGPT-GDEs) on the same essay topic using Coh-Metrix. It focused on three Coh-Metrix indices, lexical density, syntactic complexity, and referential cohesion as the basis for the comparative analysis. The authors also conducted a t test on the Coh-Metrix results, especially the mean scores, in relation to these three linguistic indices. Using convenience sampling, the study selected seven SDEs from the essays that were submitted as part of an assignment for an English Studies module in the second semester of 2020 at the University of South Africa. ChatGPT was prompted with the same essay topic that had been used for the SDEs. Overall, at raw mean score levels, the SDEs outperformed ChatGPT-GDEs in lexical density and referential cohesion, while ChatGPT-GDEs did so in syntactic complexity. Nonetheless, at a t test level, there was no statistically significant difference between the mean scores of the two essay sets in relation to the three Coh-Metrix linguistic indices investigated in this study.

Full Text: PDF

Research
Support Tool