International Journal of Education and Development using ICT

International Journal of Education and Development using ICT > Vol. 19, No. 3 (2023)

Comparing Measures of Syntactic and Lexical Complexity in Artificial Intelligence and L2 Human-Generated Argumentative Essays

nomsa NC zindela, UNISA

Abstract
This study explores syntactic and lexical complexity in argumentative essays written by Artificial Intelligence (AI) and Humans (HS). First year Tswana second language (L2) learners of English in a South African University wrote the HS argumentative essays. The AI essays have been generated by ChatGPT-3.5 on the same topics. Using a multidimensional perspective to genre analysis, this study examines differences in the essays using linguistic measures of syntactic and lexical complexity. Even though not all researchers agree on the nature of linguistic differences, the extant literature links features such as syntactic and lexical complexity to writing style and quality. However, little is known about this variation in content generated by ChatGPT when compared to writing by university students studying English as a second language (L2). Findings from this study show differences in the use of complexity features between AI-generated-generated content (AIGC) and human-generated content (HGC). AIGC was found to be using more sophisticated and varied vocabulary. HGC was found to use more function words and less diverse vocabulary. The use of content words was comparable with HGC using a lot more content words than AIGC.

Full Text: PDF

Research
Support Tool