AI's Essays Outperform Humans (Research)

AI's Essays Outperform Humans (Research)(arxiv.org)

1 points by romanichm12 2 years ago | 3 comments

The paper linked is a large-scale study comparing human-written versus AI-generated argumentative student essays. The researchers used a large corpus of essays and had them rated by human experts (teachers) using standard criteria. They also analyzed the linguistic characteristics of the generated essays https://www.linkedin.com/pulse/top-5-best-ai-essay-generator....

Here's the main findings explained in a simple way:

1) The AI they tested, called ChatGPT, can write essays that are rated higher for quality than human-written essays. This means that the AI was able to write essays that the teachers thought were better than the ones written by people.

2) The AI's writing style is different from humans. For example, it uses fewer words that show discussion or certainty, but it uses more words that turn verbs into nouns and has a greater variety of words.

3) The researchers concluded that AI models like ChatGPT and Textero.ai are better than humans at writing argumentative essays. They suggest that teachers should start thinking about how to use these AI tools in education, just like how calculators are used in math. They believe that AI can help free up time for other learning objectives.

So, to answer your question, according to this study, AI can write good essays, even better than humans in some cases. But remember, it's not just about who writes better essays. It's also about learning and understanding, and that's something humans are still very good at!

mintaka5 2 years ago |

but is quality of an essay a artificially derived standard? i mean we can talk about how great the material is that is produced, but seriously, folks, aren't we chasing our tails here? is an AI good at wirting essays because of its own standard of quality or is it being measured against our own already established low bar? i mean if we're saying the computer is better than us, measured against a standard we dreamed up and supplied, it really is lacking objectivity here.

jfengel 2 years ago | |

Precisely. This paper used human graders, but these kinds of essays are also being graded by AI. The essays use a standard form, and they're marked objectively against it.

AI is very good at writing things that don't need to be written. It regurgitates, just as students are expected to do. The essays aren't expected to be good, for the same reason nobody makes any math discoveries in their homework. They're expected to go on and eventually do something of actual merit.

That's harder to grade, but thus far there's no indication that AI will develop the way humans do. Lousy student essays hopefully give way to good writing. AI is trained to do essays, but nobody wants essays.

Maybe this is a waypoint for AI as well. I've got my doubts that the tactics we're using can ever develop that way, but I'll try to keep my mind open in case something materializes -- a pinch of randomness that better mimics creativity and insight.