Should we try to detect AI-generated writing?

Professor Oguz A Acar, Professor of Marketing & Innovation

In the previous step, you discussed marking and feedback. In this step, you will discuss detection and fairness.

Detection of AI-written text

One question many of us are wondering about is:

Can you reliably detect writing created by generative AI tools like ChatGPT?

The short answer: not really!

Although there are multiple approaches and even specialised software designed to do just that, the truth is they are not as reliable as they should be to be used for consequential decisions in education.

Techniques to detect AI writing

There are various methods to try and catch AI in the act of writing. Some of the most advanced techniques include:

• Soft watermarking: this technique imprints specific patterns onto generated text through AI model modifications. Detection involves comparing text patterns with predefined lists.
• Zero-shot: this technique employs pre-trained classifiers to label text as human or machine-written based on content/style.
• Retrieval-based: this compares the text with a vast human-written text database where text similarity suggests human authorship, while dissimilarity suggests AI origin.
• Neural network-based: this technique utilises neural networks trained to differentiate between human and AI text using attributes like syntax, semantics and sentiment.

Limitations of current techniques

While these techniques sound advanced (and some of them indeed are) several studies point to their tendency to produce false positives; these tools sometimes incorrectly identify human-written content as AI-generated. This is concerning in an educational context where wrongly penalising students has significant repercussions.

Some educators turn to AI tools themselves to identify AI writing. It is worth noting that asking generative AI tools like ChatGPT to recognise AI writing is not advisable. These tools themselves are not reliable judges of their own work. For instance, ChatGPT has been known to incorrectly claim authorship of well-known texts, including the U.S. Constitution.

The evolving nature of AI language models suggests that as they improve, the capability of detection tools to differentiate between human and AI writing may further diminish. This challenge was so pronounced that OpenAI decided to halt its efforts to create a reliable AI-detection system due to a low rate of accuracy.

Prominent bodies, such as the Federal Trade Commission (FTC), have even cautioned against the bold marketing claims of some AI detection tools, urging users to ‘take claims about those tools with a few megabytes of salt’.

Another challenge with these detection tools is that they can be easily bypassed. Research shows that simple strategies, such as light paraphrasing or smart prompting, can fool these detection systems.

Finally, a particularly concerning aspect is the fairness of these detectors. Studies have documented that these tools often have a higher false positive rate for non-native English speakers. This raises concerns about the unintentional bias that may unfairly penalise writers who have different backgrounds.

Conclusion

While it may seem tempting to use specialised tools to detect AI-generated writing, the current state of technology suggests that these tools are far from foolproof. Whether it’s high false-positive rates, the ease of bypassing these systems or their unreliability with non-native English writing, the existing methods have many limitations. As such, educators and institutions should approach AI detection tools with caution. Relying on them as the sole determinant of a student’s writing authenticity is unwise and can lead to unfair consequences.

Now that you have completed this step, you have learned about the reliability of current AI detection tools. In the next step, you will discuss the blurred boundaries of authorship in the age of AI.

Join the conversation

With the current limitations of AI detection tools, should academic institutions continue to employ them? Or should the focus shift to educating students about ethical uses of AI assistance? Justify your stance with any relevant evidence or reasoned arguments.