Fake AI Students Score Higher Than Real Students!
@TeacherToolkit
Ross Morrison McGill founded @TeacherToolkit in 2007, and today, he is one of the ‘most followed educators’on social media in the world. In 2015, he was nominated as one of the ‘500 Most Influential People in Britain’ by The Sunday Times as a result of…
Read more about @TeacherToolkit
Could artificial intelligence help students cheat their way through exams and qualifications?
New research (June 2024) reveals that AI submissions can outperform human students in university exams, raising urgent questions for education and assessment integrity.
AI scores higher …
At a leading UK university, researchers (Scarfe et al., 2024) recently tested whether AI could infiltrate university exams unnoticed.
The study injected ChatGPT-generated responses into undergraduate psychology exams, with 33 fictitious AI students (BBC News) sitting alongside real students.
The result? A staggering 97 per cent of AI-written responses went undetected, outperforming human students in both first and second-year assessments.
At the time of the study, “the use of AI to complete exams was not allowed and fell under the standard University academic misconduct policy.”
The experiment showed that these AI submissions scored, on average, half a grade boundary higher than real students’ work. The more advanced, abstract reasoning required in third-year exams posed more of a challenge for AI.
… the AI relative grade advantage of AI submissions over real student submissions across all modules!
Image: Scarfe et al., 2024
How should schools and colleges respond?
As schools and colleges continue to embrace digital tools, the rise of artificial intelligence represents a new frontier in academic integrity. Many institutions have already shifted towards more inclusive, take-home assessments, a trend accelerated by the pandemic. While this offers flexibility, more and more studies suggests that AI could exploit these unsupervised environments.
This is important because AI can complete work that meets (or exceeds) expected academic standards, all without student input. For teachers and school leaders, this challenges the value of traditional assessments if the risk of undetected AI cheating is real.
Is your schools acting to safeguard the integrity of its assessment systems?
To resolve AI headaches, assignments completed away from the examination hall or any digital device will remain the norm for some time until clear policies and procedures are established. However, how do we resolve coursework and assignments completed away from the examination hall?
It may require us to incorporate assessments that measure deeper critical thinking and problem-solving, which AI (currently) struggles to replicate. Teachers might also increase supervision during key assessments, explore hybrid models, or use digital tools to help track unusual student performance patterns. The abundance of tracking tools available having increased exponentially. Does your school use any in particular?
Reflection questions for teachers:
- What current assessments vulnerable to AI misuse in schools and colleges?
- Could exams be designed to test skills beyond AI’s current capabilities?
- How would teachers explain the ethical use of AI to students?
- Should schools reconsider the balance between supervised and unsupervised assessments?
- How can teachers integrate AI-related tasks into lessons without compromising learning outcomes?
- What resources can help teachers identify AI-generated submissions?
- What there professional development opportunities for teachers on AI in education?
- What role could collaboration with colleagues play in improving assessment security?
- How can schools balance the benefits of AI tools with safeguarding academic integrity?
- Are there any changes schools should implement to make assessments more AI-proof?
The research concludes:
Today’s AI has been made possible by training AI models on large corpuses of data. Pick a specialist topic and a human expert might do better than GPT4, but in terms of overall “factual knowledge” GPT-4 “knows” vastly more than one human.
Students could cheat undetected using AI – and in doing so, attain a better grade than those who did not cheat!
