AI detection & academic integrity

A student wrongly accused by Turnitin needed a court ruling to clear his name, highlighting profound AI detection false positives.

In this episode:

• A New York court cleared a student wrongly accused of AI use by Turnitin, highlighting the critical issue of AI detection false positives.
• More than 40% of UK universities lack publicly accessible AI policy, contributing to student anxiety and reluctance to use AI for learning.
• Experts advocate for comprehensive AI assessment design, urging educators to focus on tasks requiring unique human judgment and critical thinking rather than relying on unreliable AI detection tools for academic integrity AI.
• The American Association of Colleges and Universities cautions that AI detection tools should only play a minor role in academic integrity cases due to high false positive rates and potential bias, especially against non-native English speakers.
• Universities must provide clear, consistent guidance on AI use and transparent processes to build trust and ensure fairness in an era of rapid technological change, as unreliable AI detection is not the answer.

Chapters:

• 00:00 — Cold open & welcome
• 00:27 — Orion Newby's case: A shocking example of AI detection false positives
• 01:21 — The scale of AI use and the rise of detection tools
• 02:08 — Why AI detection tools are failing educators and the primary concern of false positives
• 03:10 — Leading universities restrict AI detection due to ethical concerns and bias
• 03:57 — Rethinking AI assessment design for true academic integrity
• 04:51 — The 'Three Ps' of assessment: Product, Process, and Performance
• 05:43 — The urgent need for clear AI policy in universities
• 06:40 — Building trust and consistency in AI use across institutions
• 07:33 — Empowering students: Beyond surveillance to authentic human thinking

How reliable are AI detection tools like Turnitin, GPTZero, and Copyleaks in identifying AI-generated content?
AI detection tools are currently unreliable and prone to significant AI detection false positives, meaning they can falsely accuse students of using AI when they haven't.

What are the risks of using AI detection tools for academic integrity in universities?
The primary risks include false accusations, disproportionate impact on non-native English speakers, student anxiety, and undermining trust in the academic process, as reliable AI detection is not yet possible.

What is an effective approach for universities to maintain academic integrity in the age of AI?
An effective approach involves redesigning assessments to require unique human thinking and critical analysis, fostering transparency in AI policy universities, and moving away from over-reliance on unreliable AI detection tools.

Featuring: Dan Fitzpatrick, Turnitin, GPTZero, Copyleaks, OpenAI, ChatGPT, Edinburgh Napier University, Queen's University Belfast, American Association of Colleges and Universities.

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Transcript

SPEAKER_00 0:00

If this episode makes you think, please let us know in the comments and support us by subscribing and leaving a review. Thank you. Today we are exploring a really critical issue in AI and education, drawn from a special report titled AI and Education by Ema Jacksonobot, published in the Financial Times on July 24, 2026. This report really pulls back the curtain on the growing tensions around AI detection tools in universities and what it means for the future of assessment. Now listen to this opening story from the report. An Adelphi University student named Orion Newby was wrongly accused of violating academic integrity. The university, relying on the Turnaton AI detection tool, got a stunning AI generated score of a hundred percent on his essay. Can you imagine that? A perfect score for a machine when he says he wrote it himself. The report tells us Newby presented evidence from other AI detection tools that actually showed a 0% chance of AI generated content. Yet, the university initially upheld their misconduct finding. It took a New York court ruling in his favor to clear his name, largely because the university failed to follow its own disciplinary procedures. This story immediately highlights the profound and really quite terrifying implications of AI detection, false positives. And this isn't an isolated incident. The report reveals a widespread problem. A recent study by researchers at Edinburgh Napier University, based on a survey of over 6,600 students across seven UK universities, found that a staggering 32% admitted to some level of unpermitted AI use in assessments. That's nearly one in three students, which tells us this isn't a fringe activity anymore. Universities facing this surge in clandestine AI use since OpenAI's ChatGPT hit the scene in 2022, have naturally turned to tools like GPT-0, copy leaks, and as we heard, Turnitin. These tools are designed to analyse text features, things like structure and rhythm, to spot non-human patterns. But as the report emphatically states, their reliability, especially concerning false positives and potential bias, is now seriously under fire. Now here's the crucial first point the report makes clear. AI detection tools simply aren't working as intended. Edward Watson, Vice President for Digital Innovation at the American Association of Colleges and Universities, is quoted saying that the primary concern is the potential for false positives. He cautions that AI detection should at most serve a minor role in academic integrity cases, and faculty should never use AI detection as hard evidence or smoke and gun proof. Think about that. Even the companies are hedging their bets. Annie Cecchitelli, Chief Product Officer at Turnitan, describes their detector as merely a starting point and a data point rather than definitive proof. This isn't just a technical glitch, it's a fundamental misunderstanding of what learning and writing actually are. We're trying to police a product, a final essay, instead of nurturing a process. It's a classic example of outsourcing your doing, not your thinking. But in this case we're trying to outsource the policing of thinking, and it's proving disastrous. Because of these widespread reliability concerns, we're seeing a significant shift. Many institutions, including big names like Vanderbilt, Yale, Johns Hopkins, and Northwestern University, along with the University of Waterloo in Canada, the University of Cape Town in South Africa, and Curtin University in Australia, have either restricted or disabled the use of these detection tools. The report notes that these tools have been criticized for disproportionately delivering false positive results to non-native English speakers, as a widely cited 2023 Stanford study demonstrated. This really underlines the ethical challenges and equity issues at play when we rely on such opaque and potentially biased technologies. So, what's the alternative when reliable AI detection isn't cut in it? The report clearly points to assessment redesign. Judy Williams, pro-vice-chancellor for education and students at Queen's University Belfast, doesn't pull any punches. She states, AI detection tools are not the solution. The technology is still developing, false positives can be unacceptably high, and AI-generated text can easily be modified, making detection unreliable. Her conclusion, and it's a powerful one, is that if we want confidence in academic integrity, AI, the answer is good assessment design. She asks the vital question, what are we actually trying to assess? This is the core of it, isn't it? We need to move beyond just trying to catch students doing something wrong and start design and learning that simply cannot be faked because it demands depth, care, and imagination. This really brings us back to what I call the three Ps of assessment in the AI era. Looking at the product, yes, but also the process, how students got there, including their interaction logs with AI, and finally performance through live demonstrations or presentations. For a year eight geography lesson, for instance, instead of a purely written report, perhaps students could present their findings live, answering questions that probe their unique understanding and critical thinking. The report highlights that some universities are already doing this, redesigning assessments around oral components and focusing on observing the student's process. It's about designing tasks that require genuine cognitive stretch, where the student's unique context, perspective, or judgment is irreplaceable. That's how we teach students not to outsmart machines, but to outthink them. Beyond assessment, there's a big leadership piece here too, and this is where the AI policy universities are trying to grapple with a moving target. Sam Illingworth, a professor of critical AI literacy at Edinburgh Napier University, conducted an audit of 163 UK universities and found that more than 40% had no publicly accessible AI policy. This lack of clarity creates massive student anxiety. A December 2025 study by the UK's Higher Education Policy Institute found that 42% of students were less likely to use AI because they feared being falsely accused of cheating. Think about that for a moment. Students are actively avoiding a powerful learning tool because of the fear of false positives and inconsistent institutional responses. This inconsistency fuels equity issues. As Ursula Lease from the European Students Union points out, she worries that systems operate in a closed way, where students can't see how an AI tool reaches its conclusions, and that the lack of national guidelines in Europe means two students using AI in the same way might be punished differently depending on their country. This isn't how we build trust, is it? We need to provide clear, consistent guidance, using frameworks like the know like trust progression, starting with simply letting students know what the expectations are, building credibility through transparent processes, and ultimately earning their trust by being fair and consistent. Leaders, whether it's a school head or a department head, planning CPD, need to provide clear direction and create champions among staff, building from strength. If you're finding this discussion valuable, please do consider following or subscribing to the podcast for more insights on AI in education. This brings us to a critical question for all educators. The report concludes that many university policies, as Illingworth puts it, promise critical thinking but deliver audit trails. They name support yet deliver surveillance. That really resonated with me. We don't want academic integrity to become a surveillance exercise where the objective is simply to catch students doing something wrong. We want to empower them. Joachim Steinberg, a lawyer specializing in tech litigation, sums it up. The greatest challenge is keeping institutional guidance and disciplinary processes aligned with the pace of technological change. Naimal Thacker, chief executive of GradPilot and AI Tool, agrees that universities need to rethink assessment, welcome disclosure, and be open about policy. This is the way forward. It's an evolution, not a revolution. AI is helping us hold the complexity, so we have capacity for creativity. We need to create an environment where the middle eighty percent of students feel safe to explore AI as a tool for learning and augmentation rather than fearing it. We need to protect and nurture those irreplaceable human domains. Wonder, care, judgment, relationship, imagination, wisdom, and ethics. The true test of our educational systems in the AI era won't be in how well we detect AI output, but in how deeply we inspire and assess authentic human thinking. That's all for today. Thanks for listening.