Upholding academic integrity and ensuring fairness; AI detection

Dear Colleagues,

As we approach the last few weeks of this semester, instructors are preparing to receive final papers, projects and final examinations.

We are committed to upholding academic integrity and ensuring fairness. As part of that process, we want to highlight resources available to support proctoring and plagiarism detection.

Earlier this month, Turnitin – a vendor we utilize – began providing an artificial intelligence (AI) detector, integrated with Canvas, which offers an estimate of how many sentences in a written submission may have been generated by artificial intelligence.

According to the software developers, educators can use this estimate, as they do with the plagiarism detector output, to determine if further review, inquiry or discussion with the student is needed. There is not a way of opting out of seeing the AI detection score (though students cannot see it). Additionally, the tool analyzes only prose sentences in long-form writing such as paragraphs. It does not analyze languages other than English.

Below are a few points to consider – and some points of caution – as we begin to encounter the AI detection score on Turnitin reports.

We don’t confidently know much about the accuracy of AI writing detectors at this point. AI-generated writing is a novel development, and AI detectors even more so. There has not been enough time for scholarly or professional communities to verify Turnitin claims regarding its detector’s efficacy in terms of both identifying writing that was generated by AI and avoiding false positives.

In fact, Turnitin has included a warning on the AI detection score that the “percentage may not indicate academic misconduct.” Evidence suggests that the accuracy of AI detectors is low, they generate false positives and AI detectors might not flag writing that has been human edited after initially being generated by AI.

Writing that is flagged by Turnitin AI detector cannot be checked against other evidence. In the case of the Turnitin originality detector, passages in the student’s text are linked to writing that can be found either on the internet or in Turnitin’s database of writing. Instructors can view these samples side by side in the Turnitin user interface.

For the AI detector, however, there is no other source to corroborate any flags. The percentage score that the detector presents is the result of a computational analysis based on an emerging (and rapidly evolving) understanding of how AI generates writing using large language models.

The introduction of Turnitin’s AI detector at the end of the semester is potentially disruptive and, as such, should be used with appropriate caution. Students and instructors have not had time to acclimate to the AI detector score in Turnitin reports, and students have been submitting work throughout the semester without the scrutiny of this specific tool.

There is little time left in the semester to properly understand and act upon the AI detection score, and the stakes for assessment and grading can feel particularly high at the end of the term.

Different courses and instructors may have different policies and stances on the use of AI for writing, and students have not had a chance to explore those differences in the context of AI detection scores.

We want to uphold academic integrity while putting students first. Given that the reliability of AI detectors is uncertain, they may introduce unfair bias into the grading process. It is also important to note that students cannot see their AI detection score, even if the instructor has allowed students access to the Turnitin report.

Overall, we want to foster learning environments where students feel that they belong, and where they feel supported and encouraged by their instructors.

AI detector scores should be handled sensitively and with healthy skepticism. If instructors wish to proceed with the determination of an academic offense, the usual procedures outlined in the Senate Rules should be followed.

As we rapidly approach the end of the semester, we hope these points for consideration and caution are helpful as you evaluate the work of our students and work with them to be as successful as possible.

Thank you for all you do for our students and the university. We wish you all the best as the semester winds down.

Sue Nokes, Acting Associate Provost for Faculty Advancement