Continuing from our earlier blog post where we discussed the opportunities and risks of AI in education, we would now like to present an overview of specific measures and the ongoing public discourse on this matter. What recommendations do education researchers offer? What insights do computer scientists provide regarding relevant technologies? And what initiatives are practitioners implementing in schools?
Measure A – AI Detection Software
In theory, it sounds promising. You press a button and receive information on whether the text in question was generated by an AI. However, unequivocally, this method is not yet reliable.
- Washington Post: Investigation of AI Detectors Shows Significant Flaws
- In practical terms, the bottom line is that AI detectors simply aren’t effective enough yet. They produce too many false alarms and fail to detect numerous instances accurately.
- CNN: Company behind chatGPT halts its AI detection software development, due to ‘too little accuracy’
- In essence, it appears that even the creator of the AI cannot develop a tool capable of reliably identifying its own use. While there could be other motives for discontinuing development, officially OpenAI expressed a desire to successfully create an AI detector.
- Finally, two distinct scientific studies independently arrive at the conclusion that AI detectors simply don’t work effectively and, in cases of uncertainty, may even exacerbate other issues related to discrimination:
Measure B – Prohibit AI
In theory, this presents an elegant solution. The objective is to ensure that learners continue to develop the skill of independently creating texts. This is easily enforceable in scenarios where writing takes place under supervision. However, enforcing such a ban is likely to be challenging, particularly for texts generated at home. OpenAI, the company behind chatGPT, acknowledges this limitation:
Regardless of the enforceability or sustainability of such a ban, there is also the question of whether one should not integrate this technology—which will likely inevitably become prevalent in various professional domains—into education. A somewhat imperfect but valid comparison is the introduction of calculators in math classes. A comprehensive overview of why the use of generative language models can be highly beneficial for both teachers and learners can be found here:
Finally, it’s essential to note that one cannot learn how to deal with AI if its use is prohibited. The question arises whether this should not become a cultural skill: the ability to compose prompts correctly and effectively, understanding the limitations of AI, and knowing when and how one can enhance quality or overall productivity using it.
Measure C – Adjust assessments
There are undoubtedly various useful types of exams to assess learners’ knowledge and skills. The publication of ChatGPT, especially, puts all forms of ‘homework’ (essays, term papers, theses, etc.) under scrutiny. One approach to address the possibility that authors might use generative language models as a shortcut is to discontinue or severely limit these forms of examination. A notable suggestion comes from the Ministry of Education in Baden-Württemberg, where the Minister of Education proposes the introduction of more oral exams:
This is indeed a viable approach to mitigate the use of AI in exam performance. However, what is overlooked, besides the fact that learners miss the opportunity to learn how to deal with AI, are the learning objectives achieved through ‘homework’: independent work, research skills, detailed critical evaluation of arguments, and the ability to compare them. There are compelling reasons why one might not want this type of assessment to be less prominent in the learning curriculum than it is today. Some insightful case studies illustrate how schools are adapting their exam formalities:
- Business Insider:College professors are considering creative ways to stop students from using AI to cheat
- Some of the solutions include using lockdown browsers, oral exams, case studies, machine learning techniques, and proctoring software.
- NYTimes: ChatGPT, the AI Chatbot That Can Write Anything, Unsettles Universities
- Some universities are responding to the challenge of ChatGPT by revamping their courses to focus more on critical thinking, creativity, and originality.
- Leeds for Learning: ChatGPT, please write a homework policy which takes artificial intelligence into consideration
- A homework policy that takes AI into consideration can help enhance learning outcomes, provide personalised feedback, and streamline grading.
Conclusion
Generative AIs are a challenge for education. They offer the opportunity for even more efficient and effective learning by supporting tasks of both learners and teachers. Equally, they pose the risk of learners cheating on assessments while failing to meet important learning objectives. In this balancing act, Mentafy sees the solution in guiding and documenting the creation of ‘homework’ with sufficient objectivity in the future. Our writing mentor makes it easier for authors to adopt and keep transparent the use of AI, so that in the end, there is a good performance whose academic integrity can be substantiated.
No comment yet, add your voice below!