Does AI write papers for students?
Automatic text generation tools are increasingly popular because of the ease of their use and the possibilities they offer users. The use of 'machine' assistance for writing texts, particularly as a support for improving the quality and accuracy of content, increases the need for such programmes. Text production by artificial intelligence tools can increase user productivity and creativity. There are, however, attempts to generate all or much of the text and sign it with somebody’s name. Such actions raise legitimate controversy.
Tools such as ChatGPT are also popular among students, but data on the scale of this phenomenon does not yet exist. There was a demand from academics for a tool that would allow them to check whether a student had used AI to help them write their theses. It is for this purpose that the experts of the National Information Processing Institute (OPI) have developed the Uniform Anti-Plagiarism System.
The Uniform Anti-Plagiarism System (JSA) is a modern tool that has been developed in response to the expectations of Polish academics and has brought order to the anti-plagiarism market in Poland. Previously, there were many programmes based on different algorithms and reference databases. By putting JSA at the promoters' disposal in 2019, all engineering, bachelor's, master's and doctoral theses are analysed using the same tool
– says Dr inż.Jarosław Protasiewicz, Director of the National Information Processing Institute.
Thanks to the introduction of JSA, we have ensured that the results of the verification of the originality of theses are more comparable between all centres across Poland than in previous years. Recently, however, a new challenge has emerged: the growing popularity of automatic content generation tools. That is why we immediately set to work to add a function to JSA that will help promoters identify whether a thesis has been written using tools such as the popular ChatGPT
– adds the OPI Director.
It should be emphasised that the introduction of the Uniform Anti-Plagiarism System has been a great success. As recently as in 2016, only 30-40 per cent of theses were analysed for plagiarism. As of 2019, all works are covered by such an obligation. The system is constantly being developed by OPI experts, who are working to expand and modernise it. The addition of a module that helps to detect passages in the text that have been written with artificial intelligence is undoubtedly an important achievement in the fight against plagiarism.
How does the Uniform Anti-Plagiarism System detect the use of artificial intelligence?
This is not the last word from the OPI experts responsible for JSA. They are constantly developing tools to detect AI-written text. As new opportunities to apply artificial intelligence to the development of different content and analysis develop, more advanced detection solutions will also emerge. OPI experts keep their finger on the pulse all the time and improve the system by implementing both off-the-shelf solutions and tools developed at OPI. Information is currently being collected from JSA users across the country about the new solution. Such a study will help OPI experts to introduce new features in JSA that will support the work of promoters even more effectively.
The detection method implemented in the JSA system is based on the hypothesis that the greater the regularity in a text, the more likely it is to have been produced by a language model. This assumption is rooted in the used algorithm, which is able to generate regular and predictable text.
The model used in JSA was trained on a large text dataset. Its operation is most easily explained using the perplexity measure. According to our assumptions, a text produced by a random selection of words should have a high perplexity value, while a very templated and predictable text should have a low perplexity value. In the case of the latter, there is a reasonable suspicion that it was created using artificial intelligence
– says Małgorzata Wartacz, JSA project manager at OPI, who is responsible for developing the system.
We are constantly developing our system and listening to voices from the academic community. We are expanding the number of reference bases and ensuring greater internationalisation of our tool. Of course, the JSA does not give 100 per cent certainty that a text is plagiarised or has been written using artificial intelligence – it only determines the probability. Therefore, the final decision always rests with the promoter. It is them who verify the report received from the JSA and determine whether the thesis is original
– adds the OPI manager.
It should be underlined that, at the moment, there is no solution on the market that can better help answer the question if the text was written by artificial intelligence. Even the developers of the ChatGPT technology do not have such a tool that can accurately assess whether the work has been written by a bot or a human.
JSA – more than 400 institutions and 100,000 users
The role that the Uniform Anti-Plagiarism System plays in higher education is extremely important. The OPI tool is available at every institution in Poland authorised to award a professional title or degree. In the four years since the launch of JSA in January 2019, approx. 1.5 million theses have been verified. On average, there are 12 such papers per promoter. More than 440 institutions have used JSA since its launch and it already has more than 100,000 users. The OPI system provides real support for the work of every promoter in Poland. The importance of JSA's impact on the entire higher education system is best demonstrated by statistics. To date, more than 320,000 papers that have been identified by the system as potential plagiarism have not been accepted by promoters. This means that the JSA has helped prevent more than 320,000 abuses that involve non-compliance with copyright law.
It is important that students realise the seriousness of cheating such as plagiarism and respect the ethical rules of universities. Educational institutions, on the other hand, must be consistent in enforcing regulations to ensure academic integrity. Plagiarism at university is a serious breach of ethical principles and academic regulations. Its consequences can entail serious consequences for a student, including expulsion from studies and not being allowed to defend themselves. Plagiarism using artificial intelligence is more difficult to detect and leads to a loss of individuality and originality in the thesis. It can contribute to the spread of false or unverified information and the drawing of scientific conclusions based on inaccurate data. The addition by OPI's experts of a new feature in JSA, which makes it possible to determine that a text is likely to have been written with the help of artificial intelligence, is undoubtedly an important step in the fight against plagiarism.