“We have entered a new era”. These words were spoken by Marc van Oostendorp, professor at Radboud University, the Catholic university in Nijmegen. The teacher is amazed, he almost does not believe what he is going to say: ChatGPT passed the high school exam in the Netherlands. In addition, with an excellent grade (9.8) that, he comments, “practically no student can achieve”. He conducted the experiment himself, to study the dynamics of a future use in teaching the chatbot based on artificial intelligence developed by OpenAI. And the results were published in the online Dutch language journal Neerlandistiek. “I don’t think a year ago anyone would have thought there would be a computer capable of taking the high school final exam,” astonished van Oostendorp, who only wanted to use the free version of the software for this research, rather than the paid one which is preferable.
But can ChatGPT really have the ability to understand complex texts? This is the response of the linguist, originally from Rotterdam: “The discussion is wide because, in a sense, chatbots do not seem to really understand what is presented to them and what they answer correctly. But the debate is very philosophical and until we have alternative tests that measure real understanding and are done well by humans rather than robots, we don’t even know what the practical consequences are.”
The main limitation of the free version, then, is not so much in understanding the text, “but in understanding the conventions of an exam”. In fact, argues the professor, “comparing texts with each other is not feasible, but perhaps mainly because the first text is forgotten when answering questions about the second text. Another issue is that ChatGPT fails to count correctly and gives very long responses. If there’s a limit of, say, 30 words, he’ll cross it. When you proofread, you can only look at the first 30 words and maybe the correct answer isn’t there.”
Another linguist, Roberta D’Alessandro of the University of Utrecht, broke the news of the experiment in Italy. Here is her opinion: “The interesting thing is that the Dutch high school exam includes a part called ‘begrijpend lezen’, ie reading comprehension. The student must understand the implications of the text, what is suggested but not said. Here: ChatGPT managed very well, to understand. He understood everything perfectly. On the French exam, he got a good 8”. But there’s a catch: “ChatGPT failed miserably to graduate vocational schools.” This means, according to her, that “the intelligence is there, but how practical things are done is something that is unknown”.
Source: IL Tempo
John Cameron is a journalist at The Nation View specializing in world news and current events, particularly in international politics and diplomacy. With expertise in international relations, he covers a range of topics including conflicts, politics and economic trends.
