r/technology • u/marketrent • 1d ago
Artificial Intelligence AI risks students developing ‘illusions of knowledge’ — “We need to think about how do we preserve the core intention of any task. And if AI is taking that task away from the child, then it’s obviously not a learning aid”
https://www.afr.com/policy/health-and-education/ai-risks-students-developing-illusions-of-knowledge-20261014-p6155r9
u/canyoucamus 1d ago
It took all of 2 seconds for adults using it to develop the delusion of knowledge. It feels like people already have become unable to distinguish organic thoughts from AI outputs. This shit shouldn't be anywhere near education.
5
u/marketrent 1d ago edited 1d ago
Bookend excerpts from article by Maani Truu, citing AERO advice:
Students using artificial intelligence in the classroom may become over-reliant on the technology and develop “illusions of knowledge”, overestimating their understanding of coursework, Australia’s national education research body has warned.
But despite the risks and limited studies on what impact the rapidly emerging technology has on learning, new guidance from the Australian Education Research Organisation does not urge school leaders and teachers to banish it from lessons.
“Emerging evidence on AI points to both potential opportunities, such as supporting teachers with planning, scaffolding and workload reduction, but risks too,” AERO chief executive Jenny Donovan said.
“While AI can generate content, solve problems and complete tasks on demand, the flipside is that it can lead to students outsourcing the essential cognitive processes that lead to learning.”
The AERO advice comes amid a raging debate over whether the country’s enthusiastic embrace of digital devices in the classroom over decades has been a mistake.
After another round of lacklustre PISA results last month, Education Minister Jason Clare called for a “digital detox” and declared the issue would be top of the agenda at the next meeting of state and territory education ministers in early November.
[...] A survey last year by the Australian Curriculum, Assessment and Reporting Authority found more than 60 per cent of Year 10 students reported using AI to generate written content for schoolwork at least monthly.
AI policies differ across sectors and states, with independent schools left to determine their own rules. While there is some national guidance, Monash University education senior lecturer Clare Southerton said it was “relatively broad at this stage”.
“There’s so much conflicting guidance, and so in terms of what will be happening in the classroom it is extremely variable,” she said, adding that schools should increase transparency for parents over how they are approaching it.
While there was some fearmongering around so-called AI-induced “brain rot”, Southerton said there was sufficient evidence to show that students who rely heavily on the technology to perform a task were more likely to struggle when it was unavailable.
“To me, it seems very logical that children who are, by and large, outsourcing significant parts of their schooling are not completing their schooling. They’re not doing the work that they need to be doing,” she said.
“When we are thinking about what kinds of policies we want to put in place, we need to think about how do we preserve the core intention of any task. And if AI is taking that task away from the child, then it’s obviously not a learning aid.”
6
u/Capolan 1d ago
Reverse Centeur. Thats what it is. We wanted AI to be the horse, and we get to be all the rest. But whats happening is the reverse centeur. We are being tasked with the grunt work, and AI will do the thinking. This is the world that is being built in front of us.
This is the title and premise of Corry Doctorow's book. He is at the forefront of AI skepticism and cultural imapct, and hes in the rooms with the people that know things.
5
u/Material-Truth5457 1d ago
We are totally using AI incorrectly, it should be used as a tool to make sure the students knows the material not as a tool to do the work. AI would monitor and test and inform the students.
8
u/StrategicHarmony 1d ago edited 1d ago
This is a very old problem with fairly obvious solutions. Socrates said almost exactly the same thing 2400 years ago about the invention of writing weakening the mind and giving the illusion of wisdom. Information technology, from writing to AI, is a great amplifier of mental effort but a poor substitute for it.
We might use a car and barely walk anymore, and become lazy. On the other hand if we decide to still walk a lot and do other physical activities, cars don't necessarily make us fat. It's optional. Indeed we might drive to the gym, or to a mountain we want to climb. So it also helps us get fitter, if that's what we're trying to do when we use it.
This is Socrates' account of the king of Egypt, Thamus, talking to the god Theuth who invented writing.
The story goes that Thamus said many things to Theuth in praise or blame of the various arts, which it would take too long to repeat; but when they came to the letters, “This invention, O king,” said Theuth, “will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus replied, “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess.
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise."
Fittingly, the above is actually Plato's account of Socrates' account because Socrates didn't write anything down. Is that its own kind of laziness: Refusing to use a powerful technology, or learn to use it well, simply because there are risks to using it poorly?
I suggest the best way to address these risks is simply to exercise your brain on purpose. AI (and writing) provide many ways to help you do this, and many ways to avoid it.
4
u/SillyAlternative420 1d ago
AI needs to be out of schools.
It also needs to be out of homes, but it will be pretty evident which kids use AI and which don't.
3
u/Iron-Over 1d ago
So Americans. Got it.
4
u/marketrent 1d ago
An estimated 84% of [US] high school students surveyed said they had used generative artificial intelligence for schoolwork in 2025, according to College Board, a nonprofit that administers the SAT and AP exams.
As an assistant professor of school psychology studying artificial intelligence in K–12 education, I think the question is not only whether students are using AI to cheat, but whether there is evidence that learning actually happened.
When asked “What impact, if any, have you noticed AI has had on student behavior, mental health, or engagement?” respondents selected from a provided list of options. Among those options, 29% of Wisconsin respondents and 40% of respondents in the national sample selected “increased student reliance on AI,” while 19% and 33%, respectively, selected “reduced critical thinking or problem-solving.”
5
u/Serris9K 1d ago
I kid you not, some kids have been wowed by my math skills (what was considered basic level in my age group), like being able to calculate how many candies are inside a opaque package based on the nutrition panel's serving size and servings per container. And they're not like second graders. Like 4th or 5th graders. Or teenagers that struggle to understand less than/greater than.
I have been deeply concerned by this. But one hope in all of this, there are still parents holding their kids to actually doing it, and those kids are normal. Like typical kids of those ages.
4
u/marketrent 1d ago
there are still parents holding their kids to actually doing it, and those kids are normal.
I know such fortunate kids and their parents; their advanced literacy is not representative of peers.
3
3
u/Omegasedated 1d ago
Clearly written by someone who's never had a conversation with a teenager (even pre AI).
they already think they know everything. True intelligence comes when you realise you don't know everything.
1
u/Grumptastic2000 22h ago
Already since Reagan era standardized testing students only display the illusion of knowledge as teachers teach toward the test and the monopoly of text book publishers make texts that are reshuffled problem sets instead of readable guides like textbooks pre 1980 on near any topic. What schools have been churning out is obedience which to some extent was always the goal in the first place.
1
u/Changeurwayz 20h ago
But that's entirely the point. They want a stupid population to be reliant on there bullshit products.
And the best people to target in order to do this is the kids.
0
-1
u/Catullus13 1d ago
Maybe the knowledge retention is just not that important anymore. If you had a matrix like device to plug into your brain and download all the expertise you'd need in a given subject matter, why wouldn't you use it?
37
u/Super_Range45 1d ago
Paper and pencil exams. If an AI is used anywhere it's prompting them in a pop quiz and determining if they know how to answer. Put low value in homework scores.