A new study found that AI advice can make people less willing to admit uncertainty, even when the AI is wrong.
Researchers from École Normale Supérieure, Sapienza University of Rome, and the University of Milano-Bicocca found that access to AI advice reduced people’s willingness to say “I don’t know” from 44 percent to 3 percent. Accuracy also dropped from 27 percent to 9 percent, while confidence rose from 30 percent to 76 percent.
The study was authored by Chiara Marcoccia, Walter Quattrociocchi, and Valerio Capraro.
The researchers designed the tests around questions where AI models usually fail. These included visual details from films, such as the color of a team’s uniform in Bend It Like Beckham . The researchers used Step 3.5 Flash because it was usually wrong on these questions, making it easier to measure whether people were relying on bad AI advice.
Some participants who would have answered correctly on their own changed their answers after asking the AI and became wrong.
The result was not just lower accuracy.
Participants became more confident while giving worse answers. Capraro said people became much worse, with accuracy falling to about one third of the no-AI level, while confidence more than doubled.
The study’s abstract says the researchers ran five experiments with 3,132 participants, including four preregistered experiments and one direct replication. It also says AI advice reduced judgment suspension whether the advice was actively requested or simply shown to users.
The researchers also tested whether financial incentives would make people more careful.
The answer was yes, but only slightly. When accuracy was rewarded, and wrong answers were penalized, willingness to admit uncertainty rose from 3 percent to 8 percent. Accuracy improved from 9 percent to 16 percent. However, both figures remained well below the no-AI baseline of 44 percent for saying “I don’t know” and 27 percent for accuracy.
The findings add to growing concern about what researchers have called “cognitive surrender.”
Earlier Wharton research found that people accepted AI recommendations 92.7 percent of the time when the AI was correct and 79.8 percent of the time even when it was intentionally wrong. The study involved 1,372 participants across 9,593 reasoning tasks.
The new study adds another warning: AI does not only make people accept wrong answers. It may also weaken the habit of recognizing when they do not know enough to answer.
Capraro said the ability to say “I don’t know” matters because it shows people understand the limits of their own knowledge.
He is especially concerned about children, who are growing up with AI tools before fully developing critical thinking skills. The concern comes as AI-generated answers become more common in search, education, and daily decision-making tools.
The broader concern is that most AI products are designed to give fluent answers instead of encouraging uncertainty.
That can be useful when the model is right. But when the model is wrong, users may still treat its answer as reliable. The study suggests that the mere availability of AI advice can shift how people judge their own knowledge.
The main warning is simple: AI may not only give wrong answers. It may also make people less willing to stop, question, and say they do not know.
Get the latest tech news, telecom insights, and product launches wherever you prefer.
Add ProPakistani to Preferred Sources and see more of our stories in Google Search and Top Stories.
Shares
