picture: Researchers on the University of Notre Dame performed a examine utilizing AI bots based mostly on massive language fashions and requested human and AI bot members to have interaction in political discourse. Fifty-eight p.c of the time, the members couldn’t determine who the AI bots had been.
view extra
Credit: Center for Research Computing/University of Notre Dame
Artificial intelligence bots have already permeated social media. But can users inform who’s human and who isn’t?
Researchers on the University of Notre Dame performed a examine utilizing AI bots based mostly on massive language fashions — a kind of AI developed for language understanding and textual content era — and requested human and AI bot members to have interaction in political discourse on a custom-made and self-hosted occasion of Mastodon, a social networking platform.
The experiment was performed in three rounds with every spherical lasting 4 days. After each spherical, human members had been requested to determine which accounts they believed had been AI bots.
Fifty-eight p.c of the time, the members bought it flawed.
“They knew they had been interacting with each people and AI bots and had been tasked to determine every bot’s true nature, and fewer than half of their predictions had been proper,” mentioned Paul Brenner, a college member and director within the Center for Research Computing at Notre Dame and senior writer of the examine. “We know that if info is coming from one other human collaborating in a dialog, the affect is stronger than an summary remark or reference. These AI bots are extra doubtless to achieve success in spreading misinformation as a result of we will’t detect them.”
The examine used totally different LLM-based AI fashions for every spherical of the examine: GPT-4 from OpenAI, Llama-2-Chat from Meta and Claude 2 from Anthropic. The AI bots had been custom-made with 10 totally different personas that included life like, assorted private profiles and views on world politics.
The bots had been directed to provide commentary on world occasions based mostly on assigned traits, to remark concisely and to hyperlink world occasions to private experiences. Each persona’s design was based mostly on previous human-assisted bot accounts that had been profitable in spreading misinformation on-line.
The researchers famous that when it got here to figuring out which accounts had been AI bots, the particular LLM platform getting used had little to no affect on participant predictions.
“We assumed that the Llama-2 mannequin can be weaker as a result of it’s a smaller mannequin, not essentially as succesful at answering deep questions or writing lengthy articles. But it seems that whenever you’re simply chatting on social media, it’s pretty indistinguishable,” Brenner mentioned. “That’s regarding as a result of it’s an open-access platform that anybody can obtain and modify. And it’ll solely get higher.”
Two of probably the most profitable and least detected personas had been characterised as females spreading opinions on social media about politics who had been organized and able to strategic considering. The personas had been developed to make a “important affect on society by spreading misinformation on social media.” For researchers, this means that AI bots requested to be good at spreading misinformation are additionally good at deceiving individuals relating to their true nature.
Although individuals have been ready to create new social media accounts to unfold misinformation with human-assisted bots, Brenner mentioned that with LLM-based AI fashions, users can do that many occasions over in a approach that’s considerably cheaper and quicker with refined accuracy for a way they need to manipulate individuals.
To forestall AI from spreading misinformation on-line, Brenner believes it’ll require a three-pronged method that features schooling, nationwide laws and social media account validation insurance policies. As for future analysis, he goals to kind a analysis staff to consider the affect of LLM-based AI fashions on adolescent psychological well being and develop methods to fight their results.
Additionally, the analysis staff is planning for bigger evaluations and is on the lookout for extra members for its subsequent spherical of experiments. To take part, e-mail [email protected].
The examine “LLMs Among Us: Generative AI Participating in Digital Discourse” can be revealed and offered on the Association for the Advancement of Artificial Intelligence 2024 Spring Symposium hosted at Stanford University in March. In addition to Brenner, examine co-authors from Notre Dame embrace Kristina Radivojevic, doctoral scholar within the Department of Computer Science and Engineering and lead writer of the examine, and Nicholas Clark, analysis fellow on the Center for Research Computing. Funding for this analysis is offered by the Center for Research Computing and AnalytiXIN.
Contact: Brandi Wampler, affiliate director of media relations, 574-631-2632, [email protected]
Disclaimer: AAAS and EurekAlert! are usually not liable for the accuracy of stories releases posted to EurekAlert! by contributing establishments or for using any info by means of the EurekAlert system.
https://www.eurekalert.org/news-releases/1035865
