My new working paper with Will Hickman is up on SSRN: Do People Trust Humans More Than ChatGPT?
We research whether or not individuals pays for a fact-check on AI writing. ChatGPT will be very helpful, however human readers mustn’t belief each indisputable fact that it stories. Yesterday’s put up was about ChatGPT writing false issues that look actual.
The cause contributors in our experiment may pay for a fact-check is that they earn bonus funds primarily based on whether or not they appropriately determine errors in a paragraph. If contributors consider that the paragraph doesn’t include any errors, they need to not pay for a fact-check. However, if they’ve doubts, it’s rational to pay for a fact-check and earn a smaller bonus, for sure.
Abstract: We discover whether or not individuals belief the accuracy of statements produced by giant language fashions (LLMs) versus these written by people. While LLMs have showcased spectacular capabilities in producing textual content, considerations have been raised relating to the potential for misinformation, bias, or false responses. In this experiment, contributors fee the accuracy of statements beneath totally different info circumstances. Participants who are usually not explicitly knowledgeable of authorship are likely to belief statements they consider are human-written greater than these attributed to ChatGPT. However, when knowledgeable about authorship, contributors present equal skepticism in the direction of each human and AI writers. There is a rise within the fee of pricey fact-checking by contributors who’re explicitly knowledgeable. These outcomes recommend that belief in AI-generated content material is context-dependent.
Our authentic speculation was that folks could be extra trusting of human writers. That turned out to be solely partially true. Participants who are usually not explicitly knowledgeable of authorship are likely to belief statements they consider are human-written greater than these attributed to ChatGPT.
We introduced info to contributors in numerous methods. Sometimes we explicitly advised them about authorship (knowledgeable remedy) and generally we requested them to guess about authorship (uninformed remedy).
This graph (determine 5 in our paper) exhibits that the general fee of fact-checking elevated when topics got extra specific info. Something about being advised {that a} paragraph was written by a human may need aroused suspicion in our contributors. (The children as we speak would say it’s “sus.”) They grew to become much less assured in their very own capability to fee accuracy and due to this fact extra keen to pay for a fact-check. This impact is unbiased of whether or not contributors belief people greater than AI.
We are pondering of fact-checking as usually a great factor, within the context of our earlier work on ChatGPT hallucinations. So, one coverage implication is that sure sorts of labels may cause readers to suppose critically. For instance, Twitter labels automated accounts in order that readers know when content material has been chosen or created by a bot.
Our working paper is at the moment trending on SSRN prime ten lists similar to this one.
Suggested Citation:Buchanan, Joy and Hickman, William, Do People Trust Humans More Than ChatGPT? (November 16, 2023). GMU Working Paper in Economics No. 23-38, Available at SSRN: https://ssrn.com/abstract=4635674
https://economistwritingeveryday.com/2023/12/10/do-people-trust-chatgpt-writing/
