Paper Breakdown: The Messenger Problem — When a Chatbot's Owner Becomes the Message

Executive summary

• The paper: DiGiuseppe and Robison (2026), "Perceived political bias in large language models reduces persuasive abilities," PNAS Nexus, published online October 9, 2026.
• The question: does the perceived partisanship of an AI system, or of the company behind it, change how persuasive it is?
• The answer: yes. A warning that the chatbot favored the other party reduced its corrective effect by 23% to 28%. Most of the persuasion still survived.
• The lesson for leaders: in AI-mediated communication, the credibility of the platform and its owners is now part of the message.

Why this study matters now

Over the past two years, a line of research has shown that conversational AI can change minds that are usually hard to move. The best-known example is Costello, Pennycook and Rand (2024), who found that short dialogues with a large language model durably reduced belief in conspiracy theories. Results like these have fueled both hope and alarm: chatbots as tools for correcting misinformation, or as instruments of mass persuasion.

What that literature mostly left aside is the oldest variable in persuasion research: who is speaking. DiGiuseppe and Robison, political scientists at Leiden University, isolate exactly that variable.

The design

Sample and setup

• Participants: 2,144 U.S. adults recruited on Prolific between December 2025 and January 2026, with quotas approximating the population by party, age and sex.
• Preregistration: the hypotheses and analysis plan were registered in advance, which limits the room for selective reporting.
• Task: each participant held one of six common economic misconceptions and then had a three-round conversation with a chatbot (OpenAI's GPT-4.1) instructed to correct it.

The four conditions

1. Control: no information about bias.
2. Undirected bias: a general note that AI systems can be politically biased.
3. Out-party bias, light: a note that the chatbot tends to favor the participant's opposing party.
4. Out-party bias, heavy: the same note, plus information about the political affiliations of the company's chief executive and a photo of him with a politician from the opposing party.

Measurement

The main outcome was the change in agreement with the misconception on a 0–4 scale. The authors estimated treatment effects with OLS models that include topic fixed effects and pre-treatment agreement. They also analyzed the 2,010 conversation transcripts with an LLM-as-judge pairwise design, roughly 60 comparisons per transcript, scored with a Bayesian Bradley–Terry model. This second layer tells us not only whether people resisted, but how.

The findings

• The chatbot persuaded in every condition. In the control group, agreement with the misconception fell by 1.20 points on the 0–4 scale.
• Perceived out-party bias weakened it. The light warning reduced the corrective effect by 23%. The heavy warning reduced it by 28%.
• The effect was largest where it mattered most. Among strongly held misconceptions, full reversals fell from 34.4% in the control group to 22.1% under the heavy warning.
• The damage spread beyond the conversation. The heavy warning lowered general trust in AI and support for politicians using AI for policy information.
• It was bipartisan. Democrats and Republicans reacted in similar ways.
• People argued back. In the heavy condition, participants wrote more and were more argumentative, but not more dismissive.

That last pattern is the signature of what Taber and Lodge (2006) called motivated skepticism: people do not simply tune out a distrusted source; they engage it in order to counter-argue. Effort goes up, and persuasion goes down.

The theoretical frame

None of this would have surprised Carl Hovland. In their classic experiment at Yale, Hovland and Weiss (1951) showed that the same message produced more opinion change when attributed to a trustworthy source than to an untrustworthy one. Source credibility became one of the founding concepts of communication science.

The contribution of DiGiuseppe and Robison is to show that this principle transfers to machines, with a twist. A chatbot has no biography of its own, so audiences borrow one: the company that built it and the people who run it. Credibility is no longer a property of a speaker. It is a property of a supply chain.

Limitations worth noting

The authors are explicit about the boundaries of their study:

• one short, one-off interaction rather than repeated use;
• one model, one persuasion prompt and one domain (economic misconceptions);
• one online U.S. sample, with subgroups too small for strong inference by topic or party.

One should also read the headline carefully. A 28% attenuation means that more than 70% of the corrective effect survived even the strongest warning. Perceived bias dents AI persuasion; it does not neutralize it.

Lessons for senior leadership

1. Choose your tools as you would choose a spokesperson. If your organization uses AI to inform citizens, customers or employees, the public reputation of the vendor travels with every answer it gives.
2. Expect the attack on the messenger. In a polarized environment, the easiest way to blunt an AI-delivered correction is to tie the tool to a partisan figure. Plan for that framing before you deploy.
3. Separate the facts from the channel. Corrections that matter should be traceable to independent, citable sources that survive even if the chatbot itself is discredited.
4. Measure trust as an outcome. The study shows spillover effects on trust in AI overall. Any AI communication program should track trust alongside reach and accuracy.

Seventy years after Hovland, the question "who said it?" has not disappeared. It has acquired a new layer: who built the thing that said it.

References

Costello, T. H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science, 385(6714), eadq1814. https://doi.org/10.1126/science.adq1814

DiGiuseppe, M., & Robison, J. (2026). Perceived political bias in large language models reduces persuasive abilities. PNAS Nexus, 5(10), pgag324. https://doi.org/10.1093/pnasnexus/pgag324

Hovland, C. I., & Weiss, W. (1951). The influence of source credibility on communication effectiveness. Public Opinion Quarterly, 15(4), 635–650. https://doi.org/10.1086/266350

Taber, C. S., & Lodge, M. (2006). Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science, 50(3), 755–769. https://doi.org/10.1111/j.1540-5907.2006.00214.x