
Beyond Accuracy: Experts See AI Fact-Checks as Accurate but Less Useful
Chenyan Jia, Apoorva Gondimalla, Angie Zhang, David Joseph Mullings, Alexander Boltz, Min Kyung Lee
Conference Paper
|
April 13, 2026
As misinformation proliferates online, large language models (LLMs) have been proposed as a promising tool to accelerate fact-checking workflows. While LLMs demonstrate strong performance in tasks such as text annotation, their capabilities in generating fact-checking reports remain uncertain. To investigate how media experts evaluate LLM-generated fact-checking reports, we conducted a 2 (Source: human vs. LLM) X 2 (Disclosure of Source: yes or no) between-subjects online experiment with media professionals (N=274). Our analyses reveal that experts perceive LLM-generated reports as significantly less useful than human-written reports; and such differences become larger when participants are not aware of the source. However, LLM-generated fact-checking reports were rated as accurate and logical as human-authored ones. Party affiliation plays a role in predicting perceived logicalness. Our findings advance the understanding of experts’ evaluation of LLM-generated content within the context of misinformation, which provides important theoretical contributions to HCI and communication theories as well as practical implications for the field.
citation
Jia, C., Gondimalla, A., Zhang, A., Mullings, D. J., Boltz, A., & Lee, M. K. (2026). Beyond Accuracy: Experts See AI Fact-Checks as Accurate but Less Useful. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 1-22. https://doi.org/10.1145/3772318.3791625
topics
Artificial Intelligence
Misinformation
Artificial Intelligence
Artificial Intelligence
related work

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness
July 2026
HuskyAI is a web platform on which students develop AI knowledge by working through authentic
Artificial Intelligence
Artificial Intelligence
Working Paper


Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness
July 2026
HuskyAI is a web platform on which students develop AI knowledge by working through authentic
Artificial Intelligence
Artificial Intelligence
Working Paper


Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness
July 2026
HuskyAI is a web platform on which students develop AI knowledge by working through authentic
Artificial Intelligence
Artificial Intelligence
Working Paper

.png)