top of page
Wireframe - 8.png

Beyond Accuracy: Experts See AI Fact-Checks as Accurate but Less Useful

Chenyan Jia, Apoorva Gondimalla, Angie Zhang, David Joseph Mullings, Alexander Boltz, Min Kyung Lee

Conference Paper

|

April 13, 2026

As misinformation proliferates online, large language models (LLMs) have been proposed as a promising tool to accelerate fact-checking workflows. While LLMs demonstrate strong performance in tasks such as text annotation, their capabilities in generating fact-checking reports remain uncertain. To investigate how media experts evaluate LLM-generated fact-checking reports, we conducted a 2 (Source: human vs. LLM) X 2 (Disclosure of Source: yes or no) between-subjects online experiment with media professionals (N=274). Our analyses reveal that experts perceive LLM-generated reports as significantly less useful than human-written reports; and such differences become larger when participants are not aware of the source. However, LLM-generated fact-checking reports were rated as accurate and logical as human-authored ones. Party affiliation plays a role in predicting perceived logicalness. Our findings advance the understanding of experts’ evaluation of LLM-generated content within the context of misinformation, which provides important theoretical contributions to HCI and communication theories as well as practical implications for the field.

citation

Jia, C., Gondimalla, A., Zhang, A., Mullings, D. J., Boltz, A., & Lee, M. K. (2026). Beyond Accuracy: Experts See AI Fact-Checks as Accurate but Less Useful. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 1-22. https://doi.org/10.1145/3772318.3791625

topics

Artificial Intelligence

Misinformation

Artificial Intelligence

Artificial Intelligence

related work

Rectangle 35

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

July 2026

HuskyAI is a web platform on which students develop AI knowledge by working through authentic

Artificial Intelligence

Artificial Intelligence

Working Paper

Rectangle 35

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

July 2026

HuskyAI is a web platform on which students develop AI knowledge by working through authentic

Artificial Intelligence

Artificial Intelligence

Working Paper

Rectangle 35

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

July 2026

HuskyAI is a web platform on which students develop AI knowledge by working through authentic

Artificial Intelligence

Artificial Intelligence

Working Paper

bottom of page