top of page
Wireframe - 8.png

OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation

Ziyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini, Bo Sun, Yakov Bart, Weimin Lyu, Jiri Gesi, Tian Wang, Jing Huang, Yu Su, Upol Ehsan, Malihe Alikhani, Toby Jia-Jun Li, Lydia Chilton, Dakuo Wang

Conference Paper

|

July 2, 2026

Can Large Language models (LLMs) accurately simulate the next web action of a specific user? While LLMs have shown promising capabilities in generating believable human behaviors, evaluating their ability to mimic real user behaviors remains an open challenge, largely due to the lack of high-quality, publicly available datasets that capture both the observable actions and the internal reasoning of an actual human user. To address this gap, we introduce OPeRA, a novel dataset of Observation, Persona, Rationale, and Action collected from real human participants during online shopping sessions. OPeRA is the first public dataset that comprehensively captures: user personas, browser observations, fine-grained web actions, and self-reported just-in-time rationales. We developed both an online questionnaire and a custom browser plugin to gather this dataset with high fidelity. Using OPeRA, we establish the first benchmark to evaluate how well current LLMs can predict a specific user’s next action and rationale with a given persona and history. This dataset lays the groundwork for future research into LLM agents that aim to act as personalized digital twins for human.

citation

Wang, Z., Lu, Y., Li, W., Amini, A., Sun, B., Bart, Y., Lyu, W., Gesi, J., Wang, T., Huang, J., Su, Y., Ehsan, U., Alikhani, M., Li, T. J.-J., Chilton, L., & Wang, D. (2026). OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 43942-43960. https://doi.org/10.18653/v1/2026.acl-long.2033

topics

Artificial Intelligence

Research Methods & Data

Marketing & Advertising

Artificial Intelligence

related work

Rectangle 35

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

July 2026

HuskyAI is a web platform on which students develop AI knowledge by working through authentic

Artificial Intelligence

Artificial Intelligence

Working Paper

Rectangle 35

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

July 2026

HuskyAI is a web platform on which students develop AI knowledge by working through authentic

Artificial Intelligence

Artificial Intelligence

Working Paper

Rectangle 35

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

July 2026

HuskyAI is a web platform on which students develop AI knowledge by working through authentic

Artificial Intelligence

Artificial Intelligence

Working Paper

bottom of page