top of page

Measuring Agency-Amplifying AI Literacy: The HuskyAI Coaching Platform for Workforce Readiness

Abstract

HuskyAI is a web platform on which students develop AI knowledge by working through authentic challenges in supervised conversations with a generative model, with every message scored in real time on the Prompt Effectiveness Index (PEI).1 Using a multi-agent architecture, PEI grades the quality of a student's thinking while prompting — across Prompt Structural Quality, Conversation Control, Technical Sophistication, Cognitive Load Management, and Reliance Appropriateness — rather than whether the model produced a correct answer or polished artifact/deliverable. This paper situates the platform within multiple conversations (human agency, the cognitive risks of AI use, AI as an epistemic technology, AI fluency, the science of learning, and evaluative judgement) and shows how the system's architecture is built to meet the theoretical requirements those literatures impose. We argue that PEI provides a metric for detecting whether fluency is exercised in the augmentative mode in which the human remains the author of the collaboration rather than its audience — the conduct recent large-scale evidence (Potts and Sudhof, 2026) identifies as the difference between AI as "a lottery [and] a tool." PEI is a measure, we argue, of human cognitive integrity in an AI context. We describe the agentic evaluator behind PEI, the design choices through which it answers the literature's transparency, calibration, and validity demands, and the limitations of an in-conversation instrument as a learning tool.

Read Abstract

Date Posted

July 1, 2026

Authors

John P. Wihbey, Yash Phalle, Shashank Kadiyala, Caleb Okereke

Themes

Artificial Intelligence

Abstract

HuskyAI is a web platform on which students develop AI knowledge by working through authentic challenges in supervised conversations with a generative model, with every message scored in real time on the Prompt Effectiveness Index (PEI).1 Using a multi-agent architecture, PEI grades the quality of a student's thinking while prompting — across Prompt Structural Quality, Conversation Control, Technical Sophistication, Cognitive Load Management, and Reliance Appropriateness — rather than whether the model produced a correct answer or polished artifact/deliverable. This paper situates the platform within multiple conversations (human agency, the cognitive risks of AI use, AI as an epistemic technology, AI fluency, the science of learning, and evaluative judgement) and shows how the system's architecture is built to meet the theoretical requirements those literatures impose. We argue that PEI provides a metric for detecting whether fluency is exercised in the augmentative mode in which the human remains the author of the collaboration rather than its audience — the conduct recent large-scale evidence (Potts and Sudhof, 2026) identifies as the difference between AI as "a lottery [and] a tool." PEI is a measure, we argue, of human cognitive integrity in an AI context. We describe the agentic evaluator behind PEI, the design choices through which it answers the literature's transparency, calibration, and validity demands, and the limitations of an in-conversation instrument as a learning tool.

Date Posted

July 1, 2026

Authors

John P. Wihbey, Yash Phalle, Shashank Kadiyala, Caleb Okereke

Themes

Artificial Intelligence

How to Cite

Wihbey, J. P., Phalle, Y., Kadiyala, S., & Okereke, C. (2026, July 1). Measuring agency-amplifying AI literacy: The HuskyAI coaching platform for workforce readiness. An applied framework for assessing the quality of learner reasoning in human-AI conversation (AIMES Lab Working Paper, DRAFT). Institute for Information, the Internet, and Democracy, Northeastern University. https://drive.google.com/file/d/1jTlgbdWbClsU0_zkyxnhQmjthwnK3FF3/view

bottom of page