Research on AI-based conversational tools has consistently reported positive learner perceptions. However, this alone provides limited insight into whether AI use meaningfully supports language learning. Although some studies have compared AI-assisted and non-AI conditions, this research often offers little explanation of how learners engage with AI during interaction.

This study moves beyond perception- and outcome-focused evaluation by examining what learners do during AI-mediated interaction itself. The study introduces L-CARES (Learner-Centric Analysis of Response and Engagement Sequences), a session-level analytical framework designed to capture observable learner engagement across complete student–AI dialogue sessions. Using transcript data from 18 first-year Japanese English majors who completed weekly chatbot role-play tasks across two academic terms (22 sessions total), L-CARES examines patterns of contingency, agency, attentional repair, elaboration, discourse management, and self-monitoring.

Preliminary application of the framework demonstrates how session-level engagement patterns can be identified and compared across AI-mediated interactions, helping explain variability in learning trajectories despite consistently positive learner perceptions. Rather than evaluating effectiveness in terms of access or usage frequency, the framework foregrounds how learners interact with AI within task constraints. The presentation concludes by discussing implications for CALL research and classroom practice.