Individuals with diabetes face rigorous demands when it comes to managing their health, yet patients sometimes struggle to stay adherent to treatment. CHIP is an AI-based conversational platform that allows patients to report lifestyle factors and receive personalized support for making healthy lifestyle changes. However, detecting patient non-adherence remains a significant challenge in this system, as this can hinder treatment and complicate decision-making for healthcare providers.

This study presents an anomaly detection system designed to identify behavioral changes in diabetes patients through their chatbot interactions. Such shifts have previously been shown to correlate with non-adherence. The approach extracts temporal, frequency, and content features from patient-chatbot conversations and quantifies behavioral variability using entropy to detect deviations from individual baseline patterns. 

The approach was evaluated using synthetic patient-chatbot conversations generated by a locally-hosted large language model, with behavioral shifts manually introduced in the simulated users. The system detected these irregularities with an accuracy of approximately 76% and a recall of around 35%. However, the false positive rate remained high, at around  15%, primarily due to over-flagging in users with naturally high variability. Future improvements could involve machine learning-based personalization to better distinguish between true anomalies and normal variability. With refined detection thresholds, integration into CHIP may enable timely support for patients at risk of non-adherence.