JT
J.D. Top
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
3 records found
1
Detecting Patient Deception and Adherence in Diabetes Support Using AI-Generated Conversation Summaries
Leveraging chat summaries to Enhance Doctor-Patient Communication
Unreliable patient self-reporting complicates diabetes management. This study investigates how AI-generated summaries of patient-chatbot conversations can be structured to help healthcare professionals detect deception and non-adherence. To address this, we developed a novel pipeline by first identifying four key behavioral indicators from literature and then using advanced prompt engineering to automatically flag these in structured summaries. The system's effectiveness was evaluated in an annotation experiment using synthetic chat data. The results showed that the summaries did not improve detection accuracy, increased annotation time, and revealed a critically low inter-annotator agreement. These findings highlight the inherent subjectivity and complexity of the detection task, demonstrating that the developed summarization method is not an effective intervention. Although the approach was unsuccessful, this research contributes a novel summarization pipeline, an open-source annotation tool, and a synthetic dataset, establishing a baseline for future work in enhancing doctor-patient communication.
...
Unreliable patient self-reporting complicates diabetes management. This study investigates how AI-generated summaries of patient-chatbot conversations can be structured to help healthcare professionals detect deception and non-adherence. To address this, we developed a novel pipeline by first identifying four key behavioral indicators from literature and then using advanced prompt engineering to automatically flag these in structured summaries. The system's effectiveness was evaluated in an annotation experiment using synthetic chat data. The results showed that the summaries did not improve detection accuracy, increased annotation time, and revealed a critically low inter-annotator agreement. These findings highlight the inherent subjectivity and complexity of the detection task, demonstrating that the developed summarization method is not an effective intervention. Although the approach was unsuccessful, this research contributes a novel summarization pipeline, an open-source annotation tool, and a synthetic dataset, establishing a baseline for future work in enhancing doctor-patient communication.
Enhancing Diabetes Care through AI-Driven Lie Detection in a Diabetes Support System
Testing the validity of lie detection using an SVM model trained on linguistic cues
This paper presents a deception-detection module for a diabetes support system, addressing the challenge of unreliable patient self-reporting and ultimately attempting to improve diabetes care. The research is for a system called CHIP developed by the Hybrid Intelligence project group and TNO. Linguistic cues, such as motion verbs, negation terms, and exclusive terms were identified through a literature study and encoded using custom dictionaries. Cue detection was implemented using the SpaCy NLP library, which identifies and counts cue occurrences. A stylometric machine learning approach was favored over LLMs for explainability and scientific substantiation. In this research, an SVM model, selected for its alignment with prior research (the Mafiascum experiment), was trained on annotated Mafia game data, using normalized cue frequencies as features for the model. Although the SVM achieved high accuracy on truthful messages (F1 between 0.78–0.84), it performed poorly in detecting deception (F1 between 0.21–0.22), likely because of the high frequency of truthful input compared to deceptive input. The low accuracy, along with the model’s domain transferability and performance limitations, suggest further work is needed, particularly with context-specific data and possible integration with LLM-based approaches.
...
This paper presents a deception-detection module for a diabetes support system, addressing the challenge of unreliable patient self-reporting and ultimately attempting to improve diabetes care. The research is for a system called CHIP developed by the Hybrid Intelligence project group and TNO. Linguistic cues, such as motion verbs, negation terms, and exclusive terms were identified through a literature study and encoded using custom dictionaries. Cue detection was implemented using the SpaCy NLP library, which identifies and counts cue occurrences. A stylometric machine learning approach was favored over LLMs for explainability and scientific substantiation. In this research, an SVM model, selected for its alignment with prior research (the Mafiascum experiment), was trained on annotated Mafia game data, using normalized cue frequencies as features for the model. Although the SVM achieved high accuracy on truthful messages (F1 between 0.78–0.84), it performed poorly in detecting deception (F1 between 0.21–0.22), likely because of the high frequency of truthful input compared to deceptive input. The low accuracy, along with the model’s domain transferability and performance limitations, suggest further work is needed, particularly with context-specific data and possible integration with LLM-based approaches.
Entropy-Based Modeling For Detecting Behavioral Anomalies in Users of a Diabetes Lifestyle Management Support System
Identifying non-adherence indicators in a chatbot-based diabetes support system
Individuals with diabetes face rigorous demands when it comes to managing their health, yet patients sometimes struggle to stay adherent to treatment. CHIP is an AI-based conversational platform that allows patients to report lifestyle factors and receive personalized support for making healthy lifestyle changes. However, detecting patient non-adherence remains a significant challenge in this system, as this can hinder treatment and complicate decision-making for healthcare providers.
This study presents an anomaly detection system designed to identify behavioral changes in diabetes patients through their chatbot interactions. Such shifts have previously been shown to correlate with non-adherence. The approach extracts temporal, frequency, and content features from patient-chatbot conversations and quantifies behavioral variability using entropy to detect deviations from individual baseline patterns.
The approach was evaluated using synthetic patient-chatbot conversations generated by a locally-hosted large language model, with behavioral shifts manually introduced in the simulated users. The system detected these irregularities with an accuracy of approximately 76% and a recall of around 35%. However, the false positive rate remained high, at around 15%, primarily due to over-flagging in users with naturally high variability. Future improvements could involve machine learning-based personalization to better distinguish between true anomalies and normal variability. With refined detection thresholds, integration into CHIP may enable timely support for patients at risk of non-adherence.
...
Individuals with diabetes face rigorous demands when it comes to managing their health, yet patients sometimes struggle to stay adherent to treatment. CHIP is an AI-based conversational platform that allows patients to report lifestyle factors and receive personalized support for making healthy lifestyle changes. However, detecting patient non-adherence remains a significant challenge in this system, as this can hinder treatment and complicate decision-making for healthcare providers.
This study presents an anomaly detection system designed to identify behavioral changes in diabetes patients through their chatbot interactions. Such shifts have previously been shown to correlate with non-adherence. The approach extracts temporal, frequency, and content features from patient-chatbot conversations and quantifies behavioral variability using entropy to detect deviations from individual baseline patterns.
The approach was evaluated using synthetic patient-chatbot conversations generated by a locally-hosted large language model, with behavioral shifts manually introduced in the simulated users. The system detected these irregularities with an accuracy of approximately 76% and a recall of around 35%. However, the false positive rate remained high, at around 15%, primarily due to over-flagging in users with naturally high variability. Future improvements could involve machine learning-based personalization to better distinguish between true anomalies and normal variability. With refined detection thresholds, integration into CHIP may enable timely support for patients at risk of non-adherence.