Nima Esmi
Please Note
8 records found
1
Depression detection benefits from combining neurological and behavioral indicators, yet integrating heterogeneous modalities such as EEG and interview audio remains challenging. We propose a transformer-based multimodal framework that jointly models spectral, spatial, and temporal EEG features alongside linguistic and paralinguistic cues from interviews. By employing synchronized multi-head cross-attention and self-attention mechanisms, the model effectively captures intra- and inter-modal correlations. In addition, a flexible temporal sequence matching strategy reduces EEG channel requirements, enhancing device portability. Evaluated on the MODMA and DAIC-WOZ datasets, our approach achieves superior performance compared to state-of-the-art models, with a 4.7% improvement in accuracy and a 10% increase in precision. These results demonstrate the potential of the proposed framework for accurate, scalable, and cost-effective depression detection in both clinical and real-world settings.
Panoramic X-ray images are essential for dental caries detection, yet they inherently suffer from geometric distortions that complicate accurate early diagnosis. To address this, we propose a Geometry-Biased transformer that explicitly models spherical geometry. Our approach integrates equirectangular relative position embedding, distance-based attention scoring, and equirectangular-aware attention rearrangement to significantly enhance spatial feature representation. This enables the model to effectively capture both local caries lesions and global dental arch structures despite distortions. We rigorously evaluated our model primarily on a hospital-scale dataset, achieving a high accuracy of 94.90% and an AUC of 0.9603. Furthermore, extensive comparative analyses across diverse publicly available dental image datasets demonstrate the superior generalization and competitive performance of our method. Our findings highlight that geometry-aware transformers offer a robust and automated tool, revolutionizing high-precision dental caries detection and potentially other medical imaging diagnostics.
TEREE
Transformer-based emotion recognition using EEG and Eye movement data
Multimodal AI systems increasingly rely on biomedical signals such as EEG and eye movement data for emotion recognition. However, these models face challenges including limited training data, inter-subject variability, session-specific spurious correlations, and incomplete modality representation, all of which reduce generalization and reliability. We propose TEREE, a multimodal transformer-based model that integrates temporal, spatial, and spectral EEG features with eye movement data. To mitigate session-specific artifacts, Bayesian Spurious Correlation Minimization (BSCM) is applied. In addition, a holistic multimodal processing strategy enables robust handling of incomplete data. The model was trained and evaluated using the SEED and SEED-FRA benchmark datasets under one-to-one and multi-to-one transfer paradigms. TEREE achieved state-of-the-art performance, with average multi-to-one transfer accuracies of 97.7% on SEED and 98.8% on SEED-FRA. Ablation studies confirmed that fusing EEG with eye movement features consistently improved accuracy compared to unimodal baselines. Standard deviations across repeated experiments were below 5%, indicating stability. By addressing inter-subject variability, spurious correlations, and incomplete modality issues, TEREE enhances the robustness and generalization of emotion recognition systems. These findings suggest that multimodal transformer-based models can substantially improve the reliability of affective computing applications such as human–computer interaction and mental health monitoring.
Accurate and secure classifying informal documents related to mental disorders is challenging due to factors such as informal language, noisy data, cultural differences, personal information and mixed emotions. Conventional deep learning models often struggle to capture patterns in informal text, as they miss long-range dependencies, explain words and phrases literally, and have difficulty processing non-standard inputs like emojis. To address these limitations, we expand data dimensionality, transforming and fusing textual data and signs from a 1D to a 2D space. This enables the use of pre-trained 2D CNN models, such as AlexNet, Restnet-50, and VGG-16 removing the need to design and train new models from scratch. We apply this approach to a dataset of social media posts to classify informal documents as either related to suicide or non-suicide content. Our results demonstrate high classification accuracy, exceeding 99%. In addition, our 2D visual data representation conceals individual private information and helps explainability.
Accurate psychological text analysis on embedded devices is essential for privacy-preserving mental health monitoring; however, deploying transformer-based models in such constrained environments remains challenging due to limited memory, computational power, and energy budgets. We present a multimodal framework that integrates a lightweight Transformer (DistilBERT) with a CNN-based image branch (ResNet-50) accelerated by the Hailo-8L edge AI board. Text inputs are processed both as sequences and as two-dimensional image-like representations, enabling robust handling of noisy, informal text containing emojis and misspellings. Evaluations on the Depression Twitter dataset and the SST-2 benchmark demonstrate that the fusion model consistently outperforms text-only and image-only baselines, achieving up to 91.6% accuracy while remaining feasible for deployment on a Raspberry Pi 5. Overall, the proposed design offers a practical balance between efficiency and accuracy, enabling real-world, privacy-aware psychological monitoring directly on personal embedded devices.
The COVID-19 disease pandemic spread rapidly worldwide and caused extensive human death and financial losses. Therefore, finding accurate, accessible, and inexpensive methods for diagnosing the disease has challenged researchers. To automate the process of diagnosing COVID-19 disease through images, several strategies based on deep learning, such as transfer learning and ensemble learning, have been presented. However, these techniques cannot deal with noises and their propagation in different layers. In addition, many of the datasets already being used are imbalanced, and most techniques have used binary classification, COVID-19, from normal cases. To address these issues, we use the blind/referenceless image spatial quality evaluator to filter out inappropriate data in the dataset. In order to increase the volume and diversity of the data, we merge two datasets. This combination of two datasets allows multi-class classification between the three states of normal, COVID-19, and types of pneumonia, including bacterial and viral types. A weighted multi-class cross-entropy is used to reduce the effect of data imbalance. In addition, a fuzzy fine-tuned Xception model is applied to reduce the noise propagation in different layers. Quantitative analysis shows that our proposed model achieves 96.60% accuracy on the merged test set, which is more accurate than previously mentioned state-of-the-art methods.