Computers having the ability to estimate intentions to speak can improve human-computer interaction. While plenty of research has been done on next-speaker prediction, they differ from intentions to speak since these rely only on the person themselves. Previous research was done on inferring intentions to speak using accelerometer data with some useful results. This paper expands on that research by adding non-verbal vocal behaviour as an additional modality, making the model multimodal. The model is trained on successful intentions to speak, and tested on successful and unsuccessful intentions to speak. Part of the dataset was annotated for unsuccessful intentions to speak and the signals in these annotations were analyzed. In conclusion, using non-verbal vocal behaviour is a much more reliable indicator of successful intentions to speak than accelerometer data. Using a combination of both improves the score slightly, but not significantly. Training on unsuccessful intentions to speak is likely needed to estimate these reliably. Additional modalities could be investigated to possibly improve the model further.

This research paper implements, evaluates, and compares two approaches, a machine learning (ML) approach and a rule-based approach, aimed to estimate intentions to speak. The ML approach trains lexical information extracted from time windows surrounding speech events. The rule-based approach looks for specific keywords or utterances to identify intentions to speak. The results show that the ML approach is a more favourable solution to the problem due to its adaptability and potential for improvement. Sample generation and parameter tweaking showed to be vital to the performance of the model, with its best performance being when it predicted unsuccessful intentions to continue speaking. This study concludes that a machine learning approach can be a viable solution for estimating intentions to continue speaking, with there being future use cases in conversational systems and human-computer interactions.

This research paper proposes a deep learning model to infer segments of speaking intentions using body language captured by a body-worn accelerometer. The objective of the study is to detect instances where individuals exhibit a desire to speak based on their body language cues. The labeling scheme employed is a binary string, with “0” indicating no intention to speak and “1” indicating the presence of an intention to speak on a defined window size of 40 (corresponding to a 2 seconds segment recorded at a frequency of 100 Hz scaled down to 20 binary points per second). In this experiment, a real-life social event dataset was employed, and intensions to speak were manually annotated. A 10-minute segment from the dataset was selected and annotated using the ELAN software. The annotations included two categories: realized intentions, where individuals intended to speak and actually did so, and unrealized intentions, where individuals displayed intentions to speak but did not take their turn. The dataset consisted of 255 segments with realized intentions and 31 segments with unrealized intentions. Additionally, 255 negative samples were included, representing instances where no intention to speak was observed throughout the entire segment. To address the class imbalance inherent in the dataset, the model was evaluated using the Area Under the ROC Curve (AUC) metric using 5- fold cross validation. The model was tested on realized intentions, unrealized intentions, and a combination of both. Its performance was compared against a baseline model, which is assumed to always predict the start of the intention in the middle of the segment. Also, a new model was built that enables classification using varying window sizes as input. The classification task is performed to provide a comparison to another study [1] in which the training segments were predetermined window sizes instead of precisely annotated segments. The results of the study indicate that the deep learning models perform consistently better than the baseline on the segmentation task, and surpasses the performance of the model trained exclusively on window sizes on the classification task. This not only demonstrates the potential of body language as an informative cue for inferring speaking intentions, but also suggests that a supervised learning where intentions are identified with greater precision can lead to a superior outcome.

Everyone has the intention to speak sometimes. Allowing agents to estimate people's intention of speaking can increase conversation efficiency and engagement. The intention of speaking can be expressed by multiple modalities as social cues. In order to add value to existing accelerometer-based research, this research aims to build a model on body postures and explore how it performs on both successful and unsuccessful intention cases. The time segments of successful intentions are automatically generated and the segments of unsuccessful intentions are annotated in a small time period. The model uses poses extracted from the successful intention segments and evaluated on both successful and unsuccessful cases. It is shown that body posture is an effective modality to predict the intention while there are problems like visibility based on camera angles and lack of context while combining data from multiple angles. More modalities are to be added to enhance the model's generalisability and reliability.

This research aims to answer the question whether non-verbal vocal behavior can be used to estimate intention to speak. To answer this question data from a dutch social networking event is used to gather intentions to speak. The intentions to speak are split up in two categories: successful and unsuccessful intentions. The unsuccessful intentions are further split up into two categories: unsuccessful intentions to start speaking and unsuccessful intentions to continue speaking. The perceived unsuccessful intentions to speak are gathered by manually annotating a 10-minute segment of the networking event and successful intentions to speak are automatically extracted using Voice Activity Detection. From the audio, non-verbal vocal features are extracted to train a machine learning model to predict if there is an intention to speak. The model is trained on successful intentions to speak and evaluated on both successful and unsuccessful intentions to speak. From the experiment results it was concluded that the model predicted intention to speak better than random guessing.