This study investigates the impact of stimulus gender on vocal emotion recognition. To that end, 3 groups of listeners were asked to classify joy, neutral state, fear, anger and sadness in Italian meaningful and nonsensical sentences in 2 settings: a controlled laboratory setting to which a group of Dutch listeners was assigned and a naturalistic setting to which 2 groups respectively of Dutch and Italian listeners were assigned. Two levels of background noise were applied to the sentences to increase the level of task difficulty. Results showed an impact of stimulus gender on emotion classification performance, which seemed dependent on the emotion category: anger was better recognized when spoken by male speakers in both experimental settings, while joy was better recognized when spoken by female speakers by Dutch listeners in the laboratory and Italian listeners in the naturalistic setting, and fear was better recognized when spoken by female speakers in the naturalistic setting.

Introduction: This work aims to understand the contextual factors affecting speech emotion recognition (SER), more specifically the current research investigates whether the identification of vocal emotional expressions of anger, fear, sadness, joy, and neutrality is affected by three factors: (a) the experimental setting, exploring vocal emotion recognition in both a controlled, soundproof laboratory and a more natural listening environment; (b) the effect of stimuli’s background noise: sentences were presented with three different levels of noise to gradually increase the level of difficulty: one clear (no noise) condition and two noise conditions; (c) language familiarity, since the stimuli comprised Italian sentences, and participants were both native (Italians) and Dutch speakers, who did not know Italian. Method: Dutch and Italian participants were involved in a vocal emotion recognition task carried out in two different experimental settings (realistic vs. laboratory). The stimuli were vocal utterances from the Italian EMOVO dataset, conveying emotions like anger, fear, sadness, joy, and neutrality, and were presented in three different noise conditions. Results: Concerning the effect of the experimental setting, even in higher levels of background noise conditions, individuals possess the remarkable ability to discern emotional nuances conveyed through voice. Regarding familiarity with the language, differences in emotion recognition performance between the Italian and Dutch listeners were observed, but the error magnitude was contingent on the emotional categories. Higher noise levels reduced accuracy, but people could still discern emotions, especially prosody. Conclusion: The study highlighted that emotion recognition is influenced by variables such as listening context, background noise, and language familiarity. These results could be useful for developing robust Speech Emotion Recognition (SER) systems and improving human-computer interaction.