Automated Discovery of Latent Activity Contexts from Multimodal Sensing for Peer-Interaction Analysis in Preschoolers with Hearing Loss

Abstract

In inclusive classrooms, the physical presence of children with hearing loss (HL) does not guarantee social inclusion with their typically hearing (TH) peers. Self-report is biased and manual observation is too labour-intensive to cover a full school day at fine temporal resolution. This study adapts a multimodal sensing pipeline that combines ultra-wideband (UWB) spatial tracking with Language Environment Analysis (LENA) audio recordings, applied in a single inclusive preschool with 13 children (6 HL, 7 TH).

A Gaussian Mixture Model fit on four room-level features (adult word count, auditory overlap, displacement, and teacher distance) and selected by a stability-aware rule (lowest mean BIC among K values whose cluster assignments reproduce across random initialisations, mean pairwise Adjusted Rand Index >= 0.80) over 2 to 20 components recovers six latent activity contexts, each with a distinct sensor profile. Each context is then given a post-hoc descriptive label drawn from activity types familiar in inclusive preschool classrooms: dispersed transition, peer-driven activity, independent / parallel work, adult-scaffolded peer activity, seated guided work, and whole-class instruction / read-aloud. These labels describe the recovered clusters and are not validated against an external ground truth.

Within each context, a linear mixed model with a per-child random intercept compares HL and TH children on three sensor-derivable behavioural markers of inclusion: peer co-presence (time in spatial groups, where a spatial group is a set of children simultaneously co-present and mutually body-oriented; §5.2.3), vocal participation rate (utterances per minute), and peer affiliation patterns (time in same-diagnosis vs. mixed-diagnosis spatial groups).

The HL/TH signal distributes differently across contexts for each marker. Peer co-presence shows no substantial HL/TH difference in three of six contexts. HL children exceed TH children in independent / parallel work (+7.8% of the minute, q < 0.0001, d = +1.59) and fall below TH children in peer-driven activity (-4.3% of the minute, q = 0.02) and seated guided work (-5.3% of the minute, q = 0.002). Vocal participation rate shows an HL deficit concentrated in the contexts that combine high peer communicative demand with reduced adult scaffolding (peer-driven activity q = 0.03 and adult-scaffolded peer activity q = 0.05, both significant after false-discovery-rate correction; dispersed transition in the same direction but only marginal, q = 0.07), but not in the three contexts where high adult word count or very low auditory overlap slows the pace of exchange. Peer affiliation is the most consistent asymmetry: TH children concentrate grouped time in TH-only spatial groups more than HL children concentrate theirs in HL-only spatial groups in five of six contexts (significantly in four), and HL children spend more grouped time in mixed spatial groups than TH children do across all six (significantly in four). The 6:7 cohort composition produces a baseline TH-above-HL gap of approximately 0.08 on the homophily index under random affiliation; the four significant clusters exceed this baseline (observed gap 0.12-0.19, of which 0.04-0.11 is preference beyond availability), while the two non-significant clusters are within the magnitude expected from cohort composition alone.