Electroencephalography (EEG) biomarker pipelines are usually assumed to be limited by statistical computation, but much of their cost is data movement: copying, reshaping, and indexing arrays through high-level Python abstractions. We profile the data-reshaping (reduce) stage of the Neurophysiological Biomarker Toolbox, a Python EEG-analysis framework, and find it performs no arithmetic at all: its cost is an eager deep copy of the per-subject container, which also doubles the stage’s peak memory by allocating a full duplicate of the data.
We evaluate three memory-management strategies (zero-copy array views, layout pinning, and lazy materialisation) against an unmodified baseline, verifying that every variant reproduces the baseline’s statistical outputs exactly. Zero-copy views remove the duplicate, cutting reduce-stage peak memory from gigabytes to near zero; this lowers worst-case (tail) latency and, under the parallel load of a cohort sweep, lifts throughput by up to 4.4× by keeping concurrent workers out of swap. Layout pinning and lazy materialisation act only when subjects have repeated sessions, where lazy materialisation cuts reduce-stage peak memory by two orders of magnitude.
The reduce stage is thus effectively eliminated as a cost. The end-to-end speedup is a more modest 1.3×, bounded not by the optimisation but by a separate, arithmetic-bound statistics step that lies outside this paper’s scope and which we flag as the natural next target. The practical recommendation is to eliminate eager deep copies first: a small change that removes the memory doubling and, under parallel load, keeps a cohort sweep out of swap. ... Read more