Cross-sensory comparison of EEG signals for brain-based information retrieval

Translating internal information needs into textual queries poses challenges for information retrieval, particularly for users with physical impairments or ill-defined search intentions. Brain Passage Retrieval (BPR) approaches map EEG signals directly to dense passage representations, bypassing text translation. Whilst recent work reports superior performance with auditory versus visual EEG, these findings emerge from unbalanced experimental conditions where sample sizes, vocabularies, and corpus characteristics differ substantially between modalities, making it unclear whether observed differences reflect genuine neural processing advantages or dataset artefacts. We investigate EEG-based retrieval performance comparing auditory and visual modalities under controlled dataset conditions. Using the Brennan (auditory, 49 subjects) and Nieuwland (visual, 51 subjects) datasets, balanced for vocabulary overlap and sample distributions, we train BPR models with transformer EEG encoders and BERT text encoders via contrastive learning. Under controlled conditions, we observe complementary performance characteristics: audio EEG demonstrates stronger recall (Hit@5: +74%, Hit@10: +140%) whilst visual EEG achieves better precision (Hit@1: +100%). These findings suggest that modality-specific strengths could inform the design of brain-computer interfaces for information retrieval.