Pupillometry and Multimodal Processing of Beat Gesture and Pitch Accent: The Eye’s Hole is Greater than the Sum of its Parts

AbstractThis study investigated how beat gesture and pitch accent affect the cognitive load of listeners during language comprehension. Evidence from pupillometry and dwell time indicated that more cognitive resources were required to process the combination of these cues than their absence, and they suggest that beat gesture may have required more cognitive resources to process than pitch accent. Additionally, pupil size positively correlated with reaction time and decreased as the task progressed, demonstrating its usefulness as a measure of cognitive processing. These results indicate that viewing gesture in conjunction with speech may increase cognitive load during language processing, and that this increased load may result in enriched representations.

Return to previous page