Pupillometry and Multimodal Processing of Beat Gesture and Pitch Accent: The Eye’s Hole is Greater than the Sum of its Parts
- Laura Morett, Educational Studies in Psychology, Research Methodology, and Conseling, University of Alabama, Tuscaloosa, Alabama, United States
- Jennifer Roche, School of Health Sciences, Kent State University, Kent, Ohio, United States
- Scott Fraundorf, Psychology & LRDC, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- James McPartland, Child Study Center, Yale University, New Haven, Connecticut, United States
AbstractThis study investigated how beat gesture and pitch accent affect the cognitive load of listeners during language comprehension. Evidence from pupillometry and dwell time indicated that more cognitive resources were required to process the combination of these cues than their absence, and they suggest that beat gesture may have required more cognitive resources to process than pitch accent. Additionally, pupil size positively correlated with reaction time and decreased as the task progressed, demonstrating its usefulness as a measure of cognitive processing. These results indicate that viewing gesture in conjunction with speech may increase cognitive load during language processing, and that this increased load may result in enriched representations.