Subtle differences in language experience moderate performance on language-based cognitive tests
- Maury Courtland, Zevin Lab, Linguistics Department, University of Southern California, Los Angeles, California, United States
- Aida Mostafazadeh Davani, University of Southern California, Los Angeles, California, United States
- Melissa Reyes, Zevin Lab, University of Southern California, Los Angeles, California, United States
- Leigh Yeh, Computational Social Science Lab, University of Southern California, Los Angeles, California, United States
- Jun Yen Leung, Computational Social Science Lab, University of Southern California, Los Angeles, California, United States
- Brendan Kennedy, University of Southern California, Los Angeles, California, United States
- Morteza Dehghani, University of Southern California, Los Angeles, California, United States
- Jason Zevin, Zevin Lab, University of Southern California, Los Angeles, California, United States
AbstractCognitive tests used to measure individual differences are generally designed with equality in mind: the same "broadly acceptable" items are used for all participants. This has unknown consequences for equity, particularly when a single set of linguistic stimuli are used for a diverse population of language users. We hypothesized that differences in language variety would result in disparities in psycholinguistically meaningful properties of test items in two widely-used cognitive tasks, resulting in large differences in performance. As a proxy for individuals' language use, we administered a self-report survey of media consumption. We identified two substantial clusters from the survey data, roughly orthogonal to a priori groups recruited into the study (university students and members of the surrounding community). We found effects of both population and cluster membership. Comparing item-wise differences between the clusters' language models did not identify specific items driving performance differences.