Effects of Language on Asynchronized Audiovisual Speech Perception


When people are exposed to asynchronized audiovisual speech, they feel that something is unnatural. We performed tests to determine whether the language used in audiovisual speech affected the observer's perception. In experiment 1, we examined whether Japanese native speakers were more sensitive to audiovisual asynchrony in Japanese audiovisual speech than in English audiovisual speech by using simultaneity judgment tasks. Results showed that the language had no effect on sensitivity to audiovisual asynchrony. In experiment 2, we examined whether audiovisual asynchrony affected the observer's perception of loudness. Results showed that the observer perceived a voice to be quieter when the speaker's face moved after the speaker had spoken. This effect was weaker when the reverse was true and the speaker’s face moved first, followed by the speaker's voice. Moreover, the effect was also not as strong when Japanese native speakers observed asynchronized audiovisual speech in English as opposed to Japanese.

Back to Table of Contents