Using Listener Gaze to Refer in Installments Benefits Understanding

AbstractListener gaze can predict reference resolution as it reflects listeners’ understanding. Further, speakers commonly refer in installments to co-present objects by providing a description incrementally. Here, we investigate whether listener gaze could be utilized to refer incrementally, in spoken installments. Specifically, we implemented a system that generates instructions, describes objects, and reacts to listener gaze with verbal feedback. We compared unambiguous vs. ambiguous instructions supplemented by two levels of feedback specificity: either underspecified (“No, not that one!”) or more informative, contrastive responses (“Further left!”). Our findings show that ambiguous instructions with underspecified feedback did not benefit task performance. In contrast, ambiguous instructions with contrastive feedback (referring in installments) resulted in more efficient interactions. Moreover, this strategy even outperformed the one providing unambiguous instructions.

Return to previous page