Research on situated language processing has examined how visually depicted objects or concrete action events inform the comprehension of concrete sentences. By contrast, much less is known about how abstract sentence comprehension interacts with non-linguistic visual information. Moreover, while non-linguistic information can rapidly inform language comprehension when it is related to sentence content through reference or lexical-semantic associations, it is unclear to which extent this is the case when the visual context is ‘non-referential’ (i.e., not related to the sentence through reference or lexical semantic associations). We conducted two eye-tracking reading experiments to address these two open issues. In both experiments, reading times were shorter when sentences about conceptually similar abstract ideas were preceded by objects (words-on-cards in Experiment 1 and blank playing cards in Experiment 2) that were depicted close together (vs. far apart); and when sentences about conceptually dissimilar abstract ideas were preceded by objects that were depicted far apart (vs. close together). This happened rapidly (first-pass reading times) and incrementally (as the sentence unfolded). Thus, (a) comprehension of abstract language can be modulated by non-linguistic visual information (spatial distance between depicted objects) at the sentence level, and (b) online language comprehension can be informed by visual context even in the absence of an overt referential or lexical-semantic link.