Speakers involved in a communicative exchange construct an internal model of their addressees and draw upon the model to craft utterances that are likely to be understood. In many real world situations (e.g., when talking to a non-expert, non-native speaker, or a child), this process of audience design involves identifying gaps in the lexical-semantic knowledge of the listener and selecting alternative expressions. We examine speaker adaptation to a listener with incomplete lexical knowledge in the spatial domain, specifically a failure to comprehend the basic terms left/right. Experimental and modeling results provide evidence of rapid adaptation that is modulated by the availability of alternative spatial terms. We consider how our approach relates to recent work in computational pragmatics, and suggest that adaptation to the lexical knowledge of the addressee is an important but relatively understudied topic for future research.