Speakers often refer to context only implicitly when using language. "It's warm outside" could signal warm relative to other days of the year or just the current season (e.g., warm for winter). "Warm" conveys the temperature is high relative to some comparison class, but little is known about how a listener decides upon such a standard of comparison. We formalize how world knowledge and listeners' internal models of speech production can drive the resolution of a comparison class in context. We introduce a Rational Speech Act model and derive two novel predictions from it, which we validate in an experiment that measures listeners' beliefs about the likely comparison class used by a speaker. Our model makes quantitative predictions given prior knowledge for the domains in question. We triangulate this knowledge with a follow-up language task in the same domains, using Bayesian data analysis to infer priors from both data sets.