We investigate children’s ability to use social display rules to infer agents’ otherwise under-determined desires. In Experiment 1, seven-to-ten-year-olds saw a protagonist express one emotional reaction to an event in front of her social partner (the Social Context), and a different expression behind her social partner’s back (the Nonsocial Context). Children were able to use the expression in the Social Context to infer the social partner’s desire and the expression in the Nonsocial Context to infer the protagonist’s desire. This ability increased between ages seven and ten (Experiment 1). When task demands were reduced (Experiment 2), seven-to-eight-year-olds, but not five-to-six-year-olds, succeeded. These results suggest that although it is not easy for observers to infer emotions masked by social display rules, changing emotional expressions between social and non-social contexts allow even children to recover not only the desire of the person displaying the emotions, but also that of the audience.