The understanding that agents have goals, and the ability to infer them, is fundamental in social cognition. However, much of our social understanding goes beyond goal attribution. Drawing on both behavioral studies throughout development, and on the limitations of past models, we propose that humans have a naïve utility calculus to reason about the costs and rewards underlying agents’ goals. We show that the naïve utility calculus model, embedded in a Bayesian framework, can jointly infer the costs and rewards of agents navigating in complex scenarios. Using this model we test humans’ ability to make quantitative cost-reward inferences in scenarios with various sources of costs and rewards. Our results suggest the naïve utility calculus model fits human inferences better than simple goal inference models.