Many human activities require precise judgments about the physical properties and dynamics of moving objects. Classic work suggests that people's intuitive models of physics are relatively poor and error-prone, based on incorrect general principles or highly simplified heuristics that apply only in special cases. These conclusions seem at odds with the breadth and sophistication of naive physical reasoning in real-world situations. Our work measures the boundaries of people's physical reasoning and tests the richness of intuitive physics knowledge. Participants in four experiments made quantitative judgments about stability and other physical properties of virtual 3D towers. Their judgments correlate highly with a model observer that uses simulations based on realistic physical dynamics and sampling-based approximate probabilistic inference to efficiently and accurately estimate these properties. Several alternative heuristic accounts provide substantially worse fits. We conclude that rich internal physics models are likely to play a key role in guiding human common-sense reasoning.