Real-time inference of physical properties in dynamic scenes
- Kevin Smith, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Mario Belledonne, Department of Brain and Cognitive Sciences, MIT, Cambridge, Massachusetts, United States
- Ilker Yildirim, Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Jiajun Wu, EECS, MIT, Cambridge, Massachusetts, United States
- Josh Tenenbaum, Brain and Cognitive Sciences, MIT, Cambridge, Massachusetts, United States
AbstractHuman scene understanding involves not just localizing objects, but also inferring the latent causal properties that give rise to the scene – for instance, how heavy those objects are. These properties can be guessed based on visual features (e.g., material texture), but we can also infer them from how they impact the dynamics of the scene. Furthermore, these inferences are performed rapidly in response to dynamic, ongoing information. Here we propose a computational framework for understanding these inferences, and three models that instantiate this framework. We compare these models to the evolution of human beliefs about object masses. We find that while people’s judgments are generally consistent with Bayesian inference over these latent parameters, the models that best explain human judgments are approximations to this inference that hold and dynamically update beliefs. An earlier version of this work was published in the proceedings of CCN 2018 at https://ccneuro.org/2018/proceedings/1091.pdf.