Automated cognitive modeling with Bayesian active model selection
- Vishal Lall, UC Berkeley, Berkeley, California, United States
- Jordan Suchow, Stevens Institute of Technology, Hoboken, New Jersey, United States
- Gustavo Malkomes, Washington University in St. Louis, St. Louis, Missouri, United States
- Tom Griffiths, Princeton University, Princeton, New Jersey, United States
AbstractBehavioral experiments are often feed-forward: they begin with designing the experiment, and proceed by collecting the data, analyzing it, and drawing inferences from the results. Active learning is an alternative approach where partial experimental data is used to iteratively design subsequent data collection. Here, we study experimental application of Bayesian Active Model Selection (BAMS), which designs trials to discriminate between a set of candidate models. We consider a model set defined by a generative grammar of Gaussian Process kernels that can model both simple functions and complex compositions of them. To validate the method experimentally, we use BAMS to discover how factors such as contrast and number affect numerosity judgements. We compare the rate of convergence of the active-learning method to a baseline passive-learning strategy that selects trials at random. Active learning over a structured model space may increase the efficiency and robustness of behavioral data acquisition and modeling.