Workshop Understanding Exploration-Exploitation Trade-offs
- Elizabeth Bonawitz, Psychology, Rutgers University - Newark, Newark, New Jersey, United States
- Alison Gopnik, Department of Psychology, University of California at Berkeley, Berkeley, California, United States
- Celeste Kidd, Brain and Cognitive Science, University of Rochester, Rochester, New York, United States
AbstractAll cognitive systems, almost by definition, both gather information and use that information to make decisions –they explore and exploit. But there is an intrinsic tension between these two aspects of cognition. Exploration is costly-- the resources that go towards information-seeking are not available for action. Moreover, actions have to be taken before all the relevant information is available. When should a cognitive system seek new information and when should it stop seeking information and simply decide how to act? How do people and other organisms resolve this tension between exploration and exploitation? Is there an optimal way of doing so? And how can systems be motivated to seek information when that pursuit is costly and the immediate utility of the information may not be apparent?