April 18th, 2013 @ 2:00pm Tom Schaul (NYU)

Better generalization with forecasts

Abstract:

Prediction has been shown to be a promising principle on which an autonomous reinforcement learning agent might base its knowledge of the world. One particularly powerful predictive representational mechanism introduced recently is the general value function (GVF), which has greater flexibility than previous predictive methods for capturing the regularities the agent may encounter in its environment. We investigate the generalization abilities of these GVFs (or “forecasts”) by generating a focused subclass and testing them within a set of environments in which generalization can be readily measured. We compare the results with a closely related predictive method (PSRs) already shown to have good generalization ability. Our results show that GVFs provide a substantial improvement in generalization, producing features that lead to better value-function approximation than PSRs and better generalization to as-yet-unseen parts of the state space.

(This is joint work with Mark Ring)

seminars/seminaritems/2013-04-17.txt · Last modified: 2013/04/17 12:41 by silberman