Journal of Computer Science

INCORPORATING PRIOR KNOWLEDGE INTO TEMPORAL DIFFERENCE NETWORKS

Britton Wolfe and James Harpe

DOI : 10.3844/jcssp.2014.2211.2219

Journal of Computer Science

Volume 10, Issue 11

Pages 2211-2219

Abstract

Developing general purpose algorithms for learning an accurate model of dynamical systems from example traces of the system is still a challenging research problem. Predictive State Representation (PSR) models represent the state of a dynamical system as a set of predictions about future events. Our work focuses on improving Temporal Difference Networks (TD Nets), a general class of predictive state models. We adapt the internal structure of the TD Net and we present an improved algorithm for learning a TD Net model from experience in the environment. The new algorithm accepts a set of known facts about the environment and uses those facts to accelerate the learning. These facts can come from another learning algorithm (as in this study) or from a designer’s prior knowledge about the environment. Experiments demonstrate that using the new structure and learning algorithm improves the accuracy of the TD Net models. When tested in an in finite environment, our new algorithm outperforms all of the standard PSR learning algorithms.

Copyright

© 2014 Britton Wolfe and James Harpe. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.