Home > Research > Publications & Outputs > Bayesian Reinforcement Learning in Markovian an...


Text available via DOI:

View graph of relations

Bayesian Reinforcement Learning in Markovian and non-Markovian Tasks

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSNConference contribution/Paperpeer-review

Publication date7/12/2015
Host publicationComputational Intelligence, 2015 IEEE Symposium Series on
Place of PublicationCape Town
Number of pages7
ISBN (Print)9781479975600
<mark>Original language</mark>English


We present a Bayesian reinforcement learning model with a working memory module which can solve some non-Markovian decision processes. The model is tested, and compared against SARSA (lambda), on a standard working-memory task from the psychology literature. Our method uses the Kalman temporal difference framework, And its extension to stochastic state transitions, to give posterior distributions over state-action values. This framework provides a natural mechanism for using reward information to update more than the current state-action pair, and thus negates the use of eligibility traces. Furthermore, the existence of full posterior distributions allows the use of Thompson sampling for action selection, which in turn removes the need to choose an appropriately parameterised action-selection method.