Home > Research > Publications & Outputs > Context-dependent decision-making
View graph of relations

Context-dependent decision-making: a simple Bayesian model

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

Context-dependent decision-making: a simple Bayesian model. / Lloyd, Kevin; Leslie, David. S.
In: Interface, Vol. 10, No. 82, 20130069, 20.02.2013, p. 1-13.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

APA

Vancouver

Lloyd K, Leslie DS. Context-dependent decision-making: a simple Bayesian model. Interface. 2013 Feb 20;10(82):1-13. 20130069. doi: 10.1098/rsif.2013.0069

Author

Lloyd, Kevin ; Leslie, David. S. / Context-dependent decision-making : a simple Bayesian model. In: Interface. 2013 ; Vol. 10, No. 82. pp. 1-13.

Bibtex

@article{e91da980d7264058b03832c97b226a6f,
title = "Context-dependent decision-making: a simple Bayesian model",
abstract = "Many phenomena in animal learning can be explained by a context-learning process whereby an animal learns about different patterns of relationship between environmental variables. Differentiating between such environmental regimes or {\textquoteleft}contexts{\textquoteright} allows an animal to rapidly adapt its behaviour when context changes occur. The current work views animals as making sequential inferences about current context identity in a world assumed to be relatively stable but also capable of rapid switches to previously observed or entirely new contexts. We describe a novel decision-making model in which contexts are assumed to follow a Chinese restaurant process with inertia and full Bayesian inference is approximated by a sequential-sampling scheme in which only a single hypothesis about current context is maintained. Actions are selected via Thompson sampling, allowing uncertainty in parameters to drive exploration in a straightforward manner. The model is tested on simple two-alternative choice problems with switching reinforcement schedules and the results compared with rat behavioural data from a number of T-maze studies. The model successfully replicates a number of important behavioural effects: spontaneous recovery, the effect of partial reinforcement on extinction and reversal, the overtraining reversal effect, and serial reversal-learning effects.",
keywords = "Bayesian decision-making, spontaneous recovery, reversal learning, Chinese restaurant process, Thompson sampling",
author = "Kevin Lloyd and Leslie, {David. S.}",
year = "2013",
month = feb,
day = "20",
doi = "10.1098/rsif.2013.0069",
language = "English",
volume = "10",
pages = "1--13",
journal = "Interface",
issn = "1742-5689",
publisher = "Royal Society of London",
number = "82",

}

RIS

TY - JOUR

T1 - Context-dependent decision-making

T2 - a simple Bayesian model

AU - Lloyd, Kevin

AU - Leslie, David. S.

PY - 2013/2/20

Y1 - 2013/2/20

N2 - Many phenomena in animal learning can be explained by a context-learning process whereby an animal learns about different patterns of relationship between environmental variables. Differentiating between such environmental regimes or ‘contexts’ allows an animal to rapidly adapt its behaviour when context changes occur. The current work views animals as making sequential inferences about current context identity in a world assumed to be relatively stable but also capable of rapid switches to previously observed or entirely new contexts. We describe a novel decision-making model in which contexts are assumed to follow a Chinese restaurant process with inertia and full Bayesian inference is approximated by a sequential-sampling scheme in which only a single hypothesis about current context is maintained. Actions are selected via Thompson sampling, allowing uncertainty in parameters to drive exploration in a straightforward manner. The model is tested on simple two-alternative choice problems with switching reinforcement schedules and the results compared with rat behavioural data from a number of T-maze studies. The model successfully replicates a number of important behavioural effects: spontaneous recovery, the effect of partial reinforcement on extinction and reversal, the overtraining reversal effect, and serial reversal-learning effects.

AB - Many phenomena in animal learning can be explained by a context-learning process whereby an animal learns about different patterns of relationship between environmental variables. Differentiating between such environmental regimes or ‘contexts’ allows an animal to rapidly adapt its behaviour when context changes occur. The current work views animals as making sequential inferences about current context identity in a world assumed to be relatively stable but also capable of rapid switches to previously observed or entirely new contexts. We describe a novel decision-making model in which contexts are assumed to follow a Chinese restaurant process with inertia and full Bayesian inference is approximated by a sequential-sampling scheme in which only a single hypothesis about current context is maintained. Actions are selected via Thompson sampling, allowing uncertainty in parameters to drive exploration in a straightforward manner. The model is tested on simple two-alternative choice problems with switching reinforcement schedules and the results compared with rat behavioural data from a number of T-maze studies. The model successfully replicates a number of important behavioural effects: spontaneous recovery, the effect of partial reinforcement on extinction and reversal, the overtraining reversal effect, and serial reversal-learning effects.

KW - Bayesian decision-making

KW - spontaneous recovery

KW - reversal learning

KW - Chinese restaurant process

KW - Thompson sampling

U2 - 10.1098/rsif.2013.0069

DO - 10.1098/rsif.2013.0069

M3 - Journal article

VL - 10

SP - 1

EP - 13

JO - Interface

JF - Interface

SN - 1742-5689

IS - 82

M1 - 20130069

ER -