(1) Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
(2) Volen Center, Brandeis University, Waltham, MA, USA
(3) Institute of Neuroinformatics, ETH/Univ. of Zurich, Switzerland
Animals and in particular primates have a remarkable ability to modify their interpretation of events depending on the context. The same event can have different meanings in different environments, and the executive control of behavior should take into account all the task-relevant conditions (context) in which the event occurs. Animals can learn what is the task-relevant information from their experience, and eventually abstract rules which guide the behavior. Several sets of rules can be stored simultaneously and animals are able to select rapidly the proper set depending on the context. Recently, experimental [Asaad et al 1998, Wallis et al 2001, Genovesio et al 2005] and theoretical [O'Reilly and Munakata 2000, Loh and Deco 2005] studies have begun to investigate the neural basis of context-dependent behavior. Here we propose a neural mechanism for rule abstraction which produces internal representations of the rules, as attractors of neural dynamics [Amit 1989]. In the proposed scenario every event like the presentation of a contextual cue or a sensory stimulus steers the neural activity towards a previously learned attractor representing the context and containing information about the proper interpretation of future events. In particular the pattern of activation dictated by the context can then affect the final decision of the animal about action selection, in response to a stimulus. When the context changes, a different pre-existent attractor corresponding to the rule in effect is selected, without the need to modify synaptic couplings.
We illustrate this mechanism with a simple example in which visual stimuli are associated to one of two possible motor responses (say two saccadic movements, left and right). In one context the first stimulus (A) should generate Left and the second stimulus should lead to Right in order to receive a reward. In the second context the associations are reversed: A-Right and B-Left are the rewarded associations. The rules to get reward can be expressed in words as follows: Rule 1: "when A is associated to Left, then B is associated to Right" and Rule 2:"when A is associated to Right, then B is associated to Left" There are three fundamental questions that we address: 1) how are the rule representations built? 2) how can the active representation of a context lead to the decision about the motor response? 3) how can contextual cues indicate explicitly what rule is in effect? In order to answer these three questions we first need to describe the neural network which will implement the rules. We assume that there are populations of neurons which are selective to the intented motor response, similarly to [Fusi et al. 2005]. In our case we will group together all the neurons with a preference for Left (population L) and those with a preference for Right (population R). The activation of one of the two groups would express the decision of the monkey to make a saccadic movement to a specific direction. The two populations of neurons compete through a population of inhibitory neurons, as in the decision making network introduced in [Wang 2002]. Each pattern of neural activity in which one population is active (expressing the decision) and the others are inactive, is a global attractor of the neural dynamics. We now consider the heterogeneity across cells. Within each decision population (L or R), we can identify and tag the neurons that have a preference for one specific sensory stimulus. For example, we define neurons within population L with preference to A as those that exhibit the largest response when stimulus A is presented, and we tag them with the label AL. Analogously we can define population BL, again within population L, and AR, BR within population R. This kind of heterogeneity has been observed in prefrontal and in premotor cortex [Asaad et al 1998,Wallis et al 2003]. Rule representations are created by the temporal proximity of events in rewarded trials: for example when rule 1 is in effect, A-Left trials are followed by either A-Left or by B-Right trials. Previous experimental results have shown that neural representations of events that occur in a fixed order tend to merge into a single representation linking neighboring events in a sequence [Miyashita 1988, Griniasty et al 1993, Yakovlev et al 1998]. We then expect that if AL and BR are separated attractors, after long enough sequences of trials in which rule 1 is in effect, AL and BR will merge into a single attractor. Analogously AR and BL will fuse to represent rule 2. Can then the activation of one of these attractors affect the competition between L and R and express the decision which is dictated by the rule in effect? Intuitively this should be possible because the activation of the representation of a rule can bias the competition. Indeed, if rule 1 is in effect, upon the presentation of A, AL and BR receive the recurrent inputs from AL and BR due to the fact that the network is in the attractor corresponding to rule 1. In addition to the these inputs, AL receives also a stronger activation from sensory stimulus A, which can then favor L in the competition. Finally how is the rule selected? Modification of the context can be determined by the feedback the monkey receives (e.g. when the monkey applies one rule and it is not rewarded any longer), or it can explicitly signaled by one or more contextual cues (also called occasion setters in psychology literature) [Schmajuk and Holland 1998]. We shall consider the second case below.
We show that these mechanisms can be implemented in a simple rate model of a network of neurons. We illustrate the simulated network dynamics after learning in the Figure below.
A trial starts from the attractor corresponding to the representation of rule 1 (AL-BR). After one second, a contextual cue is presented to indicate that the rule in effect will be rule 2 (AR-BL). Network's activity steers toward the attractor corresponding to rule 2 (between 1.5 and 2.5s). When stimulus B is shown, the competition between L and R starts, and the previous activation of rule 2 attractor favors L, as dictated by the rule in effect. Notice that the selection of L does not disrupt the information about the rule in effect. Indeed the activity of AR remains high, surviving the decision process and its reset following the execution of the motor response.
The model proposed in this work generates several interesting predictions and provides a new way of interpreting the recorded cortical activity. In our scenario activity that is conventionally named `spontaneous' acquires functional meanings: the activity recorded in any interval between two task relevant events is expected to encode the internal state of the network which represents the rule in effect. This state of persistent activity might reflect factors which are under control in the experiment (e.g. a context dictated by a cue) or which might not be under control (e.g. the mood or the motivation of the animal). We predict that it should be possible to find a correlation between the controlled factors that specify the internal representation of a rule, and the recorded neural activity, in particular in areas like prefrontal cortex for which there is already experimental evidence for rule representation [Miller 2000]. Moreover the presence of this inter-event persistent activity should be correlated with behavior: erroneous trials can be due to a number of reasons, but those mistakes which are due to the wrong interpretation of the sensory stimuli (e.g. when the monkey believes it is in the wrong context) should be correlated with the trials in which the inter-event persistent activity is not observed or in which the representation of the wrong rule is activated. Interestingly the theoretical framework that we propose can be easily extended to other cognitive functions like attention (attending a specific feature of a stimulus to perform a task can be regarded as a rule).
Amit DJ (1989) Modeling brain function. Cambridge University Press.
Asaad WF, Rainer G, Miller EK (1998) Neural activity in the primate prefrontal cortex during associative learning. Neuron 21:1399-1407.
Fusi S, Asaad WF, Miller EK and Wang X-J (2005) A microcircuit model of arbitrary sensori-motor mapping: learning and forgetting on multiple timescales. Soc Neurosci Abstr 813:10
Genovesio A, Brasted PJ, Mitz AR, Wise SP (2005) Prefrontal cortex activity related to abstract response strategies. Neuron 47: 307-320.
Loh M and Deco G (2005) Cognitive flexibility and decision-making in a model of conditional visuomotor associations. Eur J Neurosci 22: 2927-2936.
Miller EK (2000) The prefrontal cortex and cognitive control. Nat Rev Neurosci 1: 59-65.
Miller EK and Cohen JD (2001) An integrative theory of prefrontal cortex function. Ann Rev Neurosci 24: 167-202.
Miyashita Y (1988) Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335: 817-20.
O'Reilly R and Munakata Y (2000) Computational Explorations in Cognitive Neuroscience:
Understanding the Mind by Simulating the Brain. MIT Press.
Schmajuk N and Holland PC (1998). Occasion Setting: Associative learning and cognition in animals
(American Physiological Association, Washington, D.C., 1998).
Wallis JD and Miller EK (2003) From rule to response: neuronal processes in the premotor and prefrontal cortex.J Neurophysiol 90: 1790-1806.
Wallis JD, Anderson KC, Miller EK (2001) Single neurons in prefrontal cortex encode abstract rules.
Nature 411: 953-956.
Wang X-J (2002) Probabilistic decision making by slow reverberation in neocortical circuits. Neuron 36: 955-968.
Yakovlev V, S. Fusi, E. Berman, E. Zohary, (1998) Inter-trial neuronal activity in infero-temporal cortex: a putative vehicle to generate long term association.
Nat Neurosci 1: 310-317.