Reasoning, learning, and creativity: frontal lobe function and human decision-making.

Collins A, Koechlin E. · 2012

Read Reasoning, learning, and creativity: frontal lobe function and human decision-making. by Collins A, Koechlin E. (2012) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc3313946.txt.

Introduction

Collins and Koechlin explored how the brain's frontal lobes juggle competing strategies while making decisions, in a 2012 article. Everyday choices unfold in changing settings where no single approach always works. A person must track several possible strategies, judge which is paying off, switch when one fails, and sometimes invent a fresh one altogether.

Drawing on computer models and behavior experiments, the authors argue the frontal system can keep watch over about three or four strategies at once. One part of the system adjusts the strategy in use by its recent rewards, while monitoring functions estimate how trustworthy each option is. The open puzzle is how the brain decides to create new strategies instead of recycling old ones, a problem the authors connect to statistical models that grow their own categories.

Catalog record

CREATORCollins A, Koechlin E.
YEAR2012
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC3313946

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making By Anne Collins, Etienne Koechlin, John P O'Doherty Published in: PLoS Biology DOI: 10.1371/journal.pbio.1001293 ABSTRACT Computational modeling and behavioral experimentation suggest that human frontal lobe function is capable of monitoring three or four concurrent behavioral strategies in order to select the most suitable one during decision-making. INTRODUCTION The ability to adapt to uncertain, changing, and open-ended environments is a hallmark of human intelligence. In such natural situations, decision-making involves exploring, adjusting, and exploiting multiple behavioral strategies (i.e., flexible mappings associating stimuli, actions, and expected outcomes [1]–[4]). This faculty engages the frontal lobe function that manages task sets—that is, active representations of behavioral strategies stored in long-term memory—for driving action [5]–[10]. According to reinforcement learning (RL) models [11],[12], the task set driving ongoing behavior (referred to as the actor) is adjusted according to outcome values for maximizing action utility. Uncertainty monitoring (UM) models [13],[14] further indicate that the frontal executive function infers online the actor reliability—that is, its ability to infer action outcomes—for resetting the actor whenever it becomes unreliable. Moreover, models combining RL and UM suggest that given a fixed collection of concurrent task sets, the frontal function monitors in parallel their relative reliability for adjusting and choosing the most reliable actor [15]–[17]. These models, however, do not explain how the frontal executive function controls an expanding repertoire of behavioral strategies for acting in changing and open-ended environments: that is, how this function decides to create new strategies rather than simply adjusting and switching between previously learned ones. For example, imagine you want to sell lottery tickets to people. After a few trials, you have certainly learned a strategy that appears to be successful for selling your tickets, but your strategy then starts to fail with the next person. You then decide to switch to a new strategy. After adjusting to the new strategy and several successful trials, the new strategy fails again. You may then decide to return to your first strategy or test an entirely new one, and so on. After many trials you have probably learned many different strategies and switch across them and possibly continue to invent new ones. Moreover, among this large collection of behavioral strategies, you may have further learned that several are appropriate with young people, others with older people, some with those wearing hats, others with those holding an umbrella, and so on. How do we learn and manage such an expanding collection of behavioral strategies and decide to create new ones rather than simply adjusting and switching between previously learned ones, possibly according to environmental cues? More formally, little is known about how the frontal executive function continuously arbitrates between (1) adjusting and staying with the current actor set, (2) switching to other learned task sets, and (3) creating new task sets for driving action. This issue raises a computational problem that statistical learning models based on Dirichlet process mixtures address [18]–[20]. However, it remains unclear how the frontal executive function may implement such statistical models, because they critically rely on off-line Bayesian inferences operating on expanding collections of sets that rapidly become computationally intractable [21]. Thus, a fundamental issue is to understand how with limited monitoring resources the human executive function controls online the creation of new behavioral strategies and consequently manages an expanding collection of behavioral strategies for driving action. To clarify this issue, we proposed a computational model of the frontal executive function that controls the creation, learning, storage, retrieval, and selection of behavioral strategies driving action. The model constitutes a biologically plausible, online algorithm. The algorithm approximates Dirichlet process mixtures [19] by combining reinforcement learning, limited Bayesian inferences, and hypothesis testing for arbitrating between adjusting, switching, and creating actor task sets. Consistent with the capacity limit of human working memory [22]–[24], the model assumes that the frontal executive function forms and monitors in parallel only a limited number of concurrent task sets: the executive function monitors only a small part of behavioral strategies stored in long-term memory [22],[23],[25]. As previously suggested [15]–[17], task set reliability is inferred online for choosing the actor sets that drive behavior and adjust to external contingencies. The key assumption is that new task sets are tentatively created and probed as actors whenever no current task sets appear to be reliable. Such probe actors are partly formed by recombining the strategies stored in long-term memory according to external cues [22],[23],[25]. Probe task sets adjust to external contingencies, but may be subsequently discarded when they ultimately appear unnecessary. In the converse case, task set collection is updated with probe task sets: in case the monitoring capacity would be reached, the least recently used task sets are discarded but the associated strategies remain stored in long-term memory. Thus, with limited computing resources, the executive function manages an expanding repertoire of behavioral strategies and controls the selection, learning, retrieval, and creation of behavioral strategies that drive action. We provided a proper computational formulation of this model, named the PROBE model. We tested the model predictions in behavioral experiments inspired from the standard neuropsychological test of frontal executive function, namely the Wisconsin Card Sorting Test [26],[27]. We compared the PROBE model to alternative models, ruling out successively key model assumptions: the notion of hypothesis-testing on task set creation (MAX model), that of task set creation (FORGET model, which encompasses existing models), and the notion of task set monitoring (RL models). We found that unlike these alternative models, the PROBE model predicts human decisions and their variations across individuals. Moreover, the PROBE model that best fits human data is endowed with a monitoring capacity of three or four task sets. RESULTS Standard Model Assumptions We assumed that task sets represent behavioral strategies stored in long-term memory. Each behavioral strategy consists of a selective mapping encoding stimulus-response associations, a predictive mapping encoding expected action outcomes given stimuli [13]–[15], and a contextual mapping encoding external cues predicting task set reliability (see Figure S1 and Materials and Methods). The executive function builds and monitors at most N task sets, a bound reflecting the capacity limit of human working memory [22]–[24]. Consistent with previous studies [13]–[15], task set reliability is evaluated online through forward Bayesian inference: the reliability is inferred before acting according to the perceived volatility of external contingencies [14] and the occurrence of external cues (given contextual mappings) for choosing the actor driving immediate behavior (see below). The actor selective mapping then determines the response to stimulus using a softmax policy (inverse temperature β and noise ε) [11],[15],[28]. Thus, we assumed that in agreement with previous studies (e.g., [6],[29],[30]), selection happens at the level of task sets first, then at the level of actions within task sets. After action, selective mappings then adjust according to outcome values through standard reinforcement learning (learning rate αs) [11],[31], while predictive mappings update outcome predictions [13]. Task set reliability is also updated according to action outcomes (given predictive mappings) and serves to adjust contextual mappings through a classical stochastic gradient descent (contextual learning rate αc). Contextual mappings thus learn the external cues predicting actual reliability (referred to as contextual cues for clarity). PROBE Model The PROBE model assumes that external contingencies are variable and

Read the complete text