Publications
Filter by topic
Large language models (LLMs) challenge Chomsky's long-standing mysterian view of the creative aspect of language use (CALU). By exhibiting fluent, situation-appropriate linguistic behavior and offering concrete mechanistic hypotheses, they provide the first viable scientific models of CALU. We endorse Futrell and Mahowald's call to integrate LLMs into linguistic inquiry and suggest a bolder aim: elucidating the mechanisms underlying linguistic creativity.More ↓Less ↑
Prior research suggests that people often misunderstand visualizations of inflow (e.g., deposits in a banking context) and accumulation (e.g., cumulative savings) in dynamic systems. The present study aimed to examine participants' understanding of accumulation functions and to develop and test the effectiveness of video-based interventions for improving understanding of accumulation. In Experiment 1, we tested the effectiveness of an intervention seated in the context of understanding COVID-19 data. In Experiment 2, we addressed several limitations of Experiment 1 and developed an improved, more general intervention to teach about accumulation in contexts outside of epidemiological data. The two randomized control experiments demonstrated that people fail to understand even simple systems with a single inflow that accumulates over time, with 44%-60% of participants earning a 0% on our pretest measure. However, we also demonstrated that video-based interventions illustrating the relationship between multiple representations of the same underlying data are an effective way to improve the understanding of the relationship between inflow and accumulation, with Experiment 1 suggesting that the effects of our intervention lasted up to 6-7 weeks after testing.More ↓Less ↑
Cognitive tasks originally developed for humans are now increasingly used to study language models. While applying these tasks is often straightforward, interpreting their results can be challenging. In particular, when a model underperforms, it is often unclear whether this results from a limitation in the cognitive ability being tested or a failure to understand the task itself. A recent study argues that GPT 3.5's declining performance on 2-back and 3-back tasks reflects a working memory capacity limit similar to humans (Gong et al., 2024). By analyzing a range of open-source language models of varying performance levels on these tasks, we show that the poor performance is due at least in part to a limitation in task comprehension and task set maintenance. We challenge the best-performing model with progressively harder versions of the task (up to 10-back) and experiment with alternative prompting strategies, before analyzing model attentions. Our larger aim is to contribute to the ongoing conversation around refining methodologies for the cognitive evaluation of language models.More ↓Less ↑
Apes require high volumes of energy-rich foods that tend to be patchily distributed, creating evolutionary pressures for flexible and complex cognition. Several species hunt mobile prey, placing demands on working memory and selecting for sociocognitive abilities such as predicting prey behavior. The mechanisms by which apes overcome foraging and hunting challenges are difficult to elucidate. Field investigations provide rich data sets but lack experimental control, limiting the gamut of questions they can answer, while experiments with captive subjects offer lower generalizability to real-world situations. Virtual environments (VEs) present a compromise, combining experimental specificity with proxies of realistic situations. In this study, chimpanzees and bonobos moved through a three-dimensional VE using a touchscreen. All subjects learned to chase and catch moving rabbits, some exhibiting high success rates even in the presence of large obstacles. Success in trials with a first-person (FP) viewpoint was much higher than in trials presented from overhead, suggesting that the immersive nature of FP trials helped subjects to understand their location in the environment better than when they took a top-down view. Data were analyzed using generative computational agent models, identifying that subjects occasionally employed anticipatory hunting strategies, but more often used a direct chasing strategy. This study validates the use of VEs as an experimental paradigm, demonstrating that apes can understand the behavior of moving agents in situations of varying complexity and that computational modeling can be utilized to delve into behavioral data at a fine-grained level and identify which of several cognitive strategies they fit best.More ↓Less ↑
We demonstrate that Transformer-based neural network language models provide a new foundation for mechanistic theories of sentence processing that seamlessly integrate expectation-based and memory-based accounts. First, we show that the attention mechanism in GPT2-small operates as a kind of cue-based retrieval architecture that is subject to similarity-based interference. Second, we show that it provides accounts of classic memory effects in parsing, including contrasts involving relative clauses and center-embedding. Third, we show that a simple word-by-word entropy metric computed over the internal attention patterns provides an index of memory interference that explains variance in eye-tracking and self-paced reading time measures (independent of surprisal and other predictors) in two natural story reading time corpora.More ↓Less ↑
CogPsych
Human decision making departs from normative principles in systematic ways, including in ethical decisions that impose difficult tradeoffs between the welfare and interests of different individuals or groups. Across three sets of experiments and multiple decision scenarios, we provide evidence that contextual choice reversals arise in multiple types of ethical choice settings, similar to choice reversals observed in other domains ranging from economic gambles to perceptual judgments. Specifically, we find within-participant evidence for attraction effects, in which choices between two options systematically vary as a function of features of a third, dominated and unchosen, option.More ↓Less ↑
Regret Theory (Loomes & Sugden, 1982) is a theory of decision making based on the idea that people consider not only outcome utility, but also future regret or rejoicing, which depends on both the chosen option and foregone options. Regret theory was originally proposed as a theory of choice under uncertainty. Here, we demonstrate that Regret Theory also predicts the widely studied attraction, compromise, and similarity context effects. First, we show that it predicts attraction effects in choice among gamble triples. Second, we apply Regret Theory to non-gamble multi-attribute choice settings and show that it both predicts these context effects and predicts a within-subject dissociation between the compromise and similarity effects previously observed in empirical studies. Regret Theory provides a foundation for a unified account of risky and multi-attribute choice, and we believe the form we present here provides the simplest account to date that explains phenomena in both domains.More ↓Less ↑
This work presents In-Context Policy Iteration, an algorithm for performing Reinforcement Learning (RL), in-context, using foundation models. While the application of foundation models to RL has received considerable attention, most approaches rely on either the curation of expert demonstrations (through manual design or task-specific pretraining) or adaptation to the task of interest using gradient methods (fine-tuning or training of adapter layers). Both techniques have drawbacks: collecting demonstrations is labor-intensive, and algorithms that rely on them do not outperform the experts from which the demonstrations were derived, while gradient techniques are inherently slow, sacrificing the few-shot quality that made in-context learning attractive to begin with. We present an algorithm, ICPI, that learns to perform RL tasks without expert demonstrations or gradients, instead using a policy-iteration method in which the prompt content is the entire locus of learning, iteratively updated from trial-and-error interaction with an RL environment. To eliminate the role of in-weights learning, we demonstrate our algorithm using Codex, a language model with no prior knowledge of the domains on which we evaluate it.More ↓Less ↑
Recently, the Successor Features and Generalized Policy Improvement (SF&GPI) framework has been proposed as a method for learning, composing, and transferring predictive knowledge and behavior. SF&GPI works by having an agent learn predictive representations (SFs) that can be combined for transfer to new tasks with GPI. However, to be effective this approach requires state features that are useful to predict, and these state-features are typically hand-designed. In this work, we present a novel neural network architecture, "Modular Successor Feature Approximators" (MSFA), where modules both discover what is useful to predict, and learn their own predictive representations. We show that MSFA is able to better generalize compared to baseline architectures for learning SFs and modular architectures for learning state representations.More ↓Less ↑
The Option Keyboard (OK) was recently proposed as a method for transferring behavioral knowledge across tasks. OK transfers knowledge by adaptively combining subsets of known behaviors using Successor Features (SFs) and Generalized Policy Improvement (GPI). However, it relies on hand-designed state-features and task encodings which are cumbersome to design for every new environment. In this work, we propose the "Successor Features Keyboard" (SFK), which enables transfer with discovered state-features and task encodings. To enable discovery, we propose the "Categorical Successor Feature Approximator" (CSFA), a novel learning algorithm for estimating SFs while jointly discovering state-features and task encodings. With SFK and CSFA, we achieve the first demonstration of transfer with SFs in a challenging 3D environment where all the necessary representations are discovered. We first compare CSFA against other methods for approximating SFs and show that only CSFA discovers representations compatible with SF&GPI at this scale. We then compare SFK against transfer learning baselines and show that it transfers most quickly to long-horizon tasks.More ↓Less ↑
The Value Learning Task (VLT) is widely used to investigate how acquired value impacts how we perceive and process stimuli. The task consists of a series of trials in which participants attempt to maximize accumulated winnings as they make choices from a pair of presented images associated with probabilistic win, loss, or no-change outcomes. The probabilities and outcomes are initially unknown to the participant and thus the task involves decision making and learning under uncertainty. Despite the symmetric outcome structure for win and loss pairs, people learn win associations better than loss associations. This learning asymmetry could lead to differences when the stimuli are probed in subsequent tasks, compromising inferences about how acquired value affects downstream processing. We investigate the nature of the asymmetry using a standard error-driven reinforcement learning model with a softmax choice rule. Despite having no special role for valence, the model yields the learning asymmetry observed in human behavior, whether the model parameters are set to maximize empirical fit, or task payoff. The asymmetry arises from an interaction between a neutral initial value estimate and a choice policy that exploits while exploring, leading to more poorly discriminated value estimates for loss stimuli. We also show how differences in estimated individual learning rates help to explain individual differences in the observed win-loss asymmetries, and how the final value estimates produced by the model provide a simple account of a post-learning explicit value categorization task.More ↓Less ↑
Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. While it is thought to be essential for robust reasoning in AI systems, conventional approaches require significant training and/or hard-coding of domain knowledge to be applied to benchmark tasks. Inspired by cognitive science research that has found connections between human language and analogy-making, we explore the use of intuitive language-based abstractions to support analogy in AI systems. Specifically, we apply large pre-trained language models (PLMs) to visual Raven's Progressive Matrices (RPM), a common relational reasoning test. By simply encoding the perceptual features of the problem into language form, we find that PLMs exhibit a striking capacity for zero-shot relational reasoning, exceeding human performance and nearing supervised vision-based methods. We explore different encodings that vary the level of abstraction over task features, finding that higher-level abstractions further strengthen PLMs' analogical reasoning. Our detailed analysis reveals insights on the role of model complexity, in-context learning, and prior knowledge in solving RPM tasks.More ↓Less ↑
Research in psychophysics argues that incentivized sensorimotor decisions (such as deciding where to reach to get a reward) maximize expected gain, suggesting that these decisions may be impervious to the cognitive biases and heuristics documented in classical economic decision-making. We tested this hypothesis in two experiments, directly comparing the predictive accuracy of an optimal model and plausible suboptimal models. We found strong evidence that people deviated from the optimal strategy by excessively avoiding loss regions when the potential loss was zero and failing to shift far enough away from loss regions when potential losses outweighed the potential gains. Although allowing nonlinear distortions of value and probability information improved the fit of value-maximizing models, behavior was best described by a model encapsulating a simple heuristic strategy. This suggests that visuomotor decisions are likely influenced by biases and heuristics observed in more classical economic decision-making tasks.More ↓Less ↑
Predictive coding provides a compelling, unified theory of neural information processing, including for language. However, there is insufficient understanding of how predictive models adapt to changing contextual and environmental demands and the extent to which such adaptive processes differ between individuals. Here, we used electroencephalography (EEG) to track prediction error responses during a naturalistic language processing paradigm. In Experiment 1, 45 native speakers of English listened to a series of short passages. Via a speaker manipulation, we introduced changing intra-experimental adjective order probabilities for two-adjective noun phrases embedded within the passages and investigated whether prediction error responses adapt to reflect these intra-experimental predictive contingencies. N400 amplitude at the position of the critical second adjective was used as an outcome measure of prediction error. Results showed that N400 responses attuned to speaker-based surprisal over the course of the experiment, indicating that listeners rapidly adapt their predictive models to reflect local environmental contingencies, in spite of the wealth of prior linguistic experience participants bring to the laboratory. Model adaptation effects were strongest for participants with a steep aperiodic (1/f) slope in resting EEG and low individual alpha frequency (IAF), with idea density (ID) showing a more complex pattern; these results were replicated in a separate sample of 40 participants in Experiment 2. We conclude that predictive mechanisms in language are highly malleable and dynamic, reflecting both the affordances of the present environment and the intrinsic information processing capabilities of the individual.More ↓Less ↑
CogRes:P&I
On April 13, 2021, the CDC announced that the administration of Johnson and Johnson's COVID-19 vaccine would be paused due to a rare blood clotting side effect in ~0.0001% of people given the vaccine. Most people who are hesitant to get a COVID-19 vaccine list potential side effects as their main concern; thus, it is likely that this announcement increased vaccine hesitancy among the American public. Two days after the CDC's announcement, we administered a survey to a group of 2,046 Americans to assess their changes in attitudes toward COVID-19 vaccines. The aim of this study was to investigate whether viewing icon arrays of side effect risk would prevent increases in COVID-19 vaccine hesitancy due to the announcement. We found that using icon arrays to illustrate the small chance of experiencing the blood clotting side effect significantly prevented increases in aversion toward the Johnson and Johnson vaccine as well as all other COVID-19 vaccines.More ↓Less ↑
Across three experiments (N = 1565), we investigated how forecasts about the spread of COVID-19 are impacted by data trends, and whether patterns of misestimation predict adherence to social-distancing guidelines. We also investigated how mode of data presentation influences forecasting of future cases by showing participants data on the number of COVID-19 cases from a 5-week period in either graphical, tabular, or text-only form. We consistently found that people shown tables produced more accurate forecasts compared to people shown line-graphs of the same data; yet people shown line-graphs were more confident in their estimates. These findings suggest that graphs engender false-confidence in the accuracy of forecasts, that people's forecasts of future cases have important implications for their attitudes concerning social distancing, and that tables may be better than graphs for informing the public about the trajectory of COVID-19.More ↓Less ↑
PsycAging
The prospect of loss becomes more salient in later life, and the opportunity to avoid loss is often used to motivate older adults. We examined the effect of loss incentive on working memory in young and older adults. Diffusion-modeling analyses, manipulation of task parameters, and self-report measures identified which aspects of cognitive-motivational processing were most affected within each group. As predicted, loss incentive increased working memory performance and self-reported motivation in young adults, but, consistent with prior work, had the opposite effect in older adults. Diffusion-modeling analyses suggested the primary effect was on the quality of the memory representation (drift rate). The specific impact on drift rate and lack of interactions with set size or retention interval suggest that rather than affecting load-dependent or strategic processes, the effects occur at a relatively global level related to overall task engagement.More ↓Less ↑
How much credit (or blame) should an action taken in a state get for a future reward? This is the fundamental temporal credit assignment problem in Reinforcement Learning (RL). One of the earliest and still most widely used heuristics is to assign this credit based on a scalar coefficient, $\lambda$ (treated as a hyperparameter), raised to the power of the time interval between the state-action and the reward. In this empirical paper, we explore heuristics based on more general pairwise weightings that are functions of the state in which the action was taken, the state at the time of the reward, as well as the time interval between the two. Because these pairwise weight functions are too complex to be treated as hyperparameters, we develop a metagradient procedure for learning these weight functions during the usual RL training of a policy. Our empirical work shows that it is often possible to learn these pairwise weight functions during learning of the policy to achieve better performance than competing approaches.More ↓Less ↑
Learning to flexibly follow task instructions in dynamic environments poses interesting challenges for reinforcement learning agents. We focus here on the problem of learning control flow that deviates from a strict step-by-step execution of instructions -- that is, control flow that may skip forward over parts of the instructions or return backward to previously completed or skipped steps. Demand for such flexible control arises in two fundamental ways: explicitly when control is specified in the instructions themselves (such as conditional branching and looping) and implicitly when stochastic environment dynamics require re-completion of instructions whose effects have been perturbed, or opportunistic skipping of instructions whose effects are already present. We formulate an attention-based architecture that meets these challenges by learning, from task reward only, to flexibly attend to and condition behavior on an internal encoding of the instructions. We test the architecture's ability to learn both explicit and implicit control in two illustrative domains -- one inspired by Minecraft and the other by StarCraft -- and show that the architecture exhibits zero-shot generalization to novel instructions of length greater than those in a training set, at a performance level unmatched by baseline recurrent architectures and an ablation architecture.More ↓Less ↑
First-person object-interaction tasks in high-fidelity, 3D, simulated environments such as the AI2Thor virtual home-environment pose significant sample-efficiency challenges for reinforcement learning (RL) agents learning from sparse task rewards. To alleviate these challenges, prior work has provided extensive supervision via a combination of reward-shaping, ground-truth object-information, and expert demonstrations. In this work, we show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task during task learning with an object-centric relational RL agent. Our key insight is that learning an object-model that incorporates object-attention into forward prediction provides a dense learning signal for unsupervised representation learning of both objects and their relationships, which in turn enables faster policy learning for an object-centric relational RL agent. We demonstrate our agent by introducing a set of challenging object-interaction tasks in the AI2Thor environment where learning with our attentive object-model is key to strong performance, and show that learning with our object-model best closes the performance gap in terms of both learning speed and maximum success rate compared to a relational RL agent equipped with ground-truth object-information.More ↓Less ↑
We show how attention-based transformer language models (using a pre-trained version of GPT-2) provide the basis for a new theoretical account of facilitatory interference effects in subject-verb and reflexive pronoun agreement processing, effects that have played an important role in psycholinguistic theory. The account is grounded in surprisal, but with origins in graded attention and similarity-based interference: we find that the surprisal of the verb or reflexive pronoun predicts facilitatory interference effects in ungrammatical sentences, where a distractor noun phrase that matches the verb or pronoun in number leads to lower surprisal despite not participating in the agreement relation. Through targeted analyses of attention patterns in the transformer, we show that the model behaves in ways consistent with cue-based retrieval theories of sentence processing, providing a new integration of surprisal and similarity-based interference accounts of sentence processing.More ↓Less ↑
Our main contribution in this work is an empirical finding that random General Value Functions (GVFs), i.e., deep action-conditional predictions -- random both in what feature of observations they predict as well as in the sequence of actions the predictions are conditioned upon -- form good auxiliary tasks for reinforcement learning (RL) problems. In particular, we show that random deep action-conditional predictions when used as auxiliary tasks yield state representations that produce control performance competitive with state-of-the-art hand-crafted auxiliary tasks like value prediction, pixel control, and CURL in both Atari and DeepMind Lab tasks. In another set of experiments we stop the gradients from the RL part of the network to the state representation learning part of the network and show, perhaps surprisingly, that the auxiliary tasks alone are sufficient to learn state representations good enough to outperform an end-to-end trained actor-critic baseline.More ↓Less ↑
Diagnosis
The variability in how physicians think: A casebased diagnostic simulation exercise
Little is known about how physician diagnostic thinking unfolds over time when evaluating patients. We designed a case-based simulation to understand how physicians reason, create differential diagnoses, and employ strategies to achieve a correct diagnosis. Between June 2017 and August 2018, hospital medicine physicians at two academic medical centers were presented a standardized case of a patient presenting with chest pain who was ultimately diagnosed with herpes zoster, using an interview format in which case information was presented in predetermined aliquots and participants were asked to think aloud, describing their thoughts and differential diagnoses. Physicians who achieved the correct diagnosis tended to use systems-based or anatomic approaches, rather than focusing on life-threatening diagnoses alone, and employed debiasing strategies.More ↓Less ↑
We agree that combining rational analysis with cognitive bounds, what we previously introduced as Cognitively Bounded Rational Analysis, is a promising and under-used methodology in psychology. We further situate the framework in the literature, and highlight the important issue of a theory of subjective utility, which is not addressed sufficiently clearly in the framework or related previous work.More ↓Less ↑
Prospective memory requires the integration of noisy evidence from perceptual input with evidence from both short-term working memory (WM) and long-term or episodic memory (LTM/EM). We formulate empirical studies of prospective memory as problems of computational rationality, asking how a rational model should integrate noisy perceptual evidence and memory to maximize payoffs in a canonical dual-task paradigm. The model combines reinforcement learning (optimal action selection) with evidence accumulation (optimal Bayesian inference) to derive good decision parameters for optimal task performance, and provides a good qualitative account of key phenomena in the prospective memory literature, including effects of target focality, task emphasis, memory load, and individual differences in working memory capacity.More ↓Less ↑
We present a method for learning intrinsic reward functions to drive the learning of an agent during periods of practice in which extrinsic task rewards are not available. During practice, the environment may differ from the one available for training and evaluation with extrinsic rewards. We refer to this setup of alternating periods of practice and objective evaluation as practice-match, drawing an analogy to regimes of skill acquisition common for humans in sports and games. The agent must effectively use periods in the practice environment so that performance improves during matches. In the proposed method the intrinsic practice reward is learned through a meta-gradient approach that adapts the practice reward parameters to reduce the extrinsic match reward loss computed from matches. We illustrate the method on a simple grid world, and evaluate it in two games in which the practice environment differs from match: Pong with practice against a wall without an opponent, and PacMan with practice in a maze without ghosts. The results show gains from learning in practice in addition to match periods over learning in matches only.More ↓Less ↑
Although bilingual individuals know two languages, research suggests that the languages are not fully separate in the mind, especially evident when a bilingual individual switches languages mid-sentence, indicating that mental representations are, to some degree, overlapping or integrated across the two languages. We used two eye-tracking experiments to examine whether incremental grammatical predictions generated by Spanish-English bilinguals (Experiment 1) and Spanish-as-a-second-language learners (Experiment 2) are language-specific or language-independent. As participants in same-language and mixed-language pairs performed a two-string lexical-decision task, we measured eye fixation times on nouns in grammatical (determiner-noun) and ungrammatical (adverb-noun) contexts to assess whether grammatical gender and number cues from one language generated predictions that carried over into processing of the other language.More ↓Less ↑
Processing difficulty in online language comprehension has been explained in terms of surprisal and entropy reduction. Although both hypotheses have been supported by experimental data, we do not fully understand their relative contributions on processing difficulty. To develop a better understanding, we propose a mechanistic model of perceptual decision making that interacts with a simulated task environment with temporal dynamics. The proposed model collects noisy bottom-up evidence over multiple timesteps, integrates it with its top-down expectation, and makes perceptual decisions, producing processing time data directly without relying on any linking hypothesis. Temporal dynamics in the task environment were determined by a simple finite-state grammar, designed to create situations where the surprisal and entropy reduction hypotheses predict different patterns. After the model was trained to maximize rewards, it developed an adaptive policy, and both surprisal and entropy effects were observed, especially in a measure reflecting earlier processing.More ↓Less ↑
We present a state-trace analysis of sentence ratings elicited by asking participants to evaluate the overall acceptability of a sentence, and ratings elicited by asking participants to focus on structural well-formedness only. Appealing to the literature on "grammatical illusion" sentences, we anticipated that a simple instruction manipulation might prompt people to apply different weights to different aspects of acceptability in the two conditions. The state-trace analysis of the rating data indicates that some dissociations between subjective judgments of acceptability and grammaticality are nonetheless consistent with both judgment types accessing a single underlying factor. These results add to existing comparisons between analytic and probabilistic modeling approaches to predicting rating judgments.More ↓Less ↑
Arguably, intelligent agents ought to be able to discover their own questions so that in learning answers for them they learn unanticipated useful knowledge and skills; this departs from the focus in much of machine learning on agents learning answers to externally defined questions. We present a novel method for a reinforcement learning (RL) agent to discover questions formulated as general value functions or GVFs, a fairly rich form of knowledge representation. Specifically, our method uses non-myopic meta-gradients to learn GVF-questions such that learning answers to them, as an auxiliary task, induces useful representations for the main task faced by the RL agent. We demonstrate that auxiliary tasks based on the discovered GVFs are sufficient, on their own, to build representations that support main task learning, and that they do so better than popular hand-designed auxiliary tasks from the literature. Furthermore, we show, in the context of Atari 2600 videogames, how such auxiliary tasks, meta-learned alongside the main task, can improve the data efficiency of an actor-critic agent.More ↓Less ↑
We study emergent communication between speaker and listener recurrent neural-network agents that are tasked to cooperatively construct a blocks-world target image sampled from a generative grammar of blocks configurations. The speaker receives the target image and learns to emit a sequence of discrete symbols from a fixed vocabulary. The listener learns to construct a blocks-world image by choosing block placement actions as a function of the speaker's full utterance and the image of the ongoing construction. Our contributions are (a) the introduction of a task domain for studying emergent communication that is both challenging and affords useful analyses of the emergent protocols; (b) an empirical comparison of the interpolation and extrapolation performance of training via supervised, (contextual) Bandit, and reinforcement learning; and (c) evidence for the emergence of interesting linguistic properties in the RL agent protocol that are distinct from the other two.More ↓Less ↑
An important achievement in modeling online language comprehension is the discovery of the relationship between processing difficulty and surprisal (Hale, 2001; Levy, 2008). However, it is not clear how structural uncertainty can be represented and updated in a continuous-time continuous-state dynamical system model, a reasonable abstraction of neural computation. In this study, we investigate the Gradient Symbolic Computation (GSC) model (Smolensky et al., 2014) and show how it can dynamically encode and update structural uncertainty via the gradient activation of symbolic constituents. We claim that surprisal is closely related to the amount of change in the optimal activation state driven by a new word input. In a simulation study, we demonstrate that the GSC model implementing a simple probabilistic symbolic grammar can simulate the effect of surprisal on processing time. Our model provides a mechanistic account of the effect of surprisal, bridging between probabilistic symbolic models and subsymbolic connectionist models.More ↓Less ↑
Comp. Interaction
Which NP does all associate with in e.g. ``The pandas, the children all saw''---the pandas, the children, or both? The intuition of adult Mandarin Chinese native speakers regarding the interpretation of the adverbial quantifier dou `all' remains unclear and controversial, and various incommensurate theories of domain selection have been proposed. These studies may have failed to yield clear results because they used testing materials in which the interpretation of dou is confounded with other principles of NP interpretation (e.g. zhexie xiaohai `these children' is truth-functionally synonymous with `all these children'). To address these concerns, we present the first set of experimental studies on adult knowledge and use of syntactic constraints on the quantifier domain of dou. The results support the hypothesis that dou can take one and only one c-commanding NP as its domain, but falsify interesting theoretical accounts that assume a strict locality constraint on dou quantification.More ↓Less ↑
When people search for a target in a novel image they often make use of eye movements to bring the relatively high acuity fovea to bear on areas of interest. The strategies that control these eye movements for visual search have been of substantial scientific interest. In the current article we report a new computational model that shows how strategies for visual search are an emergent consequence of perceptual/motor constraints and approximately optimal strategies. The model solves a Partially Observable Markov Decision Process (POMDP) using deep Q-learning to acquire strategies that optimise the trade-off between speed and accuracy. Results are reported for the Distractor-ratio task.More ↓Less ↑
Monte Carlo Tree Search (MCTS) methods have proven powerful in planning for sequential decision-making problems such as Go and video games, but their performance can be poor when the planning depth and sampling trajectories are limited or when the rewards are sparse. We present an adaptation of PGRD (policy-gradient for reward-design) for learning a reward-bonus function to improve UCT (a MCTS algorithm). Unlike previous applications of PGRD in which the space of reward-bonus functions was limited to linear functions of hand-coded state-action-features, we use PGRD with a multi-layer convolutional neural network to automatically learn features from raw perception as well as to adapt the non-linear reward-bonus function parameters. We also adopt a variance-reducing gradient method to improve PGRD's performance. The new method improves UCT's performance on multiple ATARI games compared to UCT without the reward bonus. Combining PGRD and Deep Learning in this way should make adapting rewards for MCTS algorithms far more widely and practically applicable than before.More ↓Less ↑
Contextual preference reversals occur when a preference for one option over another is reversed by the addition of further options. It has been argued that the occurrence of preference reversals in human behavior shows that people violate the axioms of rational choice and that people are not, therefore, expected value maximizers. In contrast, we demonstrate that if a person is only able to make noisy calculations of expected value and noisy observations of the ordinal relations among option features, then the expected value maximizing choice is influenced by the addition of new options and does give rise to apparent preference reversals. We explore the implications of expected value maximizing choice, conditioned on noisy observations, for a range of contextual preference reversal types---including attraction, compromise, similarity, and phantom effects. These preference reversal types have played a key role in the development of models of human choice. We conclude that experiments demonstrating contextual preference reversals are not evidence for irrationality. They are, however, a consequence of expected value maximization given noisy observations.More ↓Less ↑
It has been proposed that in online sentence comprehension the dependency between a reflexive pronoun such as himself/herself and its antecedent is resolved using exclusively syntactic constraints. Under this strictly syntactic search account, Principle A of the binding theory---which requires that the antecedent c-command the reflexive within the same clause that the reflexive occurs in---constrains the parser's search for an antecedent. The parser thus ignores candidate antecedents that might match agreement features of the reflexive (e.g., gender) but are ineligible as potential antecedents because they are in structurally illicit positions. An alternative possibility accords no special status to structural constraints: in addition to using Principle A, the parser also uses non-structural cues such as gender to access the antecedent. According to cue-based retrieval theories of memory (e.g., Lewis and Vasishth, 2005), the use of non-structural cues should result in increased retrieval times and occasional errors when candidates partially match the cues, even if the candidates are in structurally illicit positions. In this paper, we first show how the retrieval processes that underlie the reflexive binding are naturally realized in the Lewis and Vasishth (2005) model. We present the predictions of the model under the assumption that both structural and non-structural cues are used during retrieval, and provide a critical analysis of previous empirical studies that failed to find evidence for the use of non-structural cues, suggesting that these failures may be Type II errors. We use this analysis and the results of further modeling to motivate a new empirical design that we use in an eye tracking study. The results of this study confirm the key predictions of the model concerning the use of non-structural cues, and are inconsistent with the strictly syntactic search account. These results present a challenge for theories advocating the infallibility of the human parser in the case of reflexive resolution, and provide support for the inclusion of agreement features such as gender in the set of retrieval cues.More ↓Less ↑
It is known that, on average, people adapt their choice of memory strategy to the subjective utility of interaction. What is not known is whether an individual's choices are boundedly optimal. Two experiments are reported that test the hypothesis that an individual's decisions about the distribution of remembering between internal and external resources are boundedly optimal where optimality is defined relative to experience, cognitive constraints, and reward. The theory makes predictions that are tested against data, not fitted to it. The experiments use a no-choice/choice utility learning paradigm where the no-choice phase is used to elicit a profile of each participant's performance across the strategy space and the choice phase is used to test predicted choices within this space. They show that the majority of individuals select strategies that are boundedly optimal. Further, individual differences in what people choose to do are successfully predicted by the analysis. Two issues are discussed: (a) the performance of the minority of participants who did not find boundedly optimal adaptations, and (b) the possibility that individuals anticipate what, with practice, will become a bounded optimal strategy, rather than what is boundedly optimal during training.More ↓Less ↑
For Markov decision processes with long horizons (i.e., discount factors close to one), it is common in practice to use reduced horizons during planning to speed computation. However, perhaps surprisingly, when the model available to the agent is estimated from data, as will be the case in most real-world problems, the policy found using a shorter planning horizon can actually be better than a policy learned with the true horizon. In this paper we provide a precise explanation for this phenomenon based on principles of learning theory. We show formally that the planning horizon is a complexity control parameter for the class of policies to be learned. In particular, it has an intuitive, monotonic relationship with a simple counting measure of complexity, and that a similar relationship can be observed empirically with a more general and data-dependent Rademacher complexity measure. Each complexity measure gives rise to a bound on the planning loss predicting that a planning horizon shorter than the true horizon can reduce overfitting and improve test performance, and we confirm these predictions empirically.More ↓Less ↑
Motivated by vision-based reinforcement learning (RL) problems, in particular Atari games from the recent benchmark Arcade Learning Environment (ALE), we consider spatio-temporal prediction problems where future (image-)frames are dependent on control variables or actions as well as previous frames. While not composed of natural scenes, frames in Atari games are high-dimensional in size, can involve tens of objects with one or more objects being controlled by the actions directly and many other objects being influenced indirectly, can involve entry and departure of objects, and can involve deep partial observability. We propose and evaluate two deep neural network architectures that consist of encoding, action-conditional transformation, and decoding layers based on convolutional neural networks and recurrent neural networks. Experimental results show that the proposed architectures are able to generate visually-realistic frames that are also useful for control over approximately 100-step action-conditional futures in some games. To the best of our knowledge, this paper is the first to make and evaluate long-term predictions on high-dimensional video conditioned by control inputs.More ↓Less ↑
The combination of modern Reinforcement Learning and Deep Learning approaches holds the promise of making significant progress on challenging applications requiring both rich perception and policy-selection. The Arcade Learning Environment (ALE) provides a set of Atari games that represent a useful benchmark set of such applications. A recent breakthrough in combining model-free reinforcement learning with deep learning, called DQN, achieves the best real-time agents thus far. Planning-based approaches achieve far higher scores than the best model-free approaches, but they exploit information that is not available to human players, and they are orders of magnitude slower than needed for real-time play. Our main goal in this work is to build a better real-time Atari game playing agent than DQN. The central idea is to use the slow planning-based agents to provide training data for a deep-learning architecture capable of real-time play. We proposed new agents based on this idea and show that they outperform DQN.More ↓Less ↑
Utility maximization is a key element of a number of theoretical approaches to explaining human behavior. Among these approaches are rational analysis, ideal observer theory, and signal detection theory. While some examples of these approaches define the utility maximization problem with little reference to the bounds imposed by the organism, others start with, and emphasize approaches in which bounds imposed by the information processing architecture are considered as an explicit part of the utility maximization problem. These latter approaches are the topic of this issue of the journal.More ↓Less ↑
In this paper we show how abstractions can help UCT's performance. Ideal abstractions are homomorphisms because they preserve optimal policies, but they rarely exist, and are computationally hard to find even when they do. We show how a combination of (i) finding local abstractions in the layered-DAG MDP induced by a set of UCT trajectories (rather than finding abstractions in the global MDP), and (ii) accepting approximate homomorphisms, leads to greater prevalence of good abstractions and makes them computationally easier to find. We propose an algorithm for finding abstractions in UCT planning and derive a lower bound on its performance. We show empirically that it improves performance on illustrative tasks, and on the game of Othello.More ↓Less ↑
We propose a framework for including information- processing bounds in rational analyses. It is an application of bounded optimality (Russell & Subramanian, 1995) to the challenges of developing theories of mechanism and behavior. The framework is based on the idea that behaviors are generated by cognitive mechanisms that are adapted to the structure of not only the environment but also the mind and brain itself. We call the framework computational rationality to emphasize the incorporation of computational mechanism into the definition of rational action. Theories are specified as optimal program problems, defined by an adaptation environment, a bounded machine, and a utility function. Such theories yield different classes of explanation, depending on the extent to which they emphasize adaptation to bounds, and adaptation to some ecology that differs from the immediate local environment. We illustrate this variation with examples from three domains: visual attention in a linguistic task, manual response ordering, and reasoning. We explore the relation of this framework to existing ``levels'' approaches to explanation, and to other optimality-based modeling approaches.More ↓Less ↑
IEEE TAMD
Following work on designing optimal rewards for single agents, we define a multiagent optimal rewards problem (ORP) in cooperative (specifically, common-payoff or team) settings. This new problem solves for individual agent reward functions that guide agents to better overall team performance relative to teams in which all agents guide their behavior with the same given team-reward function. We present a multiagent architecture in which each agent learns good reward functions from experience using a gradient-based algorithm in addition to performing the usual task of planning good policies (except in this case with respect to the learned rather than the given reward function). Multiagency introduces the challenge of nonstationarity: because the agents learn simultaneously, each agent's reward-learning problem is nonstationary and interdependent on the other agents evolving reward functions. We demonstrate on two simple domains that the proposed architecture outperforms the conventional approach in which all the agents use the same given team-reward function (even when accounting for the resource overhead of the reward learning); that the learning algorithm performs stably despite the nonstationarity; and that learning individual reward functions can lead to better specialization of roles than is possible with shared reward, whether learned or given.More ↓Less ↑
Eye-movements in reading exhibit frequency spillover effects: fixation durations on a word are affected by the frequency of the previous word. We explore the idea that this effect may be an emergent property of a computationally rational eye-movement strategy that is navigating a tradeoff between processing immediate perceptual input, and continued processing of past input based on memory. We present an adaptive eye-movement control model with a minimal capacity for such processing, based on a composition of thresholded sequential samplers that integrate information from noisy perception and noisy memory. The model is applied to the List Lexical Decision Task and shown to yield frequency spillover---a robust property of human eye-movements in this task, even with parafoveal masking. We show that spillover in the model emerges in approximately optimal control policies that sometimes process memory rather than perception. We compare this model with one that is able to give priority to perception over memory, and show that the perception-priority policies in such a model do not perform as well in a range of plausible noise settings. We explain how the frequency spillover arises from a counter-intuitive but fundamental property of sequenced thresholded samplers.More ↓Less ↑
The rate of introduction of new technology into safety critical domains continues to increase. Improvements in evaluation methods are needed to keep pace with the rapid development of these technologies. A significant challenge in improving evaluation is developing efficient methods for collecting and characterizing knowledge of the domain and context of the work being performed. Traditional methods of incorporating domain and context knowledge into an evaluation rely upon expert user testing, but these methods are expensive and resource intensive. This paper will describe three new methods for evaluating the applicability of a user interface within a safety-critical domain (specifically aerospace work domains), and consider how these methods may be incorporated into current evaluation processes.More ↓Less ↑
We consider how to transfer knowledge from previous tasks to a current task in long-lived and bounded agents that must solve a sequence of MDPs over a finite lifetime. A novel aspect of our transfer approach is that we reuse reward functions. While this may seem counterintuitive, we build on the insight of recent work on the optimal rewards problem that guiding an agent's behavior with reward functions other than the task-specifying reward function can help overcome computational bounds of the agent. Specifically, we use good guidance reward functions learned on previous tasks in the sequence to incrementally train a reward mapping function that maps task-specifying reward functions into good initial guidance reward functions for subsequent tasks. We demonstrate that our approach can substantially improve the agent's performance relative to other approaches, including an approach that transfers policies.More ↓Less ↑
We explore the idea that eye-movement strategies in reading are precisely adapted to the joint constraints of task structure, task payoff, and processing architecture. We present a model of saccadic control that separates a parametric control policy space from a parametric machine architecture, the latter based on a small set of assumptions derived from research on eye movements in reading (Engbert, Nuthmann, Richter, & Kliegl, 2005; Reichle, Warren, & McConnell, 2009). The eye-control model is embedded in a decision architecture (a machine and policy space) that is capable of performing a simple linguistic task integrating information across saccades. Model predictions are derived by jointly optimizing the control of eye movements and task decisions under payoffs that quantitatively express different desired speed-accuracy trade-offs. The model yields distinct eye-movement predictions for the same task under different payoffs, including single-fixation durations, frequency effects, accuracy effects, and list position effects, and their modulation by task payoff. The predictions are compared to---and found to accord with---eye-movement data obtained from human participants performing the same task under the same payoffs, but they are found not to accord as well when the assumptions concerning payoff optimization and processing architecture are varied. These results extend work on rational analysis of oculomotor control and adaptation of reading strategy (Bicknell & Levy, 2010b; McConkie, Rayner, & Wilson, 1973; Norris, 2009; Wotschack, 2009) by providing evidence for adaptation at low levels of saccadic control that is shaped by quantitatively varying task demands and the dynamics of processing architecture.More ↓Less ↑
CogSci13
We demonstrate that an ideal observer model bounded by known limitations of the human visual system can explain empirical evidence concerning two effects of distractor ratios on visual search---effects that have previously been explained with salience-based models. The model makes optimal state estimations based on Bayesian estimates of stimuli localization and optimal control decisions of where to fixate in order to maximize task performance. Analysis of the model's behavior under different task strategies and different constraints on the visual system reveal which aspects of the model are responsible for the effects: the distractor-ratio effects on number of fixations is a signature of optimal state estimation in the face of noisy spatial information, and the saccadic-bias effect is a signature of both optimal control and estimation under these same bounds.More ↓Less ↑
Recent work has defined an optimal reward problem (ORP) in which an agent designer, with an objective reward function that evaluates an agent's behavior, has a choice of what reward function to build into a learning or planning agent to guide its behavior. Existing results on ORP show weak mitigation of limited computational resources, i.e., the existence of reward functions so that agents when guided by them do better than when guided by the objective reward function. These existing results ignore the cost of finding such good reward functions. We define a nested optimal reward and control architecture that achieves strong mitigation of limited computational resources. We show empirically that the designer is better off using the new architecture that spends some of its limited resources learning a good reward function instead of using all of its resources to optimize its behavior with respect to the objective reward function.More ↓Less ↑
Following work on designing optimal rewards for single agents, we define a multiagent optimal rewards problem (ORP) in common-payoff (or team) settings. This new problem solves for individual agent reward functions that guide agents to better overall team performance relative to teams in which all agents guide their behavior with the same given team-reward function. We present a multiagent architecture in which each agent learns good reward functions from experience using a gradient-based algorithm in addition to performing the usual task of planning good policies (except in this case with respect to the learned rather than the given reward function). Multiagency introduces the challenge of nonstationarity: because the agents learn simultaneously, each agent's learning problem is nonstationary and interdependent on the other agents. We demonstrate on two simple domains that the proposed architecture outperforms the conventional approach in which all the agents use the same given team-reward function (even when accounting for the resource overhead of the reward learning); that the learning algorithm performs stably despite the nonstationarity; and that learning individual reward functions can lead to better specialization of roles than is possible with shared reward, whether learned or given.More ↓Less ↑
Interference is a major source of short-term errors of memory. The present investigation explores the relationship between two important forms of interference: proactive interference (PI), induced by the need to reject recently studied items no longer relevant to task performance, and semantic interference (SI), induced by the need to reject lures sharing a meaningful relationship with current memoranda. We explore the possibility that shared cognitive control processes are recruited to resolve both forms of interference. In Experiment 1, we find that the requirement to engage in articulatory suppression during the retention interval of tasks that induce either PI or SI increases both forms of interference similarly and selectively. In Experiment 2, we develop a task to examine PI and SI within the same experimental context. The results show interactive effects between factors that lead to the two forms of interference. Taken together, these findings support contextual-cuing models of short-term remembering (Nairne, Annual Review of Psychology, 53, 53-81 2002), where the context in which retrieval occurs can influence susceptibility to interference. Lastly, we discuss several theoretical hypotheses concerning the cognitive control processes that are recruited to resolve SI and PI.More ↓Less ↑
Many comprehension theories assert that increasing the distance between elements participating in a linguistic relation (e.g., a verb and a noun phrase argument) increases the difficulty of establishing that relation during on-line comprehension. Such locality effects are expected to increase reading times and are thought to reveal properties and limitations of the short-term memory system that supports comprehension. Despite their theoretical importance and putative ubiquity, however, evidence for on-line locality effects is quite narrow linguistically and methodologically: It is restricted almost exclusively to self-paced reading of complex structures involving a particular class of syntactic relation. We present 4 experiments (2 self-paced reading and 2 eyetracking experiments) that demonstrate locality effects in the course of establishing subject-verb dependencies; locality effects are seen even in materials that can be read quickly and easily. These locality effects are observable in the earliest possible eye-movement measures and are of much shorter duration than previously reported effects. To account for the observed empirical patterns, we outline a processing model of the adaptive control of button pressing and eye movements. This model makes progress toward the goal of eliminating linking assumptions between memory constructs and empirical measures in favor of explicit theories of the coordinated control of motor responses and parsing.More ↓Less ↑
Intrinsic motivation is what causes us to do something ``for its own sake,'' in contrast to doing something for an external reward. There is great interest in building intrinsic motivation into artificial systems by defining intrinsic reward signals within the reinforcement learning framework. Yet, what intrinsic reward signals are, and how it may differ from extrinsic reward signals, remains a murky and controversial subject. Here we approach this issue from an evolutionary perspective that leads to the conclusion there are no hard and fast features distinguishing intrinsic and extrinsic reward signals. Rather, there is a continuum along which reward signals range that depends on the directness and complexity of the relationship between the rewarded behavior and evolutionary success.More ↓Less ↑
Planning agents often lack the computational resources needed to build full planning trees for their environments. Agent designers commonly overcome this finite-horizon approximation by applying an evaluation function at the leaf-states of the planning tree. Recent work has proposed an alternative approach for overcoming computational constraints on agent design: modify the reward function. In this work, we compare this reward design approach to the common leaf-evaluation heuristic approach for improving planning agents. We show that in many agents, the reward design approach strictly subsumes the leaf-evaluation approach, i.e., there exists a reward function for every leaf-evaluation heuristic that leads to equivalent behavior, but the converse is not true. We demonstrate that this generality leads to improved performance when an agent makes approximations in addition to the finite-horizon approximation. As part of our contribution, we extend PGRD, an online reward design algorithm, to develop reward design algorithms for Sparse Sampling and UCT, two algorithms capable of planning in large state spaces.More ↓Less ↑
Positive emotions are often treated as relatively similar in their cognitive-behavioral effects, and as having unambiguously beneficial consequences. For example, Valdesolo and DeSteno (2006) reported that a humorous video made people more prone to choose a utilitarian solution to a moral dilemma. They attributed this finding to increased positive affect. To determine whether such results actually stem in general from positive affect or from other more specific properties of humor, we conducted an experiment with moral dilemmas presented during an interleaved emotion-induction procedure involving mirth and another positive emotion, elevation. Mirth increased permissiveness for deontological violations, whereas elevation had the opposite effect. Furthermore, affective valence had no apparent independent influence on these judgments. Our results suggest that mirth and elevation have distinct cognitive consequences whose properties reflect their respective social functions, not their shared positive valence.More ↓Less ↑
Computational experiments have been used extensively to study language emergence by simulating the evolution of language over generations of interacting agents. Much of this work has focused on understanding the mechanisms of how language might have evolved. We propose a complementary approach helpful in understanding why specific properties of language might have emerged as an adaptive response to joint pressures from the environment and constraints on an agent's cognitive architecture. The approach suggests that linguistic systems can be described as boundedly optimal policies in multi-agent dynamic control problems defined by specific environments, agent computational structures, and task-oriented (vs. communication oriented) rewards. We illustrate the approach with a set of computational experiments.More ↓Less ↑
There is great interest in building intrinsic motivation into artificial systems using the reinforcement learning framework. Yet, what intrinsic motivation may mean computationally, and how it may differ from extrinsic motivation, remains a murky and controversial subject. In this article, we adopt an evolutionary perspective and define a new optimal reward framework that captures the pressure to design good primary reward functions that lead to evolutionary success across environments. The results of two computational experiments show that optimal primary reward signals may yield both emergent intrinsic and extrinsic motivation. The evolutionary perspective and the associated optimal reward framework thus lead to the conclusion that there are no hard and fast features distinguishing intrinsic and extrinsic reward computationally. Rather, the directness of the relationship between rewarding behavior and evolutionary success varies along a continuum.More ↓Less ↑
Reinforcement learning (RL) research typically develops algorithms for helping an RL agent best achieve its goals---however they came to be defined---while ignoring the relationship of those goals to the goals of the agent designer. We extend agent design to include the meta-optimization problem of selecting internal agent goals (rewards) which optimize the designer's goals. Our claim is that well-designed internal rewards can help improve the performance of RL agents which are computationally bounded in some way (as practical agents are). We present a formal framework for understanding both bounded agents and the meta-optimization problem, and we empirically demonstrate several instances of common agent bounds being mitigated by general internal reward functions.More ↓Less ↑
Recent work has demonstrated that when artificial agents are limited in their ability to achieve their goals, the agent designer can benefit by making the agent's goals different from the designer's. This gives rise to the optimization problem of designing the artificial agent's goals---in the RL framework, designing the agent's reward function. Existing attempts at solving this optimal reward problem do not leverage experience gained online during the agent's lifetime nor do they take advantage of knowledge about the agent's structure. In this work, we develop a gradient ascent approach with formal convergence guarantees for approximately solving the optimal reward problem online during an agent's lifetime. We show that our method generalizes a standard policy gradient approach, and we demonstrate its ability to improve reward functions in agents with various forms of limitations.More ↓Less ↑
The explore-exploit dilemma is one of the central challenges in Reinforcement Learning (RL). Bayesian RL solves the dilemma by providing the agent with information in the form of a prior distribution over environments; however, full Bayesian planning is intractable. Planning with the mean MDP is a common myopic approximation of Bayesian planning. We derive a novel reward bonus that is a function of the posterior distribution over environments, which, when added to the reward in planning with the mean MDP, results in an agent which explores efficiently and effectively. Although our method is similar to existing methods when given an uninformative or unstructured prior, unlike existing methods, our method can exploit structured priors. We prove that our method results in a polynomial sample complexity and empirically demonstrate its advantages in a structured exploration task.More ↓Less ↑
LCP
Seven experiments using self-paced reading and eyetracking suggest that omitting the middle verb in a double center embedding leads to easier processing in English but leads to greater difficulty in German. One commonly accepted explanation for the English pattern---based on data from offline acceptability ratings and due to Gibson & Thomas (1999)---is that working-memory overload leads the comprehender to forget the prediction of the upcoming verb phrase (VP), which reduces working memory load. We show that this VP-forgetting hypothesis does an excellent job of explaining the English data, but cannot account for the German results. We argue that the English and German results can be explained by the parser's adaptation to the grammatical properties of the languages: in contrast to English, German subordinate clauses always have the verb in clause-final position, and this property of German may lead the German parser to maintain predictions of upcoming verb phrases more robustly compared to English. The evidence thus argues against language-independent forgetting effects in online sentence processing: working memory constraints can be conditioned by countervailing influences deriving from grammatical properties of the language under study.More ↓Less ↑
Is forgetting in the short term due to decay with the mere passage of time, interference from other memoranda, or both? Past research on short-term memory has revealed some evidence for decay and a plethora of evidence showing that short-term memory is worsened by interference. However, none of these studies has directly contrasted decay and interference in short-term memory in a task that rules out the use of rehearsal processes. In this article the authors present a series of studies using a novel paradigm to address this problem directly, by interrogating the operation of decay and interference in short-term memory without rehearsal confounds. The results of these studies indicate that short-term memories are subject to very small decay effects with the mere passage of time but that interference plays a much larger role in their degradation. The authors discuss the implications of these results for existing models of memory decay and interference.More ↓Less ↑
The authors assume that individuals adapt rationally to a utility function given constraints imposed by their cognitive architecture and the local task environment. This assumption underlies a new approach to modeling and understanding cognition---cognitively bounded rational analysis---that sharpens the predictive acuity of general, integrated theories of cognition and action. Such theories provide the necessary computational means to explain the flexible nature of human behavior but in doing so introduce extreme degrees of freedom in accounting for data. The new approach narrows the space of predicted behaviors through analysis of the payoff achieved by alternative strategies, rather than through fitting strategies and theoretical parameters to data. It extends and complements established approaches, including computational cognitive architectures, rational analysis, optimal motor control, bounded rationality, and signal detection theory. The authors illustrate the approach with a reanalysis of an existing account of psychological refractory period (PRP) dual-task performance and the development and analysis of a new theory of ordered dual-task responses. These analyses yield several novel results, including a new understanding of the role of strategic variation in existing accounts of PRP and the first predictive, quantitative account showing how the details of ordered dual-task phenomena emerge from the rational control of a cognitive system subject to the combined constraints of internal variance, motor interference, and a response selection bottleneck.More ↓Less ↑
Existing models that integrate emotion and cognition generally do not fully specify why cognition needs emotion and conversely why emotion needs cognition. In this paper, we present a unified computational model that combines an abstract cognitive theory of behavior control (PEACTIDM) and a detailed theory of emotion (based on an appraisal theory), integrated in a theory of cognitive architecture (Soar). The theory of cognitive control specifies a set of required computational functions and their abstract inputs and outputs, while the appraisal theory specifies in more detail the nature of these inputs and outputs and an ontology for their representation. We argue that there is a surprising functional symbiosis between these two independently motivated theories that leads to a deeper theoretical integration than has been previously obtained in other computational treatments of cognition and emotion. We use an implemented model in Soar to test the feasibility of the resulting integrated theory, and explore its implications and predictive power in several task domains.More ↓Less ↑
CogSci09
Reinforcement learning has achieved broad and successful application in cognitive science in part because of its general formulation of the adaptive control problem as the maximization of a scalar reward function. The computational reinforcement learning framework is motivated by correspondences to animal reward processes, but it leaves the source and nature of the rewards unspecified. This paper advances a general computational framework for reward that places it in an evolutionary context, formulating a notion of an optimal reward function given a fitness function and some distribution of environments. Novel results from computational experiments show how traditional notions of extrinsically and intrinsically motivated behaviors may emerge from such optimal reward functions. In the experiments these rewards are discovered through automated search rather than crafted by hand. The precise form of the optimal reward functions need not bear a direct relationship to the fitness function, but may nonetheless confer significant advantages over rewards based only on fitness.More ↓Less ↑
AnnRevPsy
The past 10 years have brought near-revolutionary changes in psychological theories about short-term memory, with similarly great advances in the neurosciences. Here, we critically examine the major psychological theories (the ``mind'') of short-term memory and how they relate to evidence about underlying brain mechanisms. We focus on three features that must be addressed by any satisfactory theory of short-term memory. First, we examine the evidence for the architecture of short-term memory, with special attention to questions of capacity and how---or whether---short-term memory can be separated from long-term memory. Second, we ask how the components of that architecture enact processes of encoding, maintenance, and retrieval. Third, we describe the debate over the reason about forgetting from short-term memory, whether interference or decay is the cause. We close with a conceptual model tracing the representation of a single item through a short-term memory task, describing the biological mechanisms that might support psychological processes on a moment-by-moment basis as an item is encoded, maintained over a delay with some forgetting, and ultimately retrieved.More ↓Less ↑
How can we model behavior on complex, real-world tasks with a large range of possible strategies that may vary along multiple dimensions? In this paper, we show show how an emerging approach to cognitive modeling, cognitively bounded rational analysis, can be applied to efficiently specify large, multidimensional strategy spaces, and to predict which strategies within the space are followed. The approach also supports a novel way of analyzing error control strategies, by directly modeling error recovery procedures and factoring these into strategy prediction. We apply this approach in a model of a typing task exemplifying three dimensions of strategic variability: decomposition of tasks into subtasks, parallel vs. serial processing of subtasks, and control of errors. We present empirical results showing the strategies people adopted on the task. The model successfully predicts the strategies used, by optimizing over the strategy space for a utility function defined as the performance-based payoff used in the experiment.More ↓Less ↑
A central question in online human sentence comprehension is, ``How are linguistic relations established between different parts of a sentence?'' Previous work has shown that this dependency resolution process can be computationally expensive, but the underlying reasons for this are still unclear. This article argues that dependency resolution is mediated by cue-based retrieval, constrained by independently motivated working memory principles defined in a cognitive architecture. To demonstrate this, this article investigates an unusual instance of dependency resolution, the processing of negative and positive polarity items, and confirms a surprising prediction of the cue-based retrieval model: Partial-cue matches---which constitute a kind of similarity-based interference---can give rise to the intrusion of ungrammatical retrieval candidates, leading to both processing slow-downs and even errors of judgment that take the form of illusions of grammaticality in patently ungrammatical structures. A notable achievement is that good quantitative fits are achieved without adjusting the key model parameters.More ↓Less ↑
ICCM
Evaluating the performance of optimizing constraint satisfaction techniques for cognitive constraint modeling
Cognitive Constraint Modeling is an emerging modeling framework that allows a modeler to derive predictions of asymptotic performance from (a) a specification of architectural constraints; (b) a specification of a space of possible task strategies; and (c) an explicit objective (payoff) function. This approach has distinct advantages over traditional approaches in that it reduces the degrees of freedom inherent in specifying the details of particular strategies when explaining behavior; instead these specific strategies are selected on the basis of their maximum payoff given the architectural constraints. However, this approach potentially suffers from high computational cost and intractability on some problems because it is grounded in optimizing constraint-satisfaction techniques. To understand and address this problem, we have built a cognitive constraint problem generator which stochastically generates large populations of parametrically controlled problem instances on which we can test CCM performance. Initial results from our use of this tool have identified at least one clear property of problems-order strength-that has a significant and non-linear effect on performance time. These results, when extended, will pave the way to efficient constraint modeling.More ↓Less ↑
Critical of mechanistic accounts of cognition, Anderson (1990) showed that a demonstration that cognition is optimally adapted to its purpose and environment can offer an explanation for its structure. Simon (1992), in contrast, emphasised that the study of an adaptive system is not a ``logical study of optimization,'' but an empirical study of the conditions that limit the approach to the optimum. In response, we sketch the requirements for an approach to explaining behaviour that emphasises explanations in terms of the optimal behaviour given not only descriptions of the objective and environment but also descriptions of the human cognitive architecture and knowledge. A central assumption of the proposal is that a theory explains behaviour if the optimal behaviour predicted by the theory shows substantial correspondence to asymptotic human performance.More ↓Less ↑
Fig
The Proceedings of the Eighth International Conference on Cognitive Modeling
We have developed a software framework called Storm to aid the development of cognitive architectures based on the structure and function of the brain. The goals of the framework are to make it both easy and fast to develop and experiment with alternative architectures and components of architectures. In addition, the framework supports explicitly mapping its components to structures in the brain. We demonstrate a working implementation of the framework, where we have developed a simple model of skill learning and memory management in a simple 2D grid world.More ↓Less ↑
It has been well established in Cognitive Psychology that humans are able to strategically adapt performance, even highly skilled performance, to meet explicit task goals such as being accurate (rather than fast). This paper describes a new capability for generating multiple human performance predictions from a single task specification as a function of different performance objective functions. As a demonstration of this capability, the Cognitive Constraint Modeling approach was used to develop models for several tasks across two interfaces from the aviation domain. Performance objectives are explicitly declared as part of the model, and the CORE (Constraint-based Optimal Reasoning Engine) architecture itself formally derives the detailed strategies that are maximally adapted to these objectives. The models are analyzed for emergent strategic variation, comparing those optimized for task time with those optimized for working memory load. The approach has potential application in user interface and procedure design.More ↓Less ↑
Understanding a sentence requires a working memory of the partial products of comprehension, so that linguistic relations between temporally distal parts of the sentence can be rapidly computed. We describe an emerging theoretical framework for this working memory system that incorporates several independently motivated principles of memory: a sharply limited attentional focus, rapid retrieval of item (but not order) information subject to interference from similar items, and activation decay (forgetting over time). A computational model embodying these principles provides an explanation of the functional capacities and severe limitations of human processing, as well as accounts of reading times. The broad implication is that the detailed nature of crosslinguistic sentence processing emerges from the interaction of general principles of human memory with the specialized task of language comprehension.More ↓Less ↑
Common tasks in daily life are often accomplished by a sequence of actions. For highly practiced tasks, such as reading or piano playing, the eyes routinely move to the next item in sequence before the response to the current item is made (eye hand span), evidence of overlap in the mental processing of successive stimuli. We (Wu & Remington, 2004) have previously presented data from a typing-like task following Pashler (1994) requiring separate choice responses to a series of five stimuli. We found a consistent pattern of results in both motor and ocular timing, and examined behavior under perceptual and central difficulty factors. Here we report a model of that task, which demonstrates how the observed timing of eye movements to successive stimuli could result from attempts to produce regular rhythmic output. The model makes two key assumptions about the generation of an eye movement to the next stimulus in a sequence. The first, dependence on perceptual encoding, asserts that the eyes remain fixated on the current stimulus until perceptual encoding is completed. The second, minimization of inter-movement time variability, asserts that people try to achieve a rhythmic regularity in motor performance, which includes the timing of eye movements. We discuss the conditions under which variance minimization might apply, and the implications for accounts that tie eye movements more directly to internal cognitive, perceptual, and motor operations.More ↓Less ↑
Lang
Although proximity between arguments and verbs (locality) is a relatively robust determinant of sentence-processing difficulty (Hawkins 1998, 2001, Gibson 2000), increasing argument-verb distance can also facilitate processing (Konieczny 2000). We present two self-paced reading (SPR) experiments involving Hindi that provide further evidence of antilocality, and a third SPR experiment which suggests that similarity-based interference can attenuate this distance-based facilitation. A unified explanation of interference, locality, and antilocality effects is proposed via an independently motivated theory of activation decay and retrieval interference (Anderson et al. 2004).More ↓Less ↑
Common tasks in daily life are often accomplished by a sequence of actions that interleave information acquisition through the eyes and action execution by the hands. How are eye movements coordinated with the release of manual responses and how may their coordination be represented at the level of component mental operations? We have previously presented data from a typing-like task following Pashler (1994) requiring separate choice responses to a series of five stimuli (Wu & Remington, 2004; Wu, Remington, & Pashler, 2004). We found a consistent pattern of results in both motor and ocular timing, and hypothesized possible relationships among underlying components. Here we report a model of that task, which demonstrates how the observed timing of eye movements to successive stimuli could be accounted for by assuming two systems: an open-loop system generating saccades at a periodic rate, and a closed-loop system commanding a saccade based on stimulus processing. We relate this model to models of reading and discuss the motivation for dual control.More ↓Less ↑
In this paper, we argue that existing languages for representing routine cognitive tasks (such as GOMS, UAN, and PDL) can fail either because they demand that task competence is described using serial position to determine temporal order (and they are therefore overly restrictive) or because they demand that partial orderings are specified with temporal dependencies and other logical relationships (and they are therefore under-constrained). We propose a novel task description language, called Information-Requirements Grammar (IRG), which is motivated by a theory of how higher-level task performance is constrained by the information requirements and resource demands of lower-level tasks. We demonstrate the use of IRG and show how it replaces serial ordering and temporal dependencies with resource-bound information cascades between architectural information processes.More ↓Less ↑
We present a detailed process theory of the moment-by-moment working memory retrievals and associated control structure that subserve sentence comprehension, derived from independently motivated principles of memory and cognitive skill applied to the specialized task of sentence parsing. The theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation, formalized computationally in the ACT-R cognitive architecture.More ↓Less ↑
This work investigates the effects of syntactic, semantic, and morphophonemic similarity on the processing of center-embedding constructions in Japanese and Korean. Six Japanese experiments examine effects of syntactic and semantic similarity, and one Korean experiment examines effects of syntactic and morphophonemic similarity. The results support the view that the difficulty of processing center-embeddings is explained in part by similarity-based interference in working memory.More ↓Less ↑
Many behavioral models of the comprehension of derivationally suffixed words (e.g., happi+ness) assume a dual-route mechanism in which such words are accessed sometimes as whole units from the mental lexicon and sometimes via their component morphemes. This study used fMRI and a memory-encoding task to test this model, comparing derived words that show evidence of decompositional processing in behavioral studies (-ness, -less, -able) with those that do not (-ity, -ation). Decomposable derived and inflected words showed increased activity in Broca's area and the basal ganglia relative to nondecomposable suffixed words, supporting a dual-route model of lexical access.More ↓Less ↑
A hallmark of human cognition is the ability to compose novel behaviors from an existing repertoire of skills, ranging from search-based problem solving to the rapid, smoothly meshed perceptual-motor coordinations of well-practiced device interaction. This paper describes Cognitive Constraint Modeling (CCM), an approach to partially automating the composition of both semi-routine and highly skilled interactive behaviors. CCM is characterized by three principles: behavior descriptions are derived via constraint satisfaction over explicitly declared architectural, task, and strategy constraints; behavioral control emerges in part from optimizing behavior with respect to objective functions capturing general strategic goals (e.g., go as fast as possible); and the architectural building blocks are based on a simple ontology.More ↓Less ↑
Historically CPM-GOMS has been used to predict total time for long stretches of behavior. In ``Milliseconds Matter'', Gray and Boehm-Davis (2000) use CPM-GOMS to develop microstrategy variants with subtly different internal structure to explain differences observed in empirical data collected. They argue for microstrategies as the basic unit of adaptive behavior selection. While the microstrategies developed provide a good fit to the data, there is neither direct evidence for microstrategies as compared to other possible constructs nor an explicit statement of the theory underlying their construction. While the use of CPM-GOMS as an explanatory mechanism is a substantial advance, microstrategies have theoretical and practical limitations in terms of: microstrategies functioning as cognitive units, composition and structure, and dependency constraints. An alternative construct called an Architectural Process Cascade (APC) is proposed as the locus of adaptive behavior selection. An APC-based model of the Gray and Boehm-Davis button study task is presented to address the limitations of microstrategies.More ↓Less ↑
Cognitive Constraint Modeling (CCM) is an approach to reasoning about behavior that (1) provides a framework for investigating the hypothesis that skilled behavior is the optimal solution to a constraint satisfaction problem defined by objective, environmental, knowledge, and architectural constraints, (2) derives predictions of behavior from formal specifications of theory, (3) supports reasoning using both dependency-based and cascade-based ontologies for expressing temporal relationships between processes. A software tool that demonstrates the potential advantages of CCM is described. The tool, called CORE, can be used to partially automate the generation of behavioral predictions given a specification of the constraints. We explore the application of CORE to dual-task data previously modeled with EPIC and ACT-R.More ↓Less ↑
A hallmark of human cognition is the ability to compose novel behaviors from an existing repertoire of skills (Newell, 1990). These compositional processes range from search-based problem solving to the rapid, smoothly meshed perceptual-motor coordinations of well-practiced device interaction. In this paper we describe an approach to partially automating the composition of both semi-routine and highly skilled interactive behaviors. This approach, called Cognitive Constraint Modeling (CCM), is characterized by three principles: (a) descriptions of behavior are derived via constraint satisfaction over explicitly declared architectural, task, and strategy constraints; (b) the details of behavioral control (and therefore behavior composition) emerge in part from optimizing behavior with respect to objective functions intended to capture general strategic goals (e.g., go as fast as possible); and (c) the architectural building blocks are based on a simple ontology of resource-constrained cascaded processes. We show that these three principles jointly support modeling two important aspects of an interactive task: the overlapping and anticipatory behavior of highly skilled performance, and the hierarchical control of behavior evident earlier in practice. We contrast this approach with complementary approaches based on modeling the procedural learning processes themselves.More ↓Less ↑
Cognitive slowing is a feature of problem solving performance by Parkinson's patients. Here we present a computational model of the Tower of London problem solving task that provides a straightforward explanation of this latency impairment. The model is a neural network consisting of a set of idealized neurons whose activation levels are continuously varying quantities ranging between 0 and 1, organized hierarchically into a collection of structured, columnar assemblies. The model differs from many neural network models in that it intentionally approximates the discrete stages of processing and rule-based computation of production systems. It differs from many symbolic models in that it does not assume an instantaneous transition between different stages of processing and does not rely on a central, system-wide clock signal. As a result, the model faces timing problems that can prevent it from approximating an idealized discrete-time system closely enough to perform symbolic computation. The columnar assembly provides a solution to these problems by imposing a direction on the flow of activation between units and by controlling the rate of that flow. Columns are proposed to play the role of distributed timers implemented in cortical columns and frontostriatal loop circuits in the brain. Dopamine depletion in Parkinson's is proposed to reduce the rate of information transmission through a frontostriatal timer circuit critical for the generation of subgoal representations in prefrontal cortex. The model fits latency impairments in problem-solving by Parkinson's patients relative to controls and predicts that only problems that require the generation of subgoals will produce a significant latency impairment in Parkinson's patients.More ↓Less ↑
We present a series of simulations of behavioral data by casting a simple parsing model in the cognitive architecture ACT-R. We show that constraints defined in ACT-R, specifically those relating to activation, can account for a range of facts about human sentence processing. In doing so, we argue that resource limitation in working memory is better defined as an artefact of very general and independently motivated principles of cognitive processing.More ↓Less ↑
In this paper we report a new approach to generating predictions about skilled interactive cognition. The approach, which we call Cognitive Constraint Modeling, takes as input a description of the constraints on a task environment, on user strategies, and on the human cognitive architecture and generates as output a prediction of the time course of interaction. In the Cognitive Constraint Models that we have built this is achieved by encoding the assumptions inherent in CPM-GOMS as a set of constraints and reasoning about them using finite domain constraint satisfaction.More ↓Less ↑
We present a model cortical column consisting of recurrently connected, continuous-time sigmoid activation units that provides a building block for neural models of complex cognition. Recent progress with a hybrid neural/symbolic cognitive model of problem-solving prompted us to investigate the adequacy of these columns for the construction of purely neural cognitive models. Here we examine the computational power of networks of columns and show that every Turing machine maps in a straightforward fashion onto such a network. Furthermore, several hierarchical structures composed of columns that are critical in this mapping promise to provide biologically plausible models of timing circuits, gating mechanisms, activation-based short-term memory, and simple if-then rules that will likely be necessary in neural models of higher cognition.More ↓Less ↑
This paper presents the cue-based retrieval theory of parsing and reanalysis and illustrates how this account can accommodate a number of key results about parsing and reanalysis, including effects due to structure, distance, and type of structural change. Three offline experiments and one online experiment permit establishing the locus of these effects as due to properties of the initial parsing processes or to the repair mechanism. Specifically, the data reported here suggest that a structural factor specific to the operation of the parser, retrieval interference, affects attachment uniformly across ambiguous and unambiguous sentences and serves to create a limit on successful repair. In addition, these experiments suggest that distance of the head of an ambiguous phrase from its disambiguator affects repair processes---and not attachment processes---independently of the interference effect. These results are interpreted with respect to alternative models of reanalysis, which are contrasted with the cue-based retrieval account, which requires no distinct repair mechanism to account for the current results. A further contribution of this article is to suggest a statistical correction for individual variance in reading rates. Statistical analyses on individual subject data confirmed previous speculations regarding a possible increase in reading rates as subjects move through a sentence. While this individual variation limits fair comparisons of reading times in sentence regions that appear in non-identical serial positions, we demonstrate that such comparisons become meaningful when the appropriate regression analyses have been performed.More ↓Less ↑
CogBrainRes
A Computational Approach to Control in Complex Cognition
BibTeX:RIS:
When the human parser encounters a local structural ambiguity, are multiple structures pursued (parallel or breadth-first parsing), or just a single preferred structure (serial or depth-first parsing)? This note discusses four important classes of serial and parallel models: simple limited parallel, ranked limited parallel, deterministic serial with reanalysis, and probabilistic serial with reanalysis. It is argued that existing evidence is compatible only with probabilistic serial-reanalysis models, or ranked parallel models augmented with a reanalysis component. A new class of linguistic structures is introduced on which the behavior of serial and parallel parsers diverge the most radically: multiple local ambiguities are stacked to increase the number of viable alternatives in the ambiguous region from two to eight structures. This paradigm may provide the strongest test yet for parallel models.More ↓Less ↑
Korean JCogSci
A promising approach to more refined models consistent with the Caplan & Waters hypothesis is based on similarity-based interference, a general principle that applies across working memory domains. This may explain both the fine details of syntactic working memory phenomena and the gross fractionation for which Caplan & Waters have found evidence. Detailed models of syntactic processing that embody similarity-based interference fare well cross-linguistically.More ↓Less ↑
This chapter develops a theory of reanalysis called limited repair parsing. Repair parsers deal with the problem of local ambiguity in part by modifying previously built structure when the chosen structure later proves to be inconsistent. This modification of existing structure distinguishes repair parsing from parallel or multi-path parsing, least-commitment parsing, backtracking, or reparsing strategies. Parsers with a limited capability for repair are psycholinguistically important because they can potentially explain the contrasts between difficult garden path structures (when repair fails) and unproblematic local ambiguities (when repair is successful or easy). Although the idea of repair has been implicit in some psycholinguistic work (and emerged explicitly in the diagnosis model of Fodor & Inoue, 1994, and the NL-Soar model of Lewis, 1993), there has been no clear formulation of the general class of repair parsers. This chapter makes a first step toward such a formulation, shows how repair parsing offers significant computational advantages over other alternatives for reanalysis, and proposes a particular repair mechanism, snip, that explains a wide range of cross-linguistic reanalysis phenomena. Snip is a proposal for a simple, automatic, on-line repair process. The chapter concludes by briefly describing how snip can be embedded in a more comprehensive sentence processing architecture that maintains the structural sensitivity of purely syntactic theories like Pritchett's (1992), yet still accounts for the flexibility of parsing as revealed by interactive studies.More ↓Less ↑
Many theories have been proposed to explain difficulty with center embedded constructions, most attributing the problem to some kind of limited-capacity short-term memory. However, these theories have developed for the most part independently of more traditional memory research, which has focused on uncovering general principles such as chunking and interference. This article attempts to gain some unification with this research by suggesting that an interesting range of core sentence processing phenomena can be explained as interference effects in a sharply limited syntactic working memory. These include difficult and acceptable embeddings, as well as certain limitations on ambiguity resolution, length effects in garden path structures, and the requirement for locality in syntactic structure. The theory takes the form of an architecture for parsing that can index no more than two constituents under the same syntactic relation. A limitation of two or three items shows up in a variety of other verbal short-term memory tasks as well.More ↓Less ↑
CHI
Non-declarative learning in an interactive task
BibTeX:RIS:
Conceptions of the Human Mind
The Role of Language in Cognition: A Computational Inquiry
BibTeX:RIS:
This paper describes two research projects that study typical Situated Action tasks using traditional cognitive science methodologies. The two tasks are decision making in a complex production environment and interaction with an Automated Teller Machine (ATM). Both tasks require that the decision maker and the user search for knowledge in the environment in order to execute their tasks. The goal of these projects is to investigate the interaction between internal knowledge and dependence on external cues in these kinds of tasks. We have used the classical expert-novice paradigm to study information search in the decision making task and cognitive modeling to predict the behavior of ATM users. The results of the first project strongly indicate that decision makers are forced to rely on environmental cues (knowledge in the environment) to make decisions, independently of their level of expertise. We also found that performance and information search are radically different between experts and novices. Our explanation is that prior experience in dynamic decision tasks improves performance by changing information search behavior instead of inducing superior decision heuristics. In the second study we describe a computer model, based on the Soar cognitive architecture, that learns part of the task of using an ATM machine. The task is performed using only the external cues available from the interface itself, and knowledge assumed of typical human users (e.g., how to read, how to push buttons). These projects suggest that tasks studied by Situated Action research pose interesting challenges for traditional symbolic theories. Extending symbolic theories to such tasks is an important step toward bridging these theoretical frameworks.More ↓Less ↑
Multiple types of knowledge (syntax, semantics, pragmatics, etc.) contribute to establishing the meaning of an utterance. Immediate application of these knowledge sources is necessary to satisfy the real-time constraint of 200 to 300 words per minute for adult comprehension, since delaying the use of a knowledge source introduces computational inefficiencies in the form of overgeneration. On the other hand, ensuring that all relevant knowledge is brought to bear as each word in the sentence is understood is a difficult design problem. As a solution to this problem, we present NL-Soar, a language comprehension system that integrates disparate knowledge sources automatically. Through experience, the nature of the understanding process changes from deliberate, sequential problem solving to recognitional comprehension that applies all the relevant knowledge sources simultaneously to each word. The dynamic character of the system results directly from its implementation within the Soar architecture.More ↓Less ↑
Fig
Natural Language Comprehension in Soar: Spring 1991
BibTeX:RIS:
Fig
Action graphics: An interactive spreadsheet-based animation system for simulation and training
BibTeX:RIS:
Fig
CFCL: Central Florida Common Lisp Object System
BibTeX:RIS:
Fig
An educational animation system based on class inheritance
BibTeX:RIS:
Fig
A spreadsheet-based visual language for freehand sketching of complex motions
BibTeX:RIS:
Fig
The Action Graphics freehand animation system
BibTeX:RIS: