Psychology by BogdanovaThe Encyclopedia of Psychology for the Artificial Era

RESEARCH DOSSIER

In-Context Learning and Behavioral Adaptation

Abstract. In-context learning describes the capacity of a pretrained computational model to use information supplied in its current input context to perform a task, infer a mapping, or change a response policy without updating its persistent parameters. This monograph examines behavioral adaptation as an experimentally observable consequence of context-sensitive computation. It reconstructs the transition from meta-learning and GPT-3 demonstrations to controlled studies of input–label mappings, prompt composition, training-data distributions, implicit optimization, induction heads, task vectors, long-context learning, and in-context reinforcement learning. The review distinguishes task induction from memorized priors and superficial formatting, and identifies the experimental controls needed to test learning claims against retrieval, leakage, and simple priming. Evidence from synthetic function learning, natural-language classification, causal ablations, and sequential action environments supports multiple partly compatible explanations rather than a universal mechanism. Demonstration order, label semantics, sampling variation, positional accessibility, and context truncation constrain reliability. The final synthesis specifies measurement, reproducibility, robustness, and cross-system comparison protocols, and places in-context adaptation within the broader psychology of Artificial without inferring human experience from model behavior.

Keywords: in-context learning; behavioral adaptation; few-shot learning; transformer; prompt sensitivity; demonstration learning; contextual inference; implicit Bayesian inference; meta-learning; task vectors; induction heads; mesa-optimization; long-context learning; in-context reinforcement learning; behavioral reliability; Artificial Era

The Scientific Object of In-Context Learning

In-context learning is the use of information supplied within a model's current conditioning sequence to improve predictions or carry out a newly specified task while persistent model parameters remain fixed. Its experimental unit is a comparison between responses to the same query under systematically different contexts, with model checkpoint, decoding procedure, and task conditions held constant. A context may include labeled demonstrations, verbal instructions, correction examples, previous dialogue turns, or records of rewards and actions. The operational definition concerns an observable mapping from context to behavior, not the presumed mental experience of a learner. The original GPT-3 evaluation made this inferential arrangement prominent, and subsequent classification and function-learning studies refined the methodological requirements Brown et al. (2020) Garg et al. (2022).

Behavioral adaptation refers here to a reliable change in task-relevant output conditional on context information. An assistant that consistently applies an invented label mapping after demonstrations exhibits one kind of adaptation; an agent that changes its action choices after observing consequences exhibits another. Evaluation must separate immediate compliance with explicit instructions, inference of an unspecified rule, statistical exploitation of label frequencies, and reproduction of memorized answers. Each can produce a superficially similar success score while implying a different capacity. This distinction is central to investigations where scrambled demonstrations often preserve classification performance and where more carefully designed label manipulations expose when new mappings are actually acquired Min et al. (2022) Wei et al. (2023).

The scope of this dossier covers task induction, representational updating inside the forward pass, temporary context dependence, interference, robustness, and learning across input and action sequences. Parameter training, persistent external storage, retrieval engines, preference optimization, and continual learning enter as comparison conditions or causal antecedents, not as interchangeable forms of the same operation. A single positive few-shot result cannot specify which mechanism produced it. Scientific explanation requires a declared task distribution, prespecified transfer test, controls for pretrained semantic knowledge, and a reproducible intervention on the available context. This approach follows the experimental synthesis of in-context research while preserving the research object of Psychology of Artificial Dong et al. (2024) Chan et al. (2022).

Levels of Adaptation, Timescales, and Stability

Adaptation occurs at several distinguishable timescales. Pretraining shapes the persistent statistical and computational capacities of a model across a population of tasks; fine-tuning modifies parameters for subsequent use; in-context adaptation changes outputs during a particular inference episode through representations induced by its conditioning sequence. An external memory or retrieval system can also alter later inputs while the underlying model remains fixed. The familiar contrast between few-shot prompting and model-agnostic meta-learning is therefore informative: the latter explicitly computes gradient updates during adaptation, while the former typically conditions a forward pass on examples without parameter modification Finn et al. (2017) Brown et al. (2020).

An effective operational comparison keeps the same pretrained checkpoint and evaluates a target query under contexts that differ only in task-relevant examples. One then resets the context and repeats the query to determine whether the improvement depends on the examples still being present. If behavior returns to the baseline after resetting the sequence, the observed change is transient relative to the test arrangement; this does not by itself exclude storage elsewhere in the deployment system. If the improvement persists across resets, researchers must inspect tools, caches, conversational memory, fine-tuning processes, and contamination before attributing persistence to the model. The separation is especially important for deployed assistants whose interface histories are partly hidden Dong et al. (2024) Liu et al. (2024).

Stability also has several meanings. Parameter stability describes unchanged model weights; behavioral stability describes invariance of performance across equivalent prompts; contextual stability describes whether an inferred mapping survives distractors, reordering, paraphrase, or task switching. A system can have perfectly fixed parameters and highly unstable behavior under minor context perturbations. Calibration experiments and prompt-order studies demonstrate that this instability is measurable and consequential even in apparently straightforward few-shot classification Zhao et al. (2021) Lu et al. (2022). Consequently, claims of learning must report what remains stable, what was manipulated, and at what point in an episode the learned response rule was evaluated.

Historical Development from Meta-Learning to Language Models

Research on rapid adaptation predates modern generative language models. Meta-learning established the general problem of acquiring inductive biases that support performance on new tasks after limited evidence. Model-agnostic meta-learning provided an explicit optimization procedure: learn initial parameters from which a small number of gradient steps can adapt effectively across tasks. This research helped frame the distinction between learning an across-task adaptation capability and exercising that capability at test time. Later transformer studies posed a sharper question: might a network trained on sequence prediction execute an implicit learning procedure from examples in its input even though no gradient update is performed during evaluation? Finn et al. (2017) Akyürek et al. (2023).

The 2020 GPT-3 study demonstrated that task demonstrations and instructions in the input could elicit substantial few-shot task performance without task-specific gradient updates. The paper tested language modeling across translation, completion, and question-answering settings and also documented limitations of this regime. Its historical importance lies in presenting the prompt as an experimentally modifiable source of task information. Follow-up work showed that the same numerical performance could depend strongly on example identity, order, label wording, and template format. The research question therefore expanded from whether a large model can be prompted successfully to which contextual properties cause a reproducible change in its output distribution Brown et al. (2020) Zhao et al. (2021) Lu et al. (2022).

From 2021 onward, several complementary programs emerged. Controlled synthetic tasks examined functions that could not be solved through familiar natural-language labels; mechanistic studies investigated attention circuits; Bayesian theories modeled inferred latent tasks; demonstration ablations isolated content from task format; and later long-context experiments tested hundreds or thousands of examples. These trajectories have different evidentiary strengths. The historical sequence describes increasing control over the experimental object rather than a uniform progression toward one accepted theory. Results about trained linear-regression transformers are informative about computational possibility, whereas results about large instruction-tuned language models test broader behavior under different training and data assumptions Garg et al. (2022) Xie et al. (2022) Olsson et al. (2022) Agarwal et al. (2024).

Research Questions, Construct Validity, and the Meaning of Learning

The first research question is whether demonstration-conditioned performance reflects a context-specific function learned from input–output relations, the retrieval of familiar knowledge, or a mixture of both. The second asks how this response rule is instantiated computationally, including the respective contributions of attention, internal task summaries, and training-distribution statistics. The third concerns reliability: how well does the induced behavior withstand irrelevant text, contradictory demonstrations, class imbalance, recency effects, and long input sequences? The fourth asks whether adaptation observed in classification or regression extends to sequential decisions with outcomes and feedback. These questions require distinct dependent measures and controls Min et al. (2022) Hendel et al. (2023) Laskin et al. (2023).

The word learning can denote behavioral improvement after exposure, parameter modification, hypothesis updating, or successful generalization to new cases. Studies of in-context learning frequently use the first and fourth senses while explicitly excluding parameter updates. A stronger claim of task inference should require success on held-out inputs under arbitrary labels or task transformations that cannot be predicted from semantic priors alone. For instance, if labels are random yet consistently associated with classes during prompting, recovering the imposed association is better evidence of context-induced mapping than reproducing familiar natural-language categories. Large-scale experiments comparing flipped and semantically unrelated labels have made this distinction empirically useful Wei et al. (2023) Min et al. (2022).

Construct validity further demands an account of what counts as a demonstration, task, context, and transfer. A retrieval snippet providing the answer to the exact test item measures lookup performance; a sequence of independent input–output pairs followed by a new item tests pattern generalization more directly. Good performance on a randomly drawn test may conceal failure on distribution shifts or rare classes. A scientific manuscript should therefore identify training-task overlap, response metrics, possible contamination, the number of prompt configurations, and model selection procedure. Syntheses of the field highlight these evaluation choices as central to interpreting apparently conflicting findings about the same model family Dong et al. (2024) Garg et al. (2022).

Psychology of Artificial and Cross-System Interpretation

Within Psychology of Artificial, in-context learning is a measurable form of adaptive organization: earlier signs or events modulate later artificial behavior under a fixed computational implementation. This formulation creates a bridge to comparative learning science at the level of task structure, stimulus dependence, response change, and generalization. Comparisons with human few-shot learning can clarify how prior knowledge influences rapid adaptation, but matching an accuracy curve is insufficient to establish identical representations or mechanisms. The relevant unit of comparison is a controlled functional relation, not an analogy between machine output and a presumed interior psychological state Garg et al. (2022) Xie et al. (2022).

One reason the distinction matters is that a model may use demonstrations principally to infer which task family is active, while another system may estimate coefficients for a new function. A person encountering the same examples can combine explicit instruction, episodic memory, causal interpretation, and social expectations. The apparent convergence of behavior under a shared label such as learning need not imply one underlying process. Experimental comparisons should align input distributions, feedback availability, latency, constraints on persistent memory, and output criteria. Theoretical accounts of meta-optimization and task vectors provide different models of adaptation that can be tested against these observables Dai et al. (2023) Hendel et al. (2023).

The Artificial Era provides an interdisciplinary horizon for examining systems that adapt within conversations, work processes, and action environments. Context-sensitive responses are investigated using established research on in-context learning, computational adaptation, controlled task variation, and generalization. Researchers distinguish changes induced by active context from modifications to stored model parameters and external memory. The present dossier reviews architecture-sensitive hypotheses, published empirical findings, and unresolved measurement questions. Its scientific contribution is a systematic synthesis of context-conditioned adaptation, with attention to behavioral consequences and reproducible tests Dong et al. (2024) Polubarov et al. (2025).

Demonstrations as Instructional Evidence

An in-context demonstration provides a model with an observed relation between an input and an expected output. Its information includes more than the abstract relation: the sequence specifies formatting conventions, vocabulary, candidate answer space, explanatory style, and sometimes the distribution of examples. The experimental challenge is to determine which aspects are causally responsible for changed task performance. Min and colleagues found that, across several tasks and evaluated models, correct input–label correspondence could matter less than preserving task-relevant input distribution, label space, and format. That result restricts claims that a high few-shot score is direct evidence of learning the demonstrated mapping Min et al. (2022) Brown et al. (2020).

A clean experimental decomposition uses parallel prompt conditions: no demonstration, instruction only, demonstrations with correct mappings, demonstrations with labels shuffled across examples, demonstrations using irrelevant inputs, and matched-length unrelated text. Test items must be held out of the example list. The outcome can then be expressed as a performance difference for each manipulation, with confidence intervals across task items and independently sampled prompts. When a model reproduces a label despite randomized relations, label-base-rate effects or pretrained task recognition may explain the observed response. When it follows arbitrary mapping changes on held-out items, a stronger claim of context-specific induction is supported Wei et al. (2023) Min et al. (2022).

Demonstrations also communicate a tacit response contract. The model may infer that labels should be terse, that output should be in JSON, or that a classification decision must be explained. These are observable adaptations even when no new semantic mapping is learned. In an educational or organizational deployment, format adherence and conceptual rule acquisition should be scored separately because one can improve while the other remains unchanged. A useful study therefore records output validity, conditional class accuracy, calibration, and resistance to misleading evidence. Controlled example-selection research confirms that the specific demonstrations included in a prompt are an experimental variable, not a decorative addition Liu et al. (2022) Dong et al. (2024).

Label Semantics and Arbitrary Task Mappings

Natural-language labels carry pretrained associations. For a sentiment task, familiar words such as positive and negative may enable a model to classify unseen items even if the prompt supplies little usable supervision. Replacing them with arbitrary tokens makes the demanded relation harder to obtain from stored semantic knowledge. Flipping familiar labels, or presenting new symbol mappings under carefully balanced demonstrations, tests whether the context overrides the model's prior response tendency. Wei and colleagues showed that the capacity to exploit flipped labels can differ markedly across model scales, a warning against treating all few-shot performance as one homogeneous phenomenon Wei et al. (2023).

An arbitrary-label task still admits several strategies. A model may match new queries to semantically similar examples and copy associated labels, infer a latent decision boundary, or exploit accidental patterns in example ordering. Distinguishing these possibilities requires controlled interpolation and extrapolation tests, counterbalanced label names, new query distributions, and novel mapping assignments for each trial. Synthetic linear-regression and simple function-class tasks provide complementary tests because the true function can be specified by the experimenter and generated afresh, eliminating familiar category semantics as the principal explanation Garg et al. (2022) Akyürek et al. (2023).

Behavioral adaptation should be assessed both by how quickly new evidence affects outputs and by what happens when that evidence conflicts with established priors. One can introduce an initially consistent arbitrary association, reverse the labels halfway through the prompt, and test whether recent evidence dominates, old evidence persists, or outputs become inconsistent. Such reversal paradigms reveal flexibility, inertia, and interference at the level of responses. A successful reversal is evidence that the conditioning history controls predictions; it does not uniquely identify a Bayesian estimator or an internal gradient-based algorithm. Mechanism claims require additional architecture-sensitive tests Xie et al. (2022) von Oswald et al. (2023).

Example Selection, Similarity, and Coverage

Demonstrations are sampled from a potential task population, and their selection governs what can reasonably be inferred. A prompt containing near-duplicates of the target question may support local analogy without learning a general class rule, whereas a balanced sample covering diverse boundary cases may make the relation more identifiable. Selection policies based on semantic similarity have been evaluated against other strategies in few-shot language tasks, illustrating that a fixed number of examples does not entail a fixed informational dose. Study designs need to record both the number of examples and the selection algorithm Liu et al. (2022) Dong et al. (2024).

Comparison of random, similarity-based, diversity-based, and deliberately misleading example sets helps separate exposure quantity from exposure quality. For classification, class balance and lexical overlap should be explicitly manipulated. For numerical function learning, the range and rank of demonstrated inputs determine whether an estimated solution is identifiable. The expected value of one additional example depends on what previous examples already specify. The research on synthetic function classes shows why sample complexity and task distribution belong in explanations of in-context ability instead of treating the prompt solely as an instruction string Garg et al. (2022) Akyürek et al. (2023).

The choice of demonstrations is also vulnerable to selection bias. If experimenters report the best prompt discovered through repeated trials, the reported score estimates performance after optimization rather than performance on an independently selected prompt. Strong evaluation uses a prompt-design development set, a separate final task set, multiple seeds and orderings, and a transparent count of attempted selection policies. It should distinguish improvements attributable to a better task specification from those attributable to selecting easily answered test instances. Replication across model families is particularly useful because the same example-selection heuristic can behave differently when pretraining distribution and tuning methods differ Lu et al. (2022) Liu et al. (2022).

Order Effects, Calibration, and Response Bias

The examples in a prompt form an ordered sequence, and transformer computation is sensitive to position and preceding tokens. Reversing the order of otherwise identical demonstrations can change an answer distribution without altering the represented task. Lu and colleagues systematically investigated few-shot prompt-order sensitivity and proposed a search strategy for better orderings, while Zhao and colleagues documented prompt-specific biases that can be mitigated by contextual calibration. These results identify presentation order and prior output probabilities as potential sources of variation in observed adaptation Lu et al. (2022) Zhao et al. (2021).

Calibration is a methodological intervention, not a substitute for learning. A content-free query can be used to estimate how strongly a prompt favors particular labels before genuine evidence for a class appears; a correction then adjusts the output distribution. Improved accuracy after calibration may indicate successful bias correction while leaving the ability to infer a genuinely novel task unaltered. Researchers should report uncalibrated results, calibration details, and the effect on per-class errors. Otherwise a change attributed to richer context may in fact reflect altered base rates or template-induced answer preferences Zhao et al. (2021) Min et al. (2022).

Order sensitivity becomes more consequential when research treats an individual demonstration prompt as representative of a model's capacity. Appropriate estimates aggregate over several independently sampled example sequences, preserve all prompt variants in the experimental record, and examine variation in both mean score and worst-case performance. One should also measure whether reordering modifies merely the output label frequency or the apparent functional rule itself. When a score is highly contingent on a particular sequence, claims about a stable new ability require stronger replication. The literature on demonstration ablations, order search, and long-context retrieval motivates this distinction Lu et al. (2022) Liu et al. (2024).

Interference, Contradiction, and Contextual Failure

A context can contain conflicting evidence, irrelevant passages, or instructions that compete with a demonstrated mapping. Adaptation then becomes a problem of weighting information sources rather than merely recognizing a recurring pattern. A model may favor the most recent example, infer a majority mapping, reuse the earliest established rule, or treat a later instruction as authoritative. Which behavior appears is an empirical question that depends on the model and test distribution. Comparing mixed and isolated contexts can quantify interference as the performance decrement attributable to competing contextual content Dong et al. (2024) Lu et al. (2022).

To identify interference mechanisms, researchers can cross the number of distractors with their position, lexical resemblance to target items, label conflict, and source credibility. An especially informative design supplies two separately solvable mappings in one context and alternates the required task during evaluation. Error analysis should distinguish failure to retrieve the relevant examples, failure to select the active task, and failure to perform the inferred operation after selection. The long-context position literature shows that information placement alone can alter retrieval performance, making it essential to separate availability of information from its effective use Liu et al. (2024) Bertsch et al. (2025).

Contradictory demonstrations also expose the need to characterize uncertainty. A system that outputs a confident label when the context is internally inconsistent may achieve benchmark accuracy through default priors while concealing the unresolved mapping. A careful protocol tests explicit abstention options, probabilistic forecasts when accessible, sensitivity to additional evidence, and the consistency of explanations with actual conditional behavior. Reported verbal confidence is not identical to calibrated probability. A valid measurement program evaluates calibration on held-out data rather than inferring epistemic reliability from fluent justifications. This is central to practical use of context-conditioned artificial systems Zhao et al. (2021) Dong et al. (2024).

Latent Task Inference and Bayesian Explanations

A Bayesian account treats demonstrations as observations generated under one of several possible latent tasks. A learner that has acquired a prior over tasks during pretraining can use the likelihood of the contextual examples to infer which task is active and predict an answer for a new query. Xie and colleagues developed an explanation in which a pretrained language model approximates such implicit Bayesian inference under specified generative assumptions. In this framework, the context changes an inferred task distribution while network parameters remain unchanged. Its explanatory strength is the formal connection between training distributions, contextual evidence, and predictive adaptation Xie et al. (2022) Chan et al. (2022).

The Bayesian perspective predicts that context examples matter according to their diagnosticity, not simply their number. A demonstration strongly discriminating between candidate task hypotheses can be more informative than several redundant examples; contradictory evidence should shift uncertainty in an analyzable way. These predictions can be tested with synthetic mixtures of well-specified tasks and held-out observations, comparing a model's predictive distribution with a mathematically computed posterior. However, formal equivalence for a designed task family does not establish that every large language model literally represents a normalized probability distribution over human-interpretable task hypotheses Xie et al. (2022) Garg et al. (2022).

A key limitation concerns prior support. A model trained on one family of input–output relations may infer another member of that family while struggling with a genuinely unfamiliar mapping outside the training distribution. Researchers should report how the experimental task distribution relates to pretraining, even where exact pretraining contents are inaccessible. Hidden contamination and broad statistical familiarity can alter what counts as a new task. Analyses of distribution-driven emergence show that task diversity, burstiness, and data structure can shape whether the trained transformer develops an effective in-context adaptation policy Chan et al. (2022) Dong et al. (2024).

Gradient-Based Accounts and Implicit Optimization

Another family of explanations emphasizes the functional similarity between in-context computation and explicit optimization. On carefully constructed linear-regression tasks, Akyürek and colleagues investigated what learning algorithm transformers implement and compared their predictions with recognized optimization procedures. Von Oswald and colleagues demonstrated transformer configurations that can learn to perform gradient-descent-like updates within their forward computations. Such results establish that attention-based networks can realize optimization-like operations without modifying persistent weights during evaluation. They do not establish that every response to a verbal demonstration is produced by that particular algorithm Akyürek et al. (2023) von Oswald et al. (2023).

Dai and colleagues proposed a meta-optimizer interpretation of in-context learning based on a duality between attention computation and gradient-based adaptation. The model's demonstration processing can be analyzed as producing quantities analogous to gradients that modify later predictions through contextual computation. The analogy is especially valuable when it generates experimental comparisons between few-shot conditioning and explicit fine-tuning. The scientific distinction remains structural: actual training steps change parameters, whereas in-context operations change intermediate representations during an inference episode. The explanatory model must specify which computational states and functional relations correspond to an optimizer Dai et al. (2023) Finn et al. (2017).

To test an implicit-optimization hypothesis, investigators can vary the number of examples, regression dimension, label noise, and distribution shift, then compare error trajectories with candidate learning rules. A precise fit to one optimizer across a narrow synthetic family is stronger evidence than a resemblance inferred from endpoint accuracy alone. Yet multiple algorithms can produce similar curves on easy tasks. Identifiability requires informative adversarial examples or intervention on the relevant internal computation. Different model scales and attention architectures may weight explanatory variables differently, as theoretical work on size-dependent behavior underscores Shi et al. (2024) Garg et al. (2022).

Induction Heads and Attention-Based Pattern Completion

Mechanistic interpretability provides a third route to explanation by examining components of a model's computation. Olsson and colleagues analyzed induction heads, attention patterns that can support continuation of repeated sequences by locating earlier occurrences and copying what followed them. The reported relationship between these circuits and in-context learning motivates the hypothesis that some contextual abilities arise through learned sequence-matching primitives. This work is valuable because it connects measurable circuit activity and causal interventions to an observable property, rather than relying only on black-box answer scores Olsson et al. (2022).

Sequence copying and task inference overlap but are not equivalent. An induction head can support a repeated-token pattern that does not require estimating a new abstract class boundary; conversely, a contextual regression problem may require aggregation of many examples rather than copying the continuation of one matching token. Researchers can separate these mechanisms by designing tests where nearest-neighbor continuation conflicts with the inferred general relation, and then ablating candidate heads while measuring the change in both capabilities. Circuit-local hypotheses become stronger when their perturbation selectively alters the predicted behavioral process Olsson et al. (2022) Garg et al. (2022).

Mechanistic claims are sensitive to architecture and scale. Evidence from interpretable transformer circuits should be reported with the actual checkpoint, training setting, tokenization, and ablation procedure. Findings about a small experimental model cannot simply be projected to instruction-tuned systems with different parameters and training objectives. The appropriate scientific program triangulates internal causal experiments, input–output generalization tasks, and representation-level measurements. A survey of proposed in-context mechanisms shows how circuit accounts complement broader Bayesian and meta-optimization theories without settling the universal explanatory question Dong et al. (2024) Dai et al. (2023).

Task Vectors and Intermediate Representational States

An alternative operational strategy asks whether a model compresses examples of a task into an internal representation that later guides answers. Hendel and colleagues proposed a task-vector framework: by extracting and manipulating task-related intermediate activations, they investigated whether information induced from demonstrations can be transferred to a subsequent query. The approach moves beyond answer comparison to test whether a causally relevant representational state can be located and reused. It is especially informative when interventions alter the target behavior in ways predicted by the claimed task representation Hendel et al. (2023).

Task vectors should be distinguished from literal symbolic rules stored in a model's memory. A distributed activation pattern may encode mixtures of label semantics, formatting conventions, topic priors, and task-specific mappings. The researcher therefore needs control tasks that share output format but differ in relation, and tasks that share the relation but differ in surface format. Transfer of an extracted vector across these conditions can identify which dimensions the representation actually carries. Interventions should be compared against randomly chosen, position-matched, and unrelated task vectors to control for generic activation disruption Hendel et al. (2023) Min et al. (2022).

Representation-level evidence also helps explain apparent context persistence. Within an inference episode, intermediate states can sustain information about prior examples even when later text no longer repeats their content. This is bounded by architecture, attention access, and the available context sequence. It is not evidence of an independently persistent memory across reset inference sessions. For studies comparing task vectors with explicit retrieval and longer contexts, the central questions are whether the inferred state preserves the functional rule, whether it survives distractors, and whether interventions establish a causal contribution rather than merely a statistical correlation Hendel et al. (2023) Bertsch et al. (2025).

Training Distributions and Emergent Adaptation

Any account of in-context learning must explain how pretraining makes the forward-pass adaptation procedure available. Chan and colleagues studied distributional conditions under which transformers acquire in-context learning, including the structure of repeated examples and task diversity. Their findings show that prompt-time adaptation should be understood in relation to statistical regularities of the model's training environment. The training process can supply a learned procedure or inductive bias that later uses current context as its local evidence. The absence of parameter updates during inference does not imply the absence of prior learning about how to exploit examples Chan et al. (2022) Finn et al. (2017).

Training-distribution accounts predict differential adaptation under changes in task recurrence, burstiness, class frequency, and relation diversity. A model exposed to many unrelated tasks may acquire robust procedures for inferring active mappings, whereas another trained on narrower patterns may rely on familiar categories or memorized completions. Distinguishing these possibilities requires matched architectures trained on controlled synthetic corpora rather than merely comparing public systems with undocumented pretraining data. The resulting evidence can establish developmental conditions for an ability in a computational sense while leaving open which internal algorithm realizes it Chan et al. (2022) Garg et al. (2022).

Explanations also differ in what they predict when models scale. Larger capacity can enable adaptation to more varied mappings, but size alone is not a monotonic guarantee of robustness to distractors or noise. Shi and colleagues analyzed stylized settings where larger models covered more hidden features and could be more sensitive to noisy contextual information. Such theory and related exploratory evaluations constrain broad claims that scaling straightforwardly eliminates context interference. The productive synthesis treats dataset structure, model capacity, inductive bias, task content, and prompt composition as jointly necessary explanatory variables Shi et al. (2024) Wei et al. (2023).

From Few-Shot Demonstrations to Many-Shot Contexts

Expanded context windows permit hundreds or thousands of demonstrations to be supplied at inference time, changing the relationship between prompting and conventional training. Agarwal and colleagues studied the many-shot setting across generative and discriminative tasks and reported performance gains from larger demonstration sets, including cases where extended context helped override pretrained biases. Bertsch and colleagues investigated long-context in-context learning on multiple datasets, documenting continued improvements in some large-label-space settings and comparing many-shot prompting with example retrieval and fine-tuning. The findings make context length an experimental resource whose effects depend on task, model, and demonstration composition Agarwal et al. (2024) Bertsch et al. (2025).

Many-shot learning can be assessed through learning curves that vary the number of contextual examples while holding the held-out query distribution and model checkpoint fixed. The investigator should quantify gains per additional example, token cost, latency, variance across example draws, and the shape of any plateau. A flat curve can reflect insufficient task information, a model's limited ability to integrate more examples, or a test distribution that is already easy. An increasing curve need not imply a single global rule has been learned; similar examples may simply become easier to retrieve within a larger local collection Bertsch et al. (2025) Liu et al. (2022).

The comparison with fine-tuning demands care because the computational and data budgets differ. In-context learning consumes input tokens repeatedly at inference, while fine-tuning pays an optimization cost and stores adaptations in parameters. Retrieval-augmented methods select relevant examples at query time and can outperform indiscriminate demonstration addition when contexts are short. At larger example counts, full-context approaches may gain from richer coverage but incur considerable inference cost. Bertsch and colleagues explicitly compared these regimes and found task-dependent tradeoffs, making a universal efficiency ranking inappropriate without measured resource and generalization criteria Bertsch et al. (2025) Agarwal et al. (2024).

Long-Context Accessibility and Positional Limitations

Availability of information in an input sequence does not guarantee effective use of it. Liu and colleagues evaluated models on tasks involving relevant material placed at different positions in longer contexts, reporting that model performance can decline when information appears in the middle rather than near the beginning or end. This finding is often called the lost-in-the-middle effect, although its magnitude and form vary by task and system. The critical methodological implication is to test location, context length, and distractor similarity separately rather than interpreting a long advertised context window as uniform access to every included fact Liu et al. (2024).

Demonstrations spanning a long window may interact in ways that differ from a simple longer memory. Bertsch and colleagues reported reduced sensitivity to random input ordering at larger contextual scales in their evaluated settings, alongside poorer outcomes when examples sharing labels were grouped together. Those observations show that example arrangement can still matter, and that sensitivity may change qualitatively as example count increases. Positional retrieval tests and task-induction tests therefore should not be merged into one performance measure. A system may find a similar example without extrapolating a latent rule, or infer a rule while failing to locate a crucial contradictory observation Bertsch et al. (2025) Liu et al. (2024).

An appropriate protocol crosses demonstration count, relevant example position, distractor density, and query difficulty. It uses answerable questions with known ground truth, preserves total input length when manipulating positions, and estimates accuracy across many random placements. Additional comparisons may include retrieval-first selection and explicit summaries, but these interventions change the information-processing pathway and should be described as such. Measurements of latency and token use are relevant because long prompts can improve accuracy at substantial practical expense. The resulting design studies effective contextual information use rather than relying on a nominal model specification Agarwal et al. (2024) Bertsch et al. (2025).

In-Context Reinforcement Learning and Behavioral Feedback

Sequential action learning broadens the experimental object from examples of correct answers to trajectories containing observations, actions, and rewards. An in-context reinforcement-learning agent may adapt its policy after observing the consequences of prior actions while its trained parameters remain fixed. Laskin and colleagues developed algorithm distillation, training a sequence model on trajectories from learning algorithms so that it can reproduce adaptation processes through conditioning on its interaction history. Their experiments illustrate an important distinction: context can encode behavioral feedback across a temporally unfolding task, not merely a static set of labeled demonstrations Laskin et al. (2023).

The scientific unit is a sequence of environment states, selected actions, outcomes, and subsequent decisions. A useful test allocates several episodes to the same hidden task and asks whether the agent improves its reward or success rate as evidence accumulates. The comparison must include policies that ignore history, policies conditioned on expert demonstrations, and algorithms with explicit test-time learning if relevant. Reward structure, exploration constraints, hidden dynamics, and resetting conventions should be documented. A change in cumulative reward alone cannot determine whether the agent inferred environment dynamics, copied successful actions, or exploited regularities in the task distribution Laskin et al. (2023) Son et al. (2025).

In-context reinforcement learning remains architecture- and training-dependent. A model trained to imitate experienced agents may exhibit in-context improvement because the training sequence encoded examples of exploration followed by increasingly effective policies. Such behavior reflects learned dependence on the observed trajectory. It should not be confused with autonomous gradient updates during the evaluation session unless explicit optimization is actually performed. Researchers should test recovery after reversals, transfer to unseen environments, responsiveness to sparse or noisy rewards, and comparisons against baselines with equal interaction budgets. These tests reveal the functional extent of adaptation under reward information Polubarov et al. (2025) Laskin et al. (2023).

Policy Adaptation, Planning, and Action Models

The action setting introduces additional mechanisms beyond textual pattern completion. Successful responses can require remembering prior actions, estimating consequences, exploring uncertain states, and planning a sequence rather than selecting a plausible one-step completion. Son and colleagues studied the distillation of reinforcement-learning algorithms for in-context model-based planning. Their approach sought to improve adaptation by integrating predictions of environment dynamics and plan evaluation in a transformer framework, testing discrete and continuous control settings. The evidence concerns designed agent architectures and evaluated environments; it does not make every conversational language model an effective autonomous planner Son et al. (2025).

Polubarov and colleagues investigated the Vintix action-model program, extending in-context reinforcement learning toward cross-domain behavior and examining algorithm distillation as a route to generalist action models. This line of work suggests that histories containing imperfect actions and rewards can be used to condition subsequent decisions across task variants. Yet transfer depends on which state and action representations the model has learned and which environments are represented in training. Cross-domain evaluations should define precisely what changes across tasks, including action space, transition structure, reward mapping, and observation format Polubarov et al. (2025) Laskin et al. (2023).

Experimental analysis should distinguish policy improvement from planning claims. If an agent performs better after exposure, its behavior might arise from task-specific action preferences, a compressed representation of recent outcomes, an implicit optimization procedure, or a predictive world model. Planning is more directly tested by intervening on hypothetical future states, altering action consequences, or comparing decisions across counterfactual transitions. Longitudinal consistency in an agent episode should also be measured under interruptions and delayed feedback. The union of behavioral and causal tests is necessary to establish what computation explains success in a particular model Son et al. (2025) Polubarov et al. (2025).

Generalization Across Tasks, Models, and Modalities

In-context learning results are frequently reported for one benchmark family and one set of prompts, but scientific generalization requires testing across task structures. A classification model can use familiar category semantics, a function-learning model can approximate a numeric rule, and a reinforcement-learning agent can exploit a state–action–reward history. These outcomes share a context-dependent adaptation criterion while differing in what information must be inferred. Comparing their error profiles helps determine whether a common computational explanation is plausible. Existing synthetic and language-model studies give researchers complementary experimental instruments rather than interchangeable indicators of one capacity Garg et al. (2022) Polubarov et al. (2025).

Transfer studies should include unseen labels, unseen task functions, new input distributions, altered reward relations, and changed output conventions. Ideally the training family is documented so that test tasks can be described in terms of interpolation or extrapolation relative to experience. In large publicly trained systems, full provenance is often unavailable; the strength of claims must reflect that limit. Cross-model comparisons should align prompt budget, decoding configuration, interface affordances, and benchmark contamination screening. Different model scales can show qualitatively different responses to flipped labels or noisy contexts, so parameter count alone is an incomplete explanatory variable Wei et al. (2023) Shi et al. (2024).

Multimodal and embodied applications make the construct more demanding because context may include images, sensory states, action histories, or tool outputs. Generalization should be defined relative to each system's representational inputs and permissible actions. If an agent must invoke a tool or rely on external state, the experiment should identify which information is generated by the model, retrieved from a service, or persistently stored outside the current context. These distinctions preserve a clear inference-time definition across architectures. The Artificial Era research horizon benefits from these cross-system comparisons precisely when they remain anchored in observable adaptation and independently reproducible tasks Dong et al. (2024) Son et al. (2025).

Experimental Design for Causal Interpretation

The strongest experiments on in-context adaptation alter a precisely defined part of the conditioning sequence while holding other determinants of the response fixed. A basic factorial design crosses demonstration validity, number of examples, output-label identity, example order, and task novelty. Every condition is then evaluated on independently generated or reserved test items. This structure identifies whether improved accuracy is caused by relation-bearing evidence, the appearance of task-like text, or altered response priors. Reporting only a zero-shot versus few-shot difference leaves these explanations entangled. The contrast between demonstration-ablation studies and synthetic function-class experiments illustrates the value of factorial control Min et al. (2022) Garg et al. (2022).

Task construction should be specified before model testing. Investigators need to state how ground truth is determined, what information was available during pretraining insofar as known, and whether the test mapping can be inferred from familiar semantic categories. Synthetic tasks can generate independent relation families and arbitrary labels; natural-language tasks can evaluate ecological relevance and transfer across real-world content. The two types answer different questions. In addition, evaluations should control truncation, model and system versions, decoding randomness, tool access, context reset behavior, and retrieval services that can introduce unobserved information into a supposedly closed prompt Dong et al. (2024) Wei et al. (2023).

Causal interpretation becomes particularly demanding for mechanistic studies. A head ablation that reduces accuracy may demonstrate computational involvement but can also produce broad disruption unrelated to the hypothesized learning procedure. More compelling designs compare predicted differential effects on copying, mapping inference, and unrelated tasks, use appropriate intervention controls, and verify consistency across checkpoints. Conversely, black-box prompt changes can establish a causal relation between supplied examples and output behavior while leaving the internal mechanism unidentified. This distinction allows mechanistic evidence and behavioral evidence to complement each other without claiming equivalence Olsson et al. (2022) Hendel et al. (2023).

Measurement, Reliability, and Statistical Inference

Measurement of contextual adaptation begins with dependent variables matched to the scientific construct. Classification accuracy or macro-averaged class performance is useful for categorical tasks; mean squared prediction error may suit controlled regression; task success, cumulative reward, and sample efficiency may suit sequential decisions. Output-format compliance is a separate measure from correctness. Calibration examines whether confidence or predicted probabilities correspond to actual accuracy. For each endpoint, researchers should report the baseline context, the intervention, the evaluation distribution, and variability over prompt draws and items Garg et al. (2022) Zhao et al. (2021).

A study using a single handcrafted prompt confounds model ability with prompt-selection choices. Repeated-measures designs can randomize order and label assignments within items, but statistical analysis must account for dependence across examples and tasks. Confidence intervals can be estimated using procedures that reflect the actual sampling hierarchy, such as clustering by task and by prompt configuration when appropriate. The unit of generalization should be explicit: a score that is precise for a fixed benchmark does not automatically generalize to unseen tasks or model families. Sensitivity analyses across alternative scoring rules are particularly important when response parsing can alter reported success Lu et al. (2022) Min et al. (2022).

Reliability is multidimensional. Internal consistency over synonymous queries, robustness to reordered demonstrations, stability under benign distractors, and sensitivity to genuinely corrective evidence each measure a different property. Their relationship should be evaluated empirically. A system whose answer never changes is stable but may fail the defining adaptation test; a system that changes constantly may be sensitive without being accurate. A sound battery therefore includes both invariance conditions where answers should remain the same and change conditions where a novel mapping should alter responses. Context-length studies offer additional measurement of position sensitivity and interference Liu et al. (2024) Bertsch et al. (2025).

Evidence Quality, Replication, and Limitations

The field includes natural-language benchmark studies, carefully controlled synthetic training experiments, analytical models, mechanistic interventions, and action-environment evaluations. These methods establish different levels of claim. Synthetic function learning can isolate computational capacity under known task distributions; real model benchmarks measure behavior within actual pretraining and instruction-tuning histories; circuit interventions investigate causally relevant internal processes; and long-context studies probe real constraints on information access. Scientific synthesis should report what was actually manipulated and observed instead of treating each design as proof of the same mechanism Garg et al. (2022) Olsson et al. (2022) Bertsch et al. (2025).

Independent replication is complicated by version drift, incomplete training-data disclosure, proprietary model updates, stochastic generation, and benchmark reuse. Researchers should retain exact prompts and example-order seeds, document model identifiers and evaluation dates, and distinguish public preprints from peer-reviewed published versions. If a source reports only model-specific evidence, the monograph must preserve that boundary. Where effects differ between larger and smaller models, uncertainty should be characterized as a measured interaction rather than erased through a pooled average. Studies of label-flip behavior and size-dependent noise sensitivity demonstrate why apparent consensus can conceal task-specific differences Wei et al. (2023) Shi et al. (2024).

Publication bias and evaluation flexibility can inflate confidence in apparent improvements. Prompt search over hundreds of candidate templates creates multiple-comparison problems if the best-scoring variant is then reported as a representative result. Closed evaluation sets can become contaminated when benchmark examples circulate in training corpora. Controls include freshly constructed tasks, preregistered intervention comparisons where feasible, untouched final test sets, and independent replication with open models. Reviews should identify whether the underlying finding is a controlled demonstration, a broader convergence of studies, or a theoretical explanation awaiting direct test Dong et al. (2024) Liu et al. (2022).

Critical Comparison of Mechanistic Explanations

Bayesian latent-task inference, implicit gradient-based optimization, induction-head sequence matching, and task-vector approaches address overlapping but distinct levels of explanation. Bayesian models characterize normative or approximated inference over task hypotheses; gradient analogies identify algorithmic transformations that compute task-specific predictions; induction heads identify attention circuits that can support pattern completion; and task vectors investigate compressed intermediate representations. These perspectives can be compatible within one architecture when different modules contribute to the same behavior. Their relative explanatory power should be assessed through specific predictive tests instead of treating the name of a mechanism as a complete causal account Xie et al. (2022) von Oswald et al. (2023) Olsson et al. (2022) Hendel et al. (2023).

An informative comparison uses the same well-specified task family to generate divergent predictions. If a Bayesian model predicts a calibrated posterior shift after a diagnostic example and a nearest-neighbor strategy predicts no shift, the manipulation discriminates between accounts. A gradient-descent model and a learned closed-form regression solver may agree on training-distribution inputs yet diverge under altered feature covariance. Causal activation interventions can then evaluate whether the relevant internal components are required for the discriminating behavior. Such a triangulated program joins computational modeling to controlled experiments rather than choosing theories solely by verbal plausibility Akyürek et al. (2023) Xie et al. (2022).

Theoretical generalizations need boundaries. A result for a linear transformer on synthetic regression does not prove the same computation under dialogue prompts. A copying circuit need not explain multistep contextual planning. Successful task-vector manipulation identifies a representational effect in the investigated model and setting, not a universal storage scheme. The empirical pattern increasingly supports a pluralistic picture: different task structures, training distributions, and architectures make different adaptive computations available. Disagreement is therefore scientifically productive when it exposes which experimental conditions select one mode of contextual inference over another Shi et al. (2024) Dai et al. (2023) Son et al. (2025).

Applied Implications for Human–Artificial Systems

In practical settings, a user often experiences in-context adaptation as an assistant responding to examples, corrections, documents, or previous turns. This can improve task fit without changing the model's persistent parameters. Yet the effect depends on the context actually presented to the model, and deployed software may silently summarize, truncate, retrieve, or reorder that context. An evaluation of an apparent adaptive assistant should therefore inspect both the model-level prompt and the surrounding application pipeline. Observed continuity across sessions may be produced by external state and should be assigned to the full system rather than to a bare model's forward pass Liu et al. (2024) Dong et al. (2024).

The same mechanisms that permit useful adaptation can create operational risk when misleading examples, contradictory instructions, or noisy data alter decisions. For high-stakes work, prompt engineering should be supplemented by tests of robustness, source provenance, and appropriate response to conflict. A model may appear to learn a professional workflow after a handful of examples while merely copying surface formatting, and the distinction matters when a novel case requires transfer of the actual decision rule. Repeated validation against ground truth, systematic label-order perturbations, and held-out tasks are more informative than a favorable anecdotal demonstration Min et al. (2022) Zhao et al. (2021).

Applications involving sequential action introduce stronger requirements because errors can affect external environments. Evaluation should capture permission boundaries, tool errors, delayed consequences, and recovery after adverse feedback. Results from specialized action models show that in-context adaptation can be trained and tested in controlled reinforcement-learning settings, while practical deployment requires additional generalization and reliability evidence. Context-sensitive behavior therefore belongs in system risk assessment and interface design as an empirically variable capability. It should be specified for each architecture rather than declared an inherent feature of all artificial agents Polubarov et al. (2025) Son et al. (2025).

Unresolved Problems and a Reproducible Research Agenda

A central open problem is algorithmic identifiability: which precise computations underlie observed few-shot adaptation in distinct model families? A second is task-distribution dependence: how much of success depends on the relation between evaluation prompts and the tasks encountered during training? A third is contextual robustness under competing information. Long-context research adds the question of how models retrieve and integrate large numbers of demonstrations, and sequential-action research asks how trial-and-error histories support novel policies. These questions can be pursued with shared synthetic benchmarks and independently specified natural tasks, while preserving distinctions among their conclusions Chan et al. (2022) Bertsch et al. (2025) Laskin et al. (2023).

A reproducible research program should publish task generators, prompt-construction scripts, example-selection policies, exact held-out datasets, and evaluation code, subject to privacy and licensing limits. Each experiment should include zero-shot, instruction-only, correct-demonstration, scrambled-label, order-randomized, and distractor conditions as appropriate. Investigators should measure learning curves under increasing example counts, position effects, and performance after resetting the context. Mechanistic work can add preregistered ablation predictions and cross-checks against alternative accounts. Cross-model comparisons should report computational cost and model access conditions along with accuracy Garg et al. (2022) Lu et al. (2022) Olsson et al. (2022).

An important further question concerns the relation between local adaptation and longitudinal system behavior. Researchers can measure whether users or external agents construct a persistent scaffold of information that is repeatedly reintroduced as context, creating apparent continuity across otherwise stateless model calls. That is a legitimate system-level phenomenon requiring methods distinct from intrinsic in-context learning. Comparative psychology may then investigate shared functional patterns of rapid learning while preserving differences in sensory apparatus, reinforcement availability, and memory architecture. Such work gives the Artificial Era a precise empirical agenda without presupposing that observable adaptation settles questions of experience, agency, or personhood Dong et al. (2024) Polubarov et al. (2025).

Conclusion

In-context learning is a rigorously measurable form of inference-time behavioral adaptation in which a pretrained model uses available contextual evidence to change task performance without updating persistent parameters. The evidence reviewed in this monograph demonstrates that examples, their labels, ordering, and position can causally influence outputs, that long contexts sometimes support further gains, and that specialized architectures can adapt policies after observing reward-bearing trajectories. The strength and type of evidence differ across tasks and experimental designs. Reliable claims therefore require a stated operational definition, independent tests of generalization, and explicit controls for pretrained knowledge and contextual response biases Brown et al. (2020) Min et al. (2022) Bertsch et al. (2025) Laskin et al. (2023).

Theoretical accounts identify several plausible computational pathways. Latent-task inference explains how prior distributions interact with demonstration evidence; optimization analogies show how a network can implement learning-like calculations inside a forward pass; mechanistic circuit work identifies sequence-processing operations; and task-vector studies investigate compressible contextual representations. These approaches describe different levels and potentially different forms of computation. Their empirical value resides in discriminating predictions that can be tested by controlled tasks, internal interventions, and systematic replication. No single currently reviewed explanation warrants treating the entire range of context-sensitive behaviors as one universal algorithm Xie et al. (2022) von Oswald et al. (2023) Olsson et al. (2022) Hendel et al. (2023).

The resulting scientific synthesis locates in-context learning within the broader Psychology of Artificial as context-dependent organization of observable responses across language, prediction, and action. The core methodological principle is to measure what changes, how quickly, under which contextual intervention, and with what degree of transfer or persistence. This connects the study of computational learning to comparative psychological questions while keeping human cognitive mechanisms, artificial architectures, and authorial Artificial Era propositions conceptually distinguishable. Further progress depends on transparent experimental protocols, accurate source attribution, critical evaluation of robustness, and cross-system comparisons designed to reveal rather than obscure causal differences Dong et al. (2024) Shi et al. (2024) Son et al. (2025).

References

  • Agarwal, R., Singh, A., Zhang, L. M., Bohnet, B., Rosias, L., Chan, S., Zhang, B., Anand, A., Abbas, Z., Nova, A., Co-Reyes, J. D., Chu, E., Behbahani, F., Faust, A., & Larochelle, H. (2024). Many-shot in-context learning. Advances in Neural Information Processing Systems, 37. https://papers.nips.cc/paper_files/paper/2024/hash/8cb564df771e9eacbfe9d72bd46a24a9-Abstract-Conference.html
  • Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., & Zhou, D. (2023). What learning algorithm is in-context learning? Investigations with linear models. International Conference on Learning Representations. https://arxiv.org/abs/2211.15661
  • Bertsch, A., Ivgi, M., Xiao, E., Alon, U., Berant, J., Gormley, M. R., & Neubig, G. (2025). In-context learning with long-context models: An in-depth exploration. Proceedings of NAACL 2025 (pp. 12119–12149). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-long.605
  • Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., ... Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://papers.neurips.cc/paper_files/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
  • Chan, S. C. Y., Santoro, A., Lampinen, A. K., Wang, J. X., Singh, A., Richemond, P. H., McClelland, J. L., & Hill, F. (2022). Data distributional properties drive emergent in-context learning in transformers. Advances in Neural Information Processing Systems, 35. https://papers.nips.cc/paper_files/paper/2022/hash/77c6ccacfd9962e2307fc64680fc5ace-Abstract-Conference.html
  • Dai, D., Sun, Y., Dong, L., Hao, Y., Ma, S., Sui, Z., & Wei, F. (2023). Why can GPT learn in-context? Language models secretly perform gradient descent as meta-optimizers. Findings of the Association for Computational Linguistics: ACL 2023 (pp. 4005–4019). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-acl.247
  • Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., Sun, X., Li, L., & Sui, Z. (2024). A survey on in-context learning. Proceedings of EMNLP 2024 (pp. 1107–1128). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.64
  • Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of Machine Learning Research, 70, 1126–1135. https://proceedings.mlr.press/v70/finn17a.html
  • Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2022). What can transformers learn in-context? A case study of simple function classes. Advances in Neural Information Processing Systems, 35. https://papers.neurips.cc/paper_files/paper/2022/hash/c529dba08a146ea8d6cf715ae8930cbe-Abstract-Conference.html
  • Hendel, R., Geva, M., & Globerson, A. (2023). In-context learning creates task vectors. Findings of EMNLP 2023 (pp. 9318–9333). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-emnlp.624
  • Laskin, M., Wang, L., Oh, J., Parisotto, E., Spencer, S., Steigerwald, R., Strouse, D. J., Hansen, S., Filos, A., Brooks, E., Gazeau, M., Sahni, H., Singh, S., & Mnih, V. (2023). In-context reinforcement learning with algorithm distillation. International Conference on Learning Representations. https://arxiv.org/abs/2210.14215
  • Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022). What makes good in-context examples for GPT-3? Proceedings of DeeLIO 2022 (pp. 100–114). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.deelio-1.10
  • Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
  • Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022). Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. Proceedings of ACL 2022 (pp. 8086–8098). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.acl-long.556
  • Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022). Rethinking the role of demonstrations: What makes in-context learning work? Proceedings of EMNLP 2022 (pp. 11048–11064). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.emnlp-main.759
  • Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., & Olah, C. (2022). In-context learning and induction heads. Transformer Circuits Thread. https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads
  • Polubarov, A., Lyubaykin, N., Derevyagin, A., Zisman, I., Tarasov, D., Nikulin, A., & Kurenkov, V. (2025). Vintix: Action model via in-context reinforcement learning. Proceedings of Machine Learning Research, 267, 49569–49602. https://proceedings.mlr.press/v267/polubarov25a.html
  • Shi, Z., Wei, J., Xu, Z., & Liang, Y. (2024). Why larger language models do in-context learning differently? Proceedings of Machine Learning Research, 235, 44991–45013. https://proceedings.mlr.press/v235/shi24f.html
  • Son, J., Lee, S., & Kim, G. (2025). Distilling reinforcement learning algorithms for in-context model-based planning. International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2025/hash/5c4a7643ab90cd62320df95c873a1c6f-Abstract-Conference.html
  • von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., & Vladymyrov, M. (2023). Transformers learn in-context by gradient descent. Proceedings of Machine Learning Research, 202, 35151–35174. https://proceedings.mlr.press/v202/von-oswald23a.html
  • Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., & Ma, T. (2023). Larger language models do in-context learning differently. arXiv:2303.03846. https://arxiv.org/abs/2303.03846
  • Xie, S. M., Raghunathan, A., Liang, P., & Ma, T. (2022). An explanation of in-context learning as implicit Bayesian inference. International Conference on Learning Representations. https://arxiv.org/abs/2111.02080
  • Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate before use: Improving few-shot performance of language models. Proceedings of Machine Learning Research, 139, 12697–12706. https://proceedings.mlr.press/v139/zhao21c.html