Survey outlines challenges and frameworks for in-context reinforcement learning in changing environments

A new arXiv preprint surveys in-context reinforcement learning (ICRL) with a focus on non-stationary environments, where the reward specification, transition dynamics, observation space, action interface, constraints, demonstrations, or memory distributions can shift over time.

The authors argue that in such settings, accumulated context is not merely additional evidence about a fixed task; instead, previously useful context can become stale, misleading, or relevant again as old regimes re-emerge. This complicates the agent’s ability to infer the current decision rule while deployed policy parameters remain fixed.

The paper defines non-stationary ICRL and situates it relative to related paradigms, including meta-reinforcement learning, decision sequence modeling, retrieval-augmented RL, value- and model-aware ICRL, and reward-feedback agents.

To organize the literature, the authors propose a taxonomy centered on three questions: what aspects of the environment change, how those changes unfold over time, and how observable the changes are to the agent. This framing aims to guide both the design of new algorithms and the evaluation of their adaptability in dynamic settings.