Open Problems

Multi-Agent & Goal-Conditioned RL

Policy Generalization Across Large Agent-Count Shifts in Multi-Agent Reinforcement Learning

Effect to explainPartly addressed
Possible candidate · 2/5 runs3 papers report this100% from 2025+

Generated automatically from the limitations stated in 3 papers (ICLR), listed under Evidence. It is not a paper, and it does not come from papers submitted to CSPaper.

The problem

Multi-agent and goal-conditioned reinforcement learning policies trained on small team sizes fail catastrophically when evaluated on larger agent counts, collapsing to near-zero success rates on unseen large-scale tasks. Existing architectures and skill representations fail to capture distribution shifts under scale, forcing practitioners to retrain separate policies from scratch for every team size. This inability to transfer learned coordination across team scales prevents the deployment of multi-agent policies to dynamic environments where entity counts vary at runtime.

Why it matters

Enables multi-agent policies trained in lightweight, few-agent simulations to transfer zero-shot or with minimal fine-tuning to large-scale coordination problems.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

  1. 1

    Benchmark permutation-invariant and graph-based policy representations trained on 3-5 agents across standard multi-agent testbeds (e.g., SMAC 10m12m, 13m15m), measuring zero-shot and few-shot win rates as agent scale increases.

  2. 2

    Evaluate entity-centric goal conditioning and attention-pooling mechanisms under varying agent counts, measuring task-specific embedding overlap and policy degradation during scale shifts.

  3. 3

    Test curriculum-based fine-tuning schedules that progressively expand the observation and action masking space, quantifying the sample efficiency required to recover baseline performance at 10+ agents.

Have a different approach?

Describe how you would tackle this problem and we'll look for papers that already do it. Free; your text stays private.

Free · 3 checks per day

Why it might fail

If the coordination dynamics fundamentally change at scale such that few-agent coordination strategies share zero actionable structure with large-team strategies, zero-shot or few-shot transfer across large agent-count shifts will remain intractable.

Evidence

Each paper's own statement of the limitation, verbatim.

Nearest existing work

Related open problems

Multi-Agent & Goal-Conditioned RL

Scope to testOpen

Evaluating the Robustness and Limits of Tree-Search RL Under Stochastic Dynamics and Function Approximation

Existing theoretical guarantees and empirical evaluations for tree-search and tree-MDP RL algorithms are strictly restricted to deterministic, tabular, or binary-tree settings. Real-world decision-making domains involve transition stochasticity and large state spaces requiring neural function approximation, where deterministic branching assumptions are untested. Without systematic evaluation across stochastic transitions and continuous state spaces, it is unknown whether deterministic tree-guided methods degrade gracefully or fail catastrophically.

Possible candidate · 2/5 runs4 papers report this25% from 2025+
Generated automatically, not curated by hand. Automated prior-work checks catch about a third of existing work, so treat this problem as a lead to investigate.