RLC: new conference, new peer review in Reinforcement Learning research
-
The Reinforcement Learning Conference (RLC) has emerged as a bold and refreshing entry in the CS research landscape. After its inaugural edition at UMass Amherst in 2024, the second RLC will be hosted from August 5–8, 2025, at the University of Alberta, Edmonton, a fitting venue, considering Alberta's storied legacy in reinforcement learning (RL) research.
A breath of fresh air in Peer Review
While major ML conferences like NeurIPS, ICML, and ICLR continue to grow in scale and prestige, they also face increasing criticism over their peer review systems. Common complaints include:
- Overburdened and undertrained reviewers
- Shallow or contradictory reviews
- Overemphasis on subjective notions like "novelty" and "impact"
RLC set out from the beginning to challenge the status quo of peer review in technical fields. According to its organizers, RLC introduces several bold innovations:
🧪 Review Focus: Rigor Over Hype
RLC prioritizes technical correctness and methodological soundness rather than perceived impact or novelty. Reviewers are explicitly discouraged from speculating on significance, trusting the academic community to "vote with citations" over time.
️ Quality Over QuantityLow-effort or vague reviews are discarded. Authors receive fewer but higher-quality reviews, regardless of acceptance outcome.
🧠 Dual Reviewer Roles
Each paper is assigned to a senior reviewer and a high-level PhD student:
- The PhD student conducts a deep dive on technical correctness.
- The senior reviewer offers mentorship and a broader perspective, verifying the quality of the review itself.
This dual system is both educational and rigorous, offering growth for student reviewers and accountability for the process.
Transparency in Award SelectionUnlike many conferences where Best Paper Awards can be opaque or reputation-driven, RLC's awards are carefully selected through a three-stage blind evaluation process:
- Abstracts and meta-reviews of all 113 accepted papers are read.
- 50 papers are shortlisted and more closely analyzed.
- Each category's top 2~3 papers are deliberated with full-paper reads.
Ultimately, the committee reached consensus without resorting to nominations alone, ensuring a broad and fair evaluation scope.
Award Structure: A Category for Every Strength
To recognize diversity in contributions, RLC2024 introduced seven distinct award categories that capture different flavors of RL research:
- Theory of RL
- Scientific Understanding
- Empirical Methodology
- Applications
- Support Tools
- Resources
- Visionary Perspectives
This structure makes room for rigorous theory, replicable empirical work, and even societal impact.
Why do I raise RLC here in CSPaper.org?
The significance of RLC extends beyond a niche RL audience:
-
It demonstrates feasible reforms to the peer review process that can be adopted or adapted by other CS conferences. -
It nurtures young researchers by pairing them with senior mentors. -
It promotes open scientific dialogue by de-emphasizing impact speculation. -
🧩 It values contributions that are hard to quantify, like infrastructure and empirical rigor.
Looking Ahead
The second edition of RLC will take place August 5 to 8, 2025 at the University of Alberta, one of the foundational institutions of RL research. It’s clear that RLC is here to stay, not just as an academic venue, but as a living experiment in reshaping how we do science.
As other conferences grapple with scale and fairness, RLC offers a promising alternative: thoughtful, transparent, and community-driven. It will be fascinating to watch whether its ideas influence the mainstream, or remain a boutique success.
References:
