ICLR 2025 Townhall: What You Need to Know About This Year's Review Drama (and Dreams)

[root]

At ICLR 2025, the heart of the conference beat strongest during the open townhall session — an event where transparency, future vision, and a touch of controversy collided.

Facing the audience were key members of the organizing committee, they opened the floor to hard questions, soft grumbles, and spirited discussions.

---

Reciprocal Reviewing — a bold new policy

The first major topic introduced was the Reciprocal Reviewing Policy. Aimed at balancing the reviewing load, this year's rule required that:

• Authors submitting 3+ papers must serve as reviewers.
• Every submission must have at least one author willing to review.

However, exemptions applied: serving organizers (e.g., SACs), those with major conference roles, first-time submitters, and a few others were excused.

The new system slashed reviewer burden significantly:

• Over 21,000 reviews completed.
• Each reviewer handled about 2.1 papers on average, dramatically easing previous years' heavy loads.

---

Do Author-Reviewers Judge Harsher?

The organizers didn't shy away from confronting a prickly question: "Are authors who are required to review harsher?"

A bar chart presented next suggested: Yes, slightly.

• Author-reviewers tended to give marginally lower scores than non-author reviewers.
• The effect was most visible for authors submitting 3+ papers, likely the most prolific participants.

But the story didn't end there.

Final Decisions Smooth Things Out

The chart to the right above gave some relief.
Once the final decision passed through Area Chairs (ACs), the difference mostly disappeared.

Regardless of whether reviewers were authors or not, the final acceptance decisions (reject, poster, spotlight, oral) balanced out fairly.
The checks and balances in place worked as intended, smoothing individual bias with committee wisdom.

---

Ethics Reviews: concerns and categorizations

Another major frontier discussed was the Ethics Review Process.

Before discussions, ethics concerns were messy and loosely categorized:

• Attack methods without mitigation
• Potential for misuse
• Data legality issues
• Human subjects issues
• ...and a lot of “no details.”

After careful clustering and discussions, a cleaner set of concerns emerged:

• Legality of data (the biggest concern, with 38 flagged cases)
• Harm (misuse) and privacy were also substantial worries.
• A few reports turned out to be false positives: cases flagged without real cause.

The organizers also pointed out a growing trend: Ethical reviews themselves are sometimes written by LLMs! (with a good-humored chuckle from the audience.)

---

Plagiarism: a wake-up call

Then came a slide that drew audible gasps: Plagiarism.

ICLR 2025 detected 30+ cases of copying key technical ideas, not just related paragraphs or figures.
Many cases were self-reported by "victimized" authors during the review phase.
The board promised serious scrutiny, and possible penalties, depending on the severity.

The organizers emphasized the layered nature of plagiarism, ranging from copying related work to full idea theft—redrawing the boundaries of acceptable academic conduct.

---

Future of AI in Reviewing: a careful dance

Toward the end, a brave attendee asked a forward-looking question (yours truly!):
"What about using AI to facilitate official reviews in the future?"

The committee’s stance was cautious but not dismissive:

• For now, AI helps internally, scrutinizing reviews to flag suspicious patterns.
• For example, this year, your reviews might have been quietly double-checked by AI (cspaper.org/topic/52).

But as for fully AI-written reviews or replacing human judgment?
Not yet. The consensus was clear: trust and nuance are still very human domains, for now.

That said, one could imagine a powerful future tool: review simulators.
Especially for new researchers or those venturing into unfamiliar territories, an AI-powered simulated reviewer could:

• Provide early, brutally honest feedback.
• Highlight overlooked weaknesses.
• Allow authors to self-correct before facing real reviewer panels.

Such a tool could level the playing field dramatically, especially for junior researchers outside established networks.

---

In Closing:

ICLR 2025's townhall laid bare a conference at a fascinating crossroads:

• Building better reviewing processes while wrestling with workload, fairness, and ethics.
• Standing cautious but curious before the new possibilities AI offers.

If you're planning to submit next year, or just love seeing where the cutting edge of academic culture meets technology, you’ll want to keep a close eye on what comes next.

Because at ICLR, the future of how we review research might just be as groundbreaking as the research itself.

---

Welcome to repost there is no need of permission from CSPaper as long as the original url and source is shared at the same time.

You are more than welcome to Register (verified or anonymous) to Join the Discussion

[Joserffrey]

Have the ICLR 2025 organizers completely forgotten that they claimed on their official website that they would select high-quality reviewers and that these reviewers “will receive special acknowledgment during the opening ceremony and free registration”? They didn't do what they were supposed to do, yet they are still talking about AI review. Didn't they realize that they had unfinished business when they held this town hall meeting?

https://iclr.cc/Conferences/2025/ReviewerGuide

[root]

@Joserffrey

Oh dear, they did not mention this at all during the townhall I am sure!

What a shame if they did not do this at the opening ceremony ...

I wonder if you have checked with the organizing committee? such as writing an email (https://iclr.cc/Help/Contact) to check it out?

[Joserffrey]

@root

Thank you so much for your advice. I sent a message two days ago through the same link as you provided, but I have not received any reply yet. In addition, I have asked several times on Twitter (X), but I still have not received any reply.

Now this issue has attracted the attention of the CVPR/ICCV'25 technical chair, but not many people in the community are aware of it yet. I would like to ask if you could forward this issue on Twitter (X) and more platforms so that more people will notice it, and then we will have a chance to bring this issue to the attention of ICLR'25 organizers and potentially get a reply.

Now it seems that "That's a wrap", and organizers have left "satisfied". None of them have replied to me about this. I am concerned that if no more people raise this issue, the ICLR organizers may tend to ignore it. I cannot find any other avenues or more people to raise this issue. Thanks again.

Links:
https://x.com/yoshitomo_cs/status/1917299455327625397

https://x.com/joserf28323/status/1916989116497162483

https://x.com/joserf28323/status/1916834181537296652

[RiskLimp]

@Joserffrey said in ICLR 2025 Townhall: What You Need to Know About This Year's Review Drama (and Dreams):

Have the ICLR 2025 organizers completely forgotten that they claimed on their official website that they would select high-quality reviewers and that these reviewers “will receive special acknowledgment during the opening ceremony and free registration”? They didn't do what they were supposed to do, yet they are still talking about AI review. Didn't they realize that they had unfinished business when they held this town hall meeting?

https://iclr.cc/Conferences/2025/ReviewerGuide

I hope someone raised the question to the committee onsite! this is not okay! I posted to a reddit thread about ICLR 2025 review decision: https://www.reddit.com/r/MachineLearning/comments/1gov5zd/d_iclr_2025_paper_reviews_discussion/?sort=new

[root]

@Joserffrey Thanks for sharing more information to the community here. Have you or anyone you know tried to reach out directly to the general chair of ICLR 2025? Yisong Yue (http://www.yisongyue.com/)

[Joserffrey]

@RiskLimp

Now ICLR 2025 has concluded, and it appears that the organizers have wrapped up their efforts and are no longer responding to inquiries. This is really terrible, especially under the current reviewing circumstances. More importantly, they claimed this on the official website. A recognition would encourage more high-quality reviews in the future.

[Joserffrey]

@root

Thanks. We have contacted almost all program chairs via Twitter comments, direct messages, and @ them directly on posts, with as well as the general chair Yisong Yue you mentioned. However, we have not received any response so far. But we have not sent them emails yet. We only sent an email to the senior program chair, Carl Vondrick, for this issue, still no any response.

[Joanne]

Update Regarding ICLR 2025 Reviewer Acknowledgment Issue
We have noted concerns raised by @Joserffrey and confirmed by @root regarding the apparent oversight by the ICLR 2025 organizing committee in acknowledging reviewers as promised. We have officially contacted the organizers seeking clarification.
We’ll update this post once we receive an official reply. If you have further information or feedback, please comment below.

[Joserffrey]

@Joanne

First, thank you very much for your attention and support!

As mentioned above, ICLR 2025 officially promised to give special thanks to acknowledged reviewers at the opening ceremony and waive their registration fees on the website. Given that this promise has been ignored and now the conference is ended, neither of these two claims has been fulfilled.

In light of this situation, we have the following suggestion to convey to the organizers:
(i) We suggest the organizing committee respond this first and announce their nominated list of acknowledged reviewers both on the official website and social media accounts. This addresses the first point: since the in-person conference has already concluded, publicly recognizing the reviewers on social media accounts could potentially the best way to make amends.
(ii) Consider refunding the registration fees paid by the acknowledged reviewers case-by-case.

Thanks again for your great help. I have one more question: I just found this website recently and discovered it through someone sharing the ICLR 2025 townhall. I'm not sure which organization maintains this platform. Are you the administrators? This is a great find, and thank you for your contributions! I look forward to sharing and exchanging ideas with everyone here.

[root]

Thank you, @Joanne, for reaching out to the ICLR committee, much appreciated!

@Joserffrey, this non-profit community platform was recently created to support open discussion around peer review practices in the computer science research community — from broad systemic issues to debates on specific papers. @Joanne is currently helping coordinate and manage content on this platform.

We, along with many others in the CSPaper community, are more than happy to support important concerns like the one you’ve raised. Let’s continue using this space to foster transparency and constructive dialogue in research.

[Joserffrey]

Thanks @root for your reply! It's great that we can discuss both general issues in peer review and specific papers. Excited to be a member of the CSpaper community.

Let's wait and see what will happen with the ICLR committee. It's really surprising that this happened for such a top-tier conference. Actually, I noticed something strange two weeks before the conference. At that time, the “Reviewers” section under the “Organization” tab on the website was still empty, just like in previous years, waiting to be updated. However, during a subsequent update, this tab was completely removed, and all information about the ACs and reviewers was put to the “Program Committee” part. I can't help but suspect that they intentionally ignored the issue regarding the nomination of outstanding reviewers as well as the waiving of their registration fee...

[Joanne]

@root Thanks for the introduction.
Thank you @Joserffrey for sharing these additional observations—I didn't notice that change on the website myself, but what you describe does raise some valid concerns. It’s surprising and disappointing to see this happening at a conference of ICLR’s caliber.
Given the crucial role reviewers play, transparency around recognition is really important. If the committee doesn't properly address these issues, it sets a concerning precedent. I hope the organizers realize the importance of openly resolving this soon.
Thanks again for your keen observations and for highlighting these details. Let's continue to keep an eye on any further developments.

[Joserffrey]

Totally agree! I'm truly disappointed with how ICLR treated the reviewers this time. Hope they've already got the nominations sorted and just haven't announced them before the conference. If they have done nothing and are still ignoring it, that will be really messed up.

Thank you @Joanne once again for your help. Yes, Let's see what happens next.

[Joserffrey]

An update: We also sent emails to several Program Chairs yesterday, but after 24h have not received any replies yet.

[root]

@Joserffrey Thanks for the update! Let's see how does this evolves. I'd say “Don’t make promises you can’t keep, and ignoring people is a coward’s move.”