[Policy Spotlight] EMNLP 2025 Criteria for “Highly Irresponsible” Reviewers — What It Means & Why It Matters

Joanne

On 6 May 2025, the EMNLP 2025 Program Chairs released a post titled “Criteria for Determining Irresponsible Reviewers.” It complements the recent ACL Rolling Review (ARR) update requiring that
all qualified authors participate in reviewing.

If you are labeled Highly Irresponsible, you will
lose the right to submit or commit papers in the next ARR cycle (which includes EMNLP 2025).

Official Blog Post: https://2025.emnlp.org/blog/irresponsible-reviewers
ARR Policy Update: https://aclrollingreview.org/blog/2025-05-reviewer-volunteering

What Triggers the Label?

#	Behavior	Details	Consequence
1. Missed reviews	No review by the deadline and no timely emergency declaration.	Auto-flagged as “Highly Irresponsible”.	Ban from next ARR/EMNLP cycle
2. Terse or unprofessional reviews	For good-faith papers: giving only 1–2 lines, with no constructive feedback. Or using rude / discriminatory language (I4 violations).	Judged case-by-case by Program Chairs.	Possible ban
3. LLM-generated reviews	LLMs can be used for paraphrasing and grammar, not for generating review content. Papers cannot be fed to commercial LLMs that store data.	Action only if there’s “no reasonable doubt” the review was LLM-generated.	Possible ban

What is a good-faith submission?

States a clear contribution.

Provides some evaluation or evidence.

Writing is understandable enough to recognize the contribution.

If there's zero evaluation or citations are completely unrelated to the field, it's not good faith and only needs a brief review.

Sanction & Appeal

Flagging follows ARR policy.
Final decisions made by Program Chairs.
You can appeal to the ACL Publication Ethics Committee.

What Should You Do?

Role	Tips
🧑‍ Authors	* If you submit to EMNLP 2025, expect to be assigned as a reviewer.* If you're unavailable, file an emergency declaration before the deadline.
🧑‍️ Reviewers	* Even rejections must include clear, civil, and specific feedback.* LLMs may help with grammar, but not content , and never share papers with tools that store data.
Area Chairs	* Meta-reviews must also follow these rules, late or rude ones may cost you future submission privileges.

Open Questions

Here are a few discussion points to get us started:

Is a one-cycle ban a strong enough deterrent?
How can organizers detect LLM-generated reviews reliably and fairly?
️ Could strict enforcement slow down the review process?
Will authors start gaming the “good-faith” definition?

What do you think? Share your thoughts and experiences below

Joanne

The original post from EMNLP 2025

Criteria for Determining Irresponsible Reviewers

This post accompanies the ARR post Changes to reviewer volunteering requirement and incentives, and defines the specific criteria for what we will deem as “Highly Irresponsible” reviewers. While the focus here is reviewers, we will use similar criteria for determining irresponsible Area chairs (ACs).

Non-submitted reviews
If a reviewer fails to submit their reviews by the official deadline and has not submitted a personal emergency declaration (note: declaring a personal emergency after the review deadline will not be considered) will automatically be flagged as “Highly Irresponsible”.
Extremely terse or unprofessional reviews
Where the submissions are good-faith work that merits a serious review (otherwise a short review can suffice, assuming it clearly explains the fundamental problems with that work), reviews that only contain a single argument (1-2 sentences) and no constructive feedback should be flagged. We may also penalize reviews that are extremely unprofessional in tone (e.g., rude, racist, sexist, ableist, etc. content; I4 in the list of 12 commonly reported issues), even if they are otherwise detailed.

Here are some guidelines for determining whether to consider a submission to be in good faith: At minimum, a good faith article states the contribution up front and provides an evaluation that supports that. If the writing is so poor that the intended contribution can’t be identified or the article is missing an evaluation positioned as supporting that, then the article does not warrant a serious review. If the issue is just that the stated contribution is not clear, or the evaluation is not sufficient or rigorous enough, that does warrant a serious review. Furthermore, if the paper shows a naivete about the state of the art, the paper still warrants a serious review, but if the paper shows a complete lack of awareness of work in the field (for example, if virtually all of the citations are from another field), then the paper is not a good faith submission. Even interdisciplinary papers should show an awareness of the audience they are submitting their work to.

LLM-generated reviews
As per the ACL Policy on Publication Ethics, it is acceptable to use LLMs for paraphrasing, grammatical checks and proof-reading, but not for the content of the (meta-)reviews. Furthermore, the content of both the submission and (meta-)review is confidential. Therefore, even for acceptable purposes such as proofreading, it must not be passed on to non-privacy-preserving third parties, such as commercial LLM services, which may store it.

Authors will be able to flag such cases and present any evidence they have to support their allegation. While there is no definitive way of determining whether a review was (entirely) generated by an LLM, the Program Chairs will review the evidence and only proceed in cases where there is no reasonable doubt.

Flagging review process
The process is specified in the ARR post. Ultimately, all decisions will be made by the Program Chairs after a careful review of all evidence. Reviewers/ACs will be able to appeal to the publication ethics committee1 if they want to dispute the Program Chairs decisions.

https://www.aclweb.org/adminwiki/index.php/Process_for_ACL_Publication_Ethics_Review

Updated: May 06, 2025

CSPaper: review sidekick

[Policy Spotlight] EMNLP 2025 Criteria for “Highly Irresponsible” Reviewers — What It Means & Why It Matters