Hey fellow KDD authors and ML researchers!

As we approach the release of the second-round review results for KDD 2025, it's time to gather, share experiences, and support each other: whether you're nervously checking for updates, celebrating a win, or figuring out your next steps.

🧩 What to Discuss Here:

• What scores did you get after Round 1 & rebuttal?
• Did your rebuttal help push your paper toward acceptance?
• Were you assigned to Research or ADS track?
• Are you seeing trends in novelty / technical quality (TQ) scores?
• What are your thoughts on this year's review quality?

---

️ Be cautious with rebuttals

Some authors reported issues with anonymous external code links (e.g., GitHub repos). Even if anonymized, external links in the rebuttal can trigger desk rejection depending on PC interpretation. If you’re unsure, it’s safest to:

• Clearly reference what's already in the submission
• Avoid linking out to anything not explicitly allowed
• Clarify any confusion in the rebuttal without adding new external content

---

Community Polls & Stats

Some have started collecting anonymized data points on scores and acceptance results; it's great to get a sense of where you stand. If you’ve got numbers (e.g., Novelty: 4 3 3 2, TQ: 3 2 3 2), feel free to drop them in the thread and compare notes!

Let’s try to keep this thread constructive and supportive. Every score is a story, and every rejection can be a redirection.

---

Looking Ahead

Whether you’re aiming to get into camera-ready or preparing a resubmission, this is the perfect moment to share, learn, and connect with others in the same boat.

Feel free to comment below with your situation, ask questions, or just vent — we’re here for it!

Stay strong, and good luck to everyone

— A fellow author + reviewer

Interesting research that got accepted by EMNLP 2023 findings.

We’re excited to introduce a new category on cspaper.org:
Anonymous Sharing & Supplementary Materials

Purpose

This category is designed specifically for anonymized sharing of supplementary materials such as:

• Additional experimental results or figures
• Extended ablation studies
• Links to datasets or demos
• Supplementary explanations that didn’t fit in the main paper

It’s especially useful during rebuttals, when you may need to share extra content with reviewers but remain compliant with anonymity and strict page limits.

---

How It Works

• Only unverified (but registered) users can post in this category: make sure to skip filling your email during registration.
• You can edit or delete your post anytime
• Use a username that doesn’t reveal your identity
• Share the link to your anonymous supplementary materials post/topic

This keeps everything in line with double-blind peer review policies while giving you a legitimate way to share supporting materials.

---

️ Stay Anonymous

Please do not use real names or affiliations when posting.
If you're unsure how to create an anonymous username, try one of these generators:

• Jimpix Random Username Generator
• UsernameGenerator.com
• NordPass Username Generator
• Dashlane Username Generator

---

Example Use Case

1. You're writing the rebuttal for your paper submitted and reviewed by NeurIPS/ICML/ACL.
2. You need to trim down your rebuttal to fit the rebuttal length limit. But, you want to share detailed results or code (causing violation of rebuttal length) with reviewers during the rebuttal.
3. Just post them anonymously in that category and include the link in your response!

Pro Tip

you can make your url even shorter by leveraging services like TinyURL and Bitly.

---

Start Posting

Visit the category here and share your materials:
https://cspaper.org/category/10/anonymous-sharing-supplementary-materials

Originally posted on Bluesky by Serge Belongie (@serge.belongie.com), this timely thread opens a conversation about decentralizing paper presentations at NeurIPS.

---

🧭 The Poll That Sparked a Discussion

Serge Belongie (University of Copenhagen) and Søren Hauberg (DTU) are exploring a simple but profound question:

Would you present your next NeurIPS paper in Europe instead of traveling to San Diego (US), if this was an option?

They’re collecting data to gauge interest in offering official NeurIPS paper presentations at a European location, without requiring long-haul travel to the US.

Link to participate in the poll (Google Form)

---

🧠 Why This Matters

Several compelling motivations were outlined:

• ️ Environmental Impact: Transatlantic flights carry a heavy carbon footprint.
• Family Considerations: Long travels can be difficult for researchers with children.
• US Entry Concerns: Visa policies and border control have become increasingly strict and unpredictable.
• Conference Scale: NeurIPS has grown large — some argue, too large for meaningful in-person interaction.

---

🧩 Community Reactions

The thread generated a flurry of responses from researchers around the world:

• Concerns about fragmentation:

“Splitting the community weakens the social connections.” — Kosta Derpanis

• Necessity due to constraints:

“Many US-based researchers fear not being let back in if they travel.” — David Marx

• Existing precedents:

“Since the pandemic, we’ve had NeurIPS@Paris and CVPR@Paris satellite meetups.” — David Picard

• Institutional support growing:

IJCAI 2025 will officially include a satellite event in Guangzhou, China to accommodate researchers unable to enter Canada.
→ IJCAI Announcement

• Feasibility discussions already underway:

“There is discussion of a NeurIPS 2025 satellite outside the US but in a similar timezone.” — Thomas Dietterich

---

Implications for Peer Review & Conference Design

This idea challenges the default notion of “main venue” = “main stage.” As the global AI/ML community grows, inclusivity and accessibility in paper presentation become just as critical as peer-review rigor. Should presenting a NeurIPS paper require thousands of dollars and international visas?

While decentralization may create new logistical and identity challenges, it could also lead to:

• More inclusive participation
• Better environmental stewardship
• Stronger regional AI communities

---

What Do You Think?

Would a European satellite venue dilute the prestige of NeurIPS, or increase its relevance and reach?

Is it time for flagship CS conferences to rethink hybrid and distributed formats, especially post-pandemic?

Drop your thoughts below.

---

Special thanks to @serge.belongie.com for initiating this much-needed discussion.

I posted more astonishingly funny reviews here:

https://cspaper.org/topic/26/the-icml-25-review-disaster-what-does-k-in-k-nn-mean

The recent ICML 2025 review cycle has sparked outrage and dark humor across the ML community. Here’s a compilation of jaw-dropping anecdotes from Zhihu (China’s Quora) exposing the chaos, ranging from clueless reviewers to systemic failures.

Buckle up!

---

1. The "k-NN" Incident

User "写条知乎混日子" dropped this gem:

"The reviewer asked me: ‘What does the ‘k’ in k-NN stand for?’"

Yes, a reviewer at ICML, a top-tier ML conference, needed clarification on one of the most basic ML concepts.

---

2. The "Pro vs. RPO" Mix-Up

User "CpGD7" shared:

"The reviewer misread ‘rpo’ as ‘pro’ and questioned why our ‘advanced version’ lost to baselines. Next time, should I rename my main experiments ‘Promax’ to get accepted?"

---

3. The "I Didn’t Have Time to Check Proofs" Confession

User "虚无", a reviewer, admitted:

"I got assigned 5 theoretical papers. Checking proofs properly takes 7–10 days per paper. I only had time to verify the first two; the rest got high scores based on ‘intuition’ because I couldn’t validate the math."

This raises a serious ethical concern: Papers are being accepted/rejected based on guesses, not rigor.

---

4. The "Citation Mafia" Reviewer

User "Jane" reported:

"A 1-star reviewer demanded we cite 7 unrelated papers — 6 of which were by the same author. We withdrew the submission."

---

5. The "I Review Papers in a Field I Don’t Understand" Dilemma

User "Highlight" (a biochemist) was roped into reviewing ML theory:

"I’m from a biochemistry background. They assigned me 5 ML papers. I’m scrambling to understand the math over the weekend. They must be desperate."

---

6. The "R is Not the Real Numbers" Debacle

User "better" vented:

"A reviewer complained: ‘What is 𝐑? You never defined it. It can’t possibly mean the real numbers!’ …What else would it be?!"

---

7. The "Dataset Police" Strike Again

User "Reicholguk" faced this absurdity:

"A reviewer demanded we test on ‘popular’ datasets like Cora/Citeseer, ignoring that we already used Amazon Computer and Roman Empire graphs (which are standard in our subfield). Is this reviewer an AI? Even AI wouldn’t be this clueless."

---

8. The "I’ll Just Give Random Scores" Strategy

Many users reported suspiciously patterned scores:

• "877129391241": "One of my papers got no reviews at all (blank). Another got all 1s and 2s."
• "陈靖邦": "Got 4443 after an ICLR desk reject. Is this luck or a sign reviewers just clicked randomly?"

---

Why This Matters

These aren’t just funny fails, they reveal deep flaws in peer review:

• Overworked reviewers (5+ papers, no opt-out).
• Mismatched expertise (biochemists judging theory).
• Lazy/bad-faith reviews (no comments, citation demands).
• Systemic randomness (scores with no justification).

As User "虚无" warned:

"If ICML keeps this up, no serious researcher will want to submit or review."

---

The Big Question

Should top conferences like ICML:
Cap reviewer workloads?
Allow expertise-based opt-outs?
Penalize low-effort reviews?

What’s your take? Share your worst review horror stories below!

(Sources: Zhihu users "877129391241", "虚无", "CpGD7", "陈靖邦", "Jane", "Highlight", "better", "Reicholguk", "写条知乎混日子". Original posts here.)

Result are released! I can provide a few data points that are visible to me:

• A paper about Counterfactual Explanation: 1, 3, 2, 2
• A paper about offline RL: 2, 2, 2
• A paper about agentic assitant: 2, 2, 1

@cqsyf it is a paper accepted by ICLR, but yeah ... I get the point! Wish all best of luck!

@river according to the information you posted earlier, this paper actually got an average score of 6.66 from ICLR workshop.

Since we all know workshop has a lower bar than ICLR main track, the AI score is not far from reality I guess??

AI Review for https://arxiv.org/abs/2503.10617v2

Summary

This paper introduces a new method, Compositional Subspace Representation Fine-tuning (CS-ReFT), designed to adapt large language models (LLMs) to multiple tasks while avoiding cross-task interference. The key innovation of CS-ReFT is its use of multiple orthonormal subspace transformations, each specialized for a distinct skill or task, and a lightweight router that composes these subspaces dynamically. The approach is evaluated on the AlpacaEval benchmark, where it demonstrates a high win rate compared to both larger models and other parameter-efficient fine-tuning methods.

Soundness

2

Presentation

3

Contribution

2

Strengths

• Novel Method: Introduces CS-ReFT to address cross-task interference in multi-task learning using multiple orthonormal subspace transformations and a lightweight router.
• Unique Approach: Distinguishes itself from traditional weight-based methods like LoRA by applying representation editing.
• Clear Structure: The paper is well-organized and clearly written, with a logical flow from the introduction to method, experiments, and conclusions.
• Visual Aids: Uses mathematical formulations and diagrams to effectively explain the subspace transformations and router mechanism.

Weaknesses

• Scalability Concerns: Does not sufficiently discuss how CS-ReFT scales, especially in scenarios with a large number of tasks, as maintaining separate low-rank transformations may become computationally and storage-intensive.
• Interpretability Issues: Lacks an analysis of the interpretability of the router's decisions. Insights into why specific weights are assigned to different subspaces could enhance trust in the model's behavior.

Questions

• Scalability: How does CS-ReFT perform as the number of tasks increases? Are there strategies to optimize or reduce the computational overhead of maintaining separate subspace transformations for each task?
• Router Interpretability: Can the router's decisions be interpreted or visualized to understand why specific subspaces are activated for certain inputs? Are there any patterns that emerge which could provide insights into the model's behavior?

Flag For Ethics Review

No ethics review needed.

Details Of Ethics Concerns

(None)

Rating

5

Confidence

4

@xiaolong yeah, right? Hopefully we do not hear such news in 2025 and onwards…

Thanks for raising this important topic!

Reproducibility is indeed crucial, especially in ML, and something our community needs to actively improve. Encouraging clearer standards during peer review could definitely help. Platforms like "Papers Without Code" are great initiatives to highlight issues.

The pushing for open code, better documentation, and transparency is an important effort. Curious to hear more thoughts from everyone: What practical steps could reviewers or conferences take to boost reproducibility?

Since 2021, ACL has moved away from Softconf and introduced the new Rolling Review system (ARR). Official website: https://aclrollingreview.org

Rolling Review divides paper submissions into two phases:

1. Submit your paper to the ARR system (“make submission”). Reviews will be provided after one month.
2. Commit your submission from step 1 to your target conference.

This means you initially submit your paper to the ARR system rather than directly to the conference, allowing flexibility in choosing conferences afterward.

For a crash course on ARR review paradigm, you can watch this video by Shubhra Kanti Karmaker Santu:

Youtube Video

The peer review process is the backbone of academic publishing — but how reliable is it, really?

NeurIPS 2021 revisited the interesting 2014 NeurIPS Experiment to measure how consistent the review process is today. The results? Still noisy, still unpredictable!

Youtube Video

Key Findings:

• 23% of papers were accepted by one committee but rejected by another.
• 50.6% of accepted papers would have been rejected if reviewed by another committee.
• Over half of spotlights were rejected by the other committee—raising questions about prestige paper selections.
• Increasing selectivity amplifies randomness, making the review process even more arbitrary.
• Despite a 5x increase in submissions since 2014, inconsistency remains unchanged.

These findings highlight the ongoing challenges in peer review for computer science research. Are we truly selecting the best work, or just rolling the dice?

What do you think? Join the discussion: how can we improve peer review in ML & CS research?

This post is a translation and repost from an original article by Sakura on Zhihu (original link).

Receiving rejection from top-tier conferences such as CVPR is an experience shared by nearly every researcher in our field. While each rejection feels uniquely personal, the frustration, hope, anxiety, and eventual disappointment described in this article resonate deeply within our research community. By sharing Sakura's experience, I hope to offer comfort to others facing similar moments, reminding everyone that these setbacks, though painful, are common milestones on our academic journeys.

---

Initially, I wasn’t planning to write this blog post. However, I realized many researchers might resonate deeply with my experience. If reading this brings you some comfort or connection, then sharing this experience was worthwhile.

Receiving a rejection from CVPR is emotionally complicated. Honestly, when I first submitted this paper, I didn’t have high expectations. But then my senior lab-mate said confidently, "CVPR acceptance should be pretty safe this time. Once accepted, you can expand it for TPAMI and even consider applying for talent programs at top tech companies." That encouragement filled me with excitement, pride, and hope.

When the initial scores came out (scored 3, 3 and 4), I checked with others around me and found mine seemed to be among the highest (or maybe I just asked fewer people!). The email arrived at midnight, waking me from shallow sleep. Immediately, I jumped into group chats, exchanging opinions about reviewer comments. My senior lab-mate, always having a different sleep pattern, quickly drafted a rebuttal. Unable to fall back asleep, I started drafting detailed rebuttal notes, outlining experiments needed and responses for each point raised by reviewers. After an exhausting night, I briefly rested before diving back into work.

We had three reviewers: one gave four points (mostly minor comments on typos and figures), and two others gave three points each. One asked for additional ablation studies, and the other misunderstood key aspects of our paper, requiring further clarification and experiments.

Rebuttal coincided with the Lunar New Year holidays. To save precious rebuttal time, I canceled planned journey home, opting instead for expensive direct flights. The entire festive period was consumed by many discussions with my senior lab-mate and advisor, carefully revising each rebuttal response multiple times. We carefully and politely addressed the factual errors made by the second reviewer, offering gentle corrections and gracious statements (somewhat like politely negotiating after an argument with your significant other!).

After submitting the rebuttal, the reviewer modification period began. I wasn't expecting significant score changes, except perhaps from the second reviewer whose concerns we addressed extensively. I anxiously refreshed OpenReview daily. Eventually, the second reviewer indeed modified their review, fueling my optimism.

The wait afterward felt endless. As the announcement date (February 26th) approached, anxiety peaked. I stayed awake until 3 a.m., refreshing emails repeatedly. Eventually, fatigue overtook me, but every hour I'd wake again, checking for updates. The official announcement was delayed without clear communication, adding to the frustration.

The results finally appeared around 4 a.m. on February 27th. Seeing "Reject" on OpenReview crushed me . It was extremely difficult to accept this outcome, especially given the high initial hopes and reassurance from everyone around me. I immediately called my senior lab-mate, who shared my disbelief but offered comforting words: "Take it easy, treat yourself to something good, and adjust your mindset."

It took an entire day to process the rejection, talking it through with family, my girlfriend, and colleagues. Eventually, acceptance set in. However, I still eagerly awaited the meta-review for clarity, hoping at least for constructive insights.

Sadly, the meta-review shifted my feelings from disappointment to utter frustration:

Reviewer 1: "The rebuttal has solved some concerns about experiments. I also keep the original rating."

Reviewer 2: "Thanks for the reply. The authors have addressed my concerns. Therefore, I maintain my score."

Reviewer 3: "I appreciate the authors' feedback, which addressed my major concerns. As such, I prefer to maintain my original rating."

The Area Chair concluded:

"Three reviewers evaluated this paper, two borderline and one positive recommendation. All reviewers agreed that experimental results were insufficient or less impressive compared to existing works. Despite addressing concerns clearly in the rebuttal, reviewers did not actively discuss or adjust their scores."

Reading this was incredibly frustrating: reviewers acknowledged our efforts but simply chose not to reconsider their decisions?!

My experience seems common among researchers. In conclusion, I'd like to share an encouraging quote my senior lab-mate gave me:

"If this place doesn’t appreciate your work, another surely will!"

The NeurIPS 2024 Best Paper controversy surrounding Tian Keyu and ByteDance reveals a deeper and complex intersection between academia, industry, and ethical accountability.

Tian, a PhD student from Peking University, achieved widespread acclaim by winning the Best Paper award at NeurIPS 2024 for his work, "Visual Autoregressive Modeling: Scalable Image Generation through Next-Scale Prediction." However, this achievement swiftly became overshadowed by ByteDance's lawsuit against Tian, alleging he maliciously sabotaged their AI research by introducing code that disrupted internal AI training processes.

This controversy sheds critical light on several deeper issues:

Academic Integrity and Ethical Misconduct

Tian’s actions, if proven true, represent an egregious breach of trust, posing serious ethical questions around the responsibilities researchers carry within collaborative projects (AI Base). His alleged misconduct didn't involve data falsification or plagiarism within his award-winning paper itself, but the ethical breach raises questions about the very integrity of the research environment. Academic integrity extends beyond accurate reporting; it encompasses the ethical conduct researchers must uphold while accessing resources and collaboration opportunities provided by others.

This incident echoes past scandals, such as the infamous sabotage of cell cultures by a University of Michigan postdoc, underscoring how career pressures can lead even talented researchers to unethical extremes (Discover Magazine). The repercussions in Tian's case, including a lawsuit and potential reputational ruin, emphasize that integrity violations carry consequences far beyond academia.

Corporate Influence and Research Ownership

ByteDance’s lawsuit highlights complexities arising from industry-academia collaboration (Reddit). While corporate partnerships significantly boost research through resources like computational power and large datasets, they introduce intricate issues around research ownership and intellectual property rights. ByteDance's quick, punitive response (a lawsuit seeking millions in damages) demonstrates corporate assertiveness in protecting internal research efforts, a stark contrast to the academic community's typically more discreet disciplinary approach.

This case recalls Google's high-profile conflict with Timnit Gebru over the publication of ethical AI research, reflecting a growing tension between corporate interests and academic freedom. ByteDance, unlike Google, didn't prevent publication but instead pursued aggressive legal action post-publication, highlighting an evolving dynamic where industry increasingly influences academic conduct through legal and financial leverage.

Scrutiny and Limitations of Peer Review

The episode also places NeurIPS's peer review process under scrutiny. While the double-blind review system is designed to assess scientific merit objectively, this controversy illustrates a critical blind spot: ethical misconduct that occurs beyond the content of the paper itself. With no current mechanism to detect off-paper misconduct, this reveals the limitations of the peer review system and calls for potential enhancements, such as broader ethical disclosures, increased transparency in the review process, or post-publication checks (Computational Complexity Blog).

AI Plus Info : "To address these weaknesses, potential reforms have been suggested, including greater transparency, stronger conflict-of-interest management, ethical disclosures, post-publication review procedures, and cultivating a robust ethical culture".

Broader Implications

Beyond this individual controversy, the intersection of corporate influence and academic research remains nuanced. Tian Keyu’s story reminds us that groundbreaking research can quickly become entangled in ethical and legal battles, profoundly impacting researchers' careers and academia-industry collaborations.

The AI community must recognize that groundbreaking research should always align with impeccable ethical standards, transparent peer review, and clear guidelines on corporate-academic collaboration.

Peer review is the backbone of academic publishing, yet it’s increasingly difficult to find reviewers willing to contribute their time. Delays in paper handling, superficial reviews, and reviewer fatigue are persistent issues — especially in fast-paced fields like computer science. Could monetary incentives help? A new study in Critical Care Medicine suggests that paying reviewers might be part of the solution.

The Experiment: Cash for Reviews

Researchers conducted a quasi-randomized, blinded trial to measure the impact of financial incentives on peer review completion rates. Over a six-month period, invitations to review were sent to researchers in two alternating conditions:

• Control Group: Standard invitation, no compensation.
• Treatment Group: Invitation offering $250 for a completed review.

What They Found

• Offering $250 increased the proportion of reviewers who accepted the request (52.7% vs. 47.8%).
• More invited reviewers completed their assignments in the incentivized group (49.8% vs. 42.2%).
• Reviews were submitted about a day faster on average.
• The quality of reviews, as assessed by journal editors, remained unchanged between the two groups.

Implications for Computer Science Peer Review

The study suggests that modest financial incentives can improve reviewer engagement without sacrificing quality. In computer science, where the pressure to publish is high and reviewing is often seen as an unpaid burden, this raises important questions:

• Would top-tier CS conferences benefit from compensating reviewers?
• Could financial incentives help reduce review backlogs in journals?
• Would an incentive system encourage more thoughtful and thorough reviews?

With AI-generated submissions on the rise and conferences struggling to secure qualified reviewers, experimenting with financial incentives might be worth considering.

Would you be more willing to review papers if offered compensation? Or would this compromise the integrity of the system? Join the discussion below!

Related reading: The $450 question: Should journals pay peer reviewers? -- Science

Thanks for your summary, interesting read!

I realize this Mamba paper was accepted later at 1st Conference on Language Modeling (COLM 2024)

The published paper can be found here: https://openreview.net/forum?id=tEYskw1VY2