Interesting research that got accepted by EMNLP 2023 findings.

It is heating up. The scale and tooling for peer review will have to catch up.

Thanks for raising this important topic! Reproducibility is indeed crucial, especially in ML, and something our community needs to actively improve. Encouraging clearer standards during peer review could definitely help. Platforms like "Papers Without Code" are great initiatives to highlight issues. The pushing for open code, better documentation, and transparency is an important effort. Curious to hear more thoughts from everyone: What practical steps could reviewers or conferences take to boost reproducibility?

Thanks for sharing these thinking! I totally resonate with your points, especially about incremental research still being valuable. Not every paper can be paradigm-shifting, and recognizing solid, incremental progress helps keep science moving forward. Plus, the emphasis on methodological rigor and ethical considerations is spot-on. Peer review isn’t easy, but clear guidelines like these definitely make the process smoother ...

Really interesting thought experiment! Compared to other fields such as medicine, I think it is a very good thing that there are usually no or very low processing feels for getting an article published. This really opens up the research to everyone. Compensating the reviewers would make it difficult to keep the fees low. Plus, the improvements in review quality seem to be rather marginal! What do you thing universities/research departments could do to incentivise better reviews?