ICML25: A Small Step Toward OpenReview, A Big Step for the Community
-
Just noticed that ICML 2025 has taken a small but meaningful step toward OpenReview: the reviews of accepted papers will eventually be made public. While this isn't full-fledged open review yet, it's a clear signal that change is coming.
As a reviewer myself, I felt overwhelmed by the sheer volume of submissions this year. Unfortunately, I felt a noticeable drop in quality. Some papers were clearly submitted in a "let's try our luck" fashion. In this context, I sincerely hope that top AI/ML conferences will eventually follow ICLR's model and adopt fully open peer review.
Why Open Review Matters
-
For Reviewers: Knowing that reviews will be public adds a layer of accountability. It encourages more thoughtful, constructive, and responsible feedback. No more careless 1-scores or copy-pasted comments.
-
For Authors: When reviews are public, authors will think twice before submitting undercooked ideas. Fear of negative reviews being visible online can act as a natural filter to avoid "lottery-style" submissions.
-
For the Community: Public reviews help newcomers learn how to write better papers and better reviews. It also reduces the burden on reviewers caused by the "Fibonacci submission strategy" (endless revise-and-resubmits across top venues), and ultimately improves the quality of accepted papers.
Final Thoughts
Open review isn't a silver bullet, but in this era of exploding submission numbers, it’s a change worth pursuing. I hope to see more top-tier conferences move toward transparent and accountable reviewing, bringing the focus back to research quality, not just acceptance rates.
-
-
Just received this email from ICML: 
While your ICML 2025 submission has not been accepted for presentation at the conference, you have the choice to opt-in to make the rejected paper public on OpenReview ... ...
This is a good move towards transparency in the peer review process. By allowing authors to make their rejected papers and associated reviews public, ICML is fostering an environment where the community can learn from both successes and failures.
Why Consider Opting In?
- Learning Opportunity: Sharing your paper and its reviews can provide valuable insights to others in the community, especially newcomers, about common pitfalls and areas for improvement.
- Transparency: Making the review process public can help demystify the decision-making process and promote fairness.
- Feedback for Improvement: Public reviews can offer constructive criticism that can be used to refine the work for future submissions.
Considerations Before Opting In
- Public Exposure: Be prepared for your work and its critiques to be publicly accessible. This transparency can be beneficial but may also be uncomfortable for some.
- Future Submissions: Ensure that making your paper public does not conflict with the policies of other conferences or journals you may wish to submit to in the future.
⸻
This initiative aligns with the broader movement towards open peer review, as seen in conferences like ICLR. While it’s not a complete shift to open review, it’s a meaningful step in that direction. By embracing transparency, we can collectively improve the quality and fairness of the peer review process.
-
For authors who resubmitted their work to NeurIPS:
Although opt in making the ICML rejection public can drive transparency and demostrating improvement. However, you need to think about the Cons too.
There’s research (https://arxiv.org/pdf/2011.14646) showing reviewers tend to score resubmissions lower if they know it got rejected before, even if it’s the exact same paper. Plus, having the rejection out there might give the impression the paper’s not great quality, consciously or subconsciously affecting reviewers.
