This study investigates the difficulty of interpreting peer review feedback, comparing the effectiveness of numerical scores versus text in predicting acceptance. The research reveals that score-based models are significantly more accurate (91%) than text-based models (81% even with LLMs), indicating textual information is considerably less reliable.
This paper argues against using current AI systems for peer review, identifying two critical issues: a "hivemind effect" that reduces perspective diversity and the trivial gameability of AI review scores through paper rewriting. Empirical comparison of human- versus AI-generated reviews shows that AI reviewers are susceptible to stylistic changes rather than scientific merit, highlighting the need for non-gameability and review diversity for automation.
The post asks for predictions on the average score threshold required for paper acceptance at ICML 2026, with author notifications set for April 30th. It's a query about the acceptance benchmark for an AI conference.
The title suggests a discussion about how the 'conference lottery culture' might be detrimental to research quality. It addresses concerns that the selection process at conferences negatively impacts scientific progress.
A user asks the community about previous experiences submitting to the CHI PLAY conference, inquiring about the helpfulness of the reviewers and the quality of the feedback.