GoodPoint : Learning Constructive Scientific Paper Feedback from Author Responses

Jimin Mun Chani Jung Xuhui Zhou Hyunwoo Kim Maarten Sap · 2026

Read GoodPoint : Learning Constructive Scientific Paper Feedback from Author Responses by Jimin Mun Chani Jung Xuhui Zhou Hyunwoo Kim Maarten Sap (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11924.txt.

Introduction

Mun and four colleagues study scientific paper feedback that authors actually act on, using author responses as success signals. They curate GoodPoint-ICLR: 19,000 ICLR papers with reviewer feedback annotated for validity and author action. Their GoodPoint training recipe fine-tunes on valid, actionable feedback and adds preference optimisation on real and synthetic pairs. On a 1,200-paper benchmark, a GoodPoint-trained Qwen3-8B lifts predicted success by 83.7 percent over its base model and tops similar-size models on matching golden human feedback, even beating Gemini-3-flash on precision.

Catalog record

CREATORJimin Mun Chani Jung Xuhui Zhou Hyunwoo Kim Maarten Sap
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60331
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11924

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GoodPoint : Learning Constructive Scientific Paper Feedback from Author Responses Jimin Mun Chani Jung Xuhui Zhou Hyunwoo Kim Maarten Sap Abstract While LLMs hold significant potential to transform scientific research, we advocate for their use to augment and empower researchers rather than to automate research without human oversight. To this end, we study constructive feedback generation , the task of producing targeted, actionable feedback that helps authors improve both their research and its presentation. In this work, we operationalize the effectiveness of feedback along two author-centric axes— validity and author action . We first curate GoodPoint-ICLR , a dataset of 19K ICLR papers with reviewer feedback annotated along both dimensions using author responses. Building on this, we introduce GoodPoint , a training recipe that leverages success signals from author responses through fine-tuning on valid and actionable feedback, together with preference optimization on both real and synthetic preference pairs. Our evaluation on a benchmark of 1.2K ICLR papers shows that a GoodPoint -trained Qwen3-8B improves the predicted success rate by 83.7% over the base model and sets a new state-of-the-art among LLMs of similar size in feedback matching on a golden human feedback set, even surpassing Gemini-3-flash in precision. We further validate these findings through an expert human study, demonstrating that GoodPoint consistently delivers higher practical value as perceived by authors. 1 1 1 We will release code, dataset, and trained models upon acceptance. 1 Introduction The growing use of LLMs across scientific research ( 18 ; 8 ; 4 ; 10 ; 17 ) has spurred proposals to automate core research processes, including peer review ( 26 ; 24 ; 13 ) . Yet unchecked automation risks eroding the critical judgment of researchers and the broader scientific community, ultimately degrading the quality of science ( 2 ; 21 ) . We argue that AI should instead empower researchers—particularly those who are traditionally disadvantaged ( 14 , junior and/or non-native English speaking researchers;) —rather than replace human scientific judgment. A high-value opportunity lies in using LLMs to deliver constructive feedback ( 11 ; 19 ) . This is especially consequential for authors who lack access to timely, expert feedback ( 14 ) . However, generating such feedback for scientific papers is non-trivial: it demands deep domain knowledge, which is challenging even for human experts ( 1 ) and unsurprisingly difficult for LLMs as well ( 13 ; 20 ) . Existing LLM-generated reviews reflect this difficulty, exhibiting weak specificity, limited actionability ( 13 ) , and poor alignment with human reviewers in feedback prioritization ( 20 ) . Addressing these gaps requires confronting two fundamental challenges: (1) rigorously defining what constitutes constructive, effective feedback for improving scientific research, and (2) building training and evaluation frameworks that operationalize this definition. In this work, we tackle both through an author-centric lens, leveraging author responses in author–reviewer discussion as a natural signal to define, train, and evaluate feedback quality. To address the first challenge, we introduce two author-centric axes of feedback quality: validity and author action (§ 3.1 ). As shown in Figure 1 , validity captures whether the authors agree with a raised concern, as invalid feedback tends to be rebutted. Author action captures whether authors propose concrete follow-up actions in response, noting that feedback can be valid yet not immediately actionable, still prompting meaningful discussion (e.g., suggestions deferred to future work). Building on this definition, we operationalize constructiveness for both training and evaluation. We curate GoodPoint-ICLR , which contains initial manuscript submissions and author–reviewer discussions for 19,534 ICLR papers (2020–2026), with each reviewer comment labeled for validity and author action (§ 3.2 ). We then introduce GoodPoint , a training recipe for constructive review generation. We first fine-tune Qwen3-8B on successful (i.e., valid and actionable) human reviews and then further align it via DPO on paired feedback data derived from both success labels and synthetic corruptions targeting key quality dimensions (§ 3 ). To evaluate LLM-generated feedback, we develop two complementary automatic evaluation frameworks—author response prediction and human consensus-based feedback evaluation—grounding on our definition of constructive feedback (§ 4 ). When evaluated on a held-out test set of 1,198 papers, GoodPoint -trained Qwen3-8B achieves 83.7% improvement in predicted success rate compared to the base model. In feedback matching against multi-reviewer consensus feedback, GoodPoint -SFT improves F1 by 58.8% over the base model and even exceeds Gemini-3-flash and GPT-5.2 in precision, suggesting that our model produces more selective critiques despite its substantially smaller scale. In an expert human study, GoodPoint -DPO outperforms Qwen3-8B across validity, actionability, specificity, and helpfulness, meaningfully reducing the gap to Gemini-3-flash. This result demonstrates that our automated evaluation results translate into practical usefulness perceived by real-world authors. Together, these findings highlight the promise of training and evaluating LLMs grounded in an author-centric definition of constructive feedback, and underscore the potential of LLMs to augment—rather than replace—human researchers. Figure 1: Overview of the two feedback quality axes and our definition of constructive feedback . Reviewer feedback is evaluated along two dimensions: validity—whether the raised concern is agreed upon by the authors—and actionability—whether it prompts concrete follow-up actions. We consider the feedback that is both valid and actionable as being successful, and build training and evaluation frameworks that operationalize this definition. 2 Related Works LLMs are increasingly integrated into scientific workflows ( 4 ; 8 ; 18 ) , yet fully autonomous systems risks distorting scientific judgment ( 2 ; 21 ) . Instead, we advocate for human-AI collaboration. Although automated peer review has emerged to alleviate the burden of reviewers ( 25 ; 13 ) , current LLM-generated reviews often lack critical depth and tend to over-positivity ( 25 ; 9 ) . Existing approaches rely on specialized fine-tuning ( 9 ) , review standardization ( 22 ) , or literature retrieval ( 26 ) . However, unlike these systems that prioritize mimicking reviewer behavior, GoodPoint focuses specifically on constructive feedback . By leveraging author responses as an explicit training signal, we prioritize feedback that is both valid and actionable from the researcher’s perspective. See Appendix A for an extended discussion. 3 GoodPoint : Constructive Feedback Learning Framework We introduce GoodPoint , a framework for generating author-centric paper feedback. Our approach builds on a dataset of ICLR 2020–2025 review discussions, from which we extract successful feedback items — those that authors both acknowledged as valid and committed to act upon — as the core training signal. We first fine-tune an LLM on this curated set via SFT to ground the model in constructive feedback patterns. We then sharpen both content and style through two DPO preference sets: one contrasting valid versus invalid feedback, and another using targeted corruptions along five dimensions — specificity, clarity, accuracy, prioritization, and supportive tone — as rejected samples. 3.1 GoodPoint Definition To operationalize our goal of generating constructive feedback, we use author responses to define a measurable signal of feedback success. Concretely, given a paper x and a set of self-contained feedback items F=\{f_{1},f_{2},\ldots,f_{n}\} with corresponding author responses R=\{r_{1},r_{2},\ldots,r_{n}\} , we define a feedback item f_{i} as successful if the author acknowledges it as both valid and actionable (examples in Figure 1 ). Formally, we define an indicator function: \mathbb{1}_{\text{success}}(f_{i},r_{i})=\begin{cases}1&\text{if }\mathcal{V}(f_{i},r_{i})=1\text{ and }\mathcal{A}(f_{i},r_{i})=1\\ 0&\text{otherwise}\end{cases} (1)

Read the complete text