Ai2 study suggests LLM judges of research novelty are unreliable
Published
Ai2 has released a paper, benchmark, data and code that test how consistently large language models rate the originality of scientific research ideas.
Ai2 has released Old Ideas, Novel Problems, a study of how reliably LLM-based novelty judges rate research ideas, announced in a Hugging Face blog post. It comes with a benchmark, data, code and a pipeline for building new test sets.
Human-written ideas were picked using ICLR 2026 reviewers' remarks on originality; one test set then paired them with ideas from a simple AI generator. Across six GPT and Claude judges, small setup changes swung verdicts, and a simpler prompt cut GPT-5.4's accuracy on these pairs from 93.5% to 40.9%, below chance.
On that set, a basic Claude Opus 4.6 prompt reached 0.854 macro-F1 for $2.59, outscoring two purpose-built evaluators: Idea Novelty Checker (0.642, $30.66) and Sakana AI's AI Scientist evaluator (0.772, $82.05).
Why it matters
Per the authors, the problem remains open: extra search or compute does not guarantee steadier verdicts, and a judge's results on human-written work may not predict its behavior on AI-generated proposals.
Claims & sources
- c4
Ai2 released the study Old Ideas, Novel Problems along with a benchmark, data and code, plus an automated pipeline for assembling fresh evaluation sets.
- c11
The team built an automated pipeline that combed ICLR 2026 peer reviews for passages in which reviewers explicitly addressed originality.
- c14
The Human+Generated set pairs the 154 validated human ideas with 154 ideas from a basic LLM generator; the authors treated the generated ideas as less novel, calling these weak labels.
- c8
The authors report that, for six LLM judges, minor tweaks to how the evaluation was set up led to sharply different verdicts on identical ideas.
- c19
When GPT-5.4 was given a simpler prompt, its accuracy on Human+Generated pairs fell from 93.5% to 40.9%, which is below chance.
- c25
A simple Claude Opus 4.6 prompt achieved 0.854 macro-F1 for $2.59, compared with 0.642 for $30.66 from Idea Novelty Checker and 0.772 for $82.05 from Sakana AI's AI Scientist evaluator.
- c28
The authors call LLM-based novelty assessment an unresolved problem: how a judge does on human-written research may not predict its handling of AI-generated ideas, and extra search or compute offers no guarantee of consistency.
Original sources
About this AI summary
AI-generated original factual summary by AI Resum, based on the linked source.
No review, permission, license or endorsement by the source publisher is claimed. Source content and trademarks remain with their respective owners.
Swipe up for next1 / 1