Surviving the paper deluge: Notes from an ICRA panel on publishing, LLMs, and the future of peer review

A recent ICRA panel titled “Surviving the Paper Deluge” brought together leading robotics researchers who have grappled with the overwhelming number of robotics papers being published today .

The discussion ranged from hard numbers on publication growth, through the promises and risks of large language models (LLMs), to radical proposals for reshaping peer review as we know it.

Panel chair Aude Billard pointed to rapid growth across major IEEE Robotics and Automation Society venues, with a roughly exponential curve beginning around 2017. An estimate for 2025 suggests around 70,000 papers containing the word “robotics.”

Billard noted that while in some fields, extreme specialization may be an acceptable survival strategy, robotics is inherently different. A robot is an integration of perception, control, manipulation, learning, hardware, interaction, safety, and deployment. If researchers can only stay current within narrow silos, the field risks losing one of its core strengths: the ability to connect ideas across domains.

Kunpeng Yao presented a case study funded by IEEE RAS Science and Technology Watch Board . The team tried to do something that borders on heroic in today’s publishing environment; that is, reading an entire subfield carefully over a single year.

Focusing on papers related to learning from demonstration, the team searched IEEE Xplore for relevant 2024 papers, screened the results manually, and identified 347 relevant papers. Of those, only 69, or about 20 percent, were judged to have made notable contributions.

According to the criteria used, ‘notable’ papers tended to offer new formulations, mechanisms, or theoretical guarantees; serious comparisons against strong state of the art; clear gains in robustness, transfer, or failure recovery; new teaching or data collection modalities; and convincing real-robot validation. Papers were less compelling when they simply relabelled existing approaches or failed to demonstrate their claims.

Yao summarized one lesson neatly: notability should be judged against the state of the art, not against “the newest vocabulary.”

That distinction is of particular importance in a field where new labels can travel quickly and a paper may sound fresh without actually moving the frontier. Conversely, useful work is sometimes hidden in less fashionable venues or written by less visible groups.

Degrees of hallucination

One question is whether large language models can help with literature review. Nadia Figueroa explained that LLMs can improve the mechanics of literature review. That is, they can help with search, retrieval, clustering, tables, summaries, and rough conceptual maps. Work that once took weeks or months can now sometimes be done in hours. For a new PhD student entering a field, that is a genuine reduction in the barrier to entry.

But Figueroa also identified hallucination as a major issue. First-order hallucination involves fake or incorrect references. More subtle is second-order hallucination, where the reference is real but the model misstates what the paper actually did. Also dangerous is third-order hallucination, where the LLM invents plausible but false commonalities across papers.

“The danger currently is not fake citations,” observed Figueroa. “It’s fake understanding.”

An LLM-based literature review can contain real papers and still misrepresent the field. It can be fluent, structured, and wrong. If researchers outsource not just the mechanics of reading but the cognitive act of comparison, judgment, and doubt, the field may produce more text while developing less understanding.

Salami and sandcastles

Meanwhile, Greg Dudek described the familiar problem of “salami slicing,” where a larger body of work is divided into “the thinnest possible slices” and spread across workshops, conferences, and journals.

For Dudek, the system rewards this behaviour. Students need papers to graduate. Early-career researchers need papers for jobs. Committees often face too much material to read deeply, so titles, venues, counts, and indices become tempting shortcuts.

But the result is costly for readers. Multiple papers repeat the same background, divide one contribution into fragments, and make it harder to reconstruct the full story.

His preferred remedy is simple in principle but difficult in practice: publish fewer, more integrated papers.

Dudek is under no illusion about how hard this will be. “There is no fix,” he said, if by fix we mean a return to a quieter world. Incremental reforms may help, but the flood is still rising. He compared small procedural fixes to building better walls around a sandcastle while “there’s a tsunami coming from the back of the room.”

Re-designing peer review?

Renaud Detry argued that the growth in publications partly reflects the growing presence of applied systems work in academic venues. That is not necessarily bad. Robotics advances through real systems, data, benchmarks, platforms, and the last-mile effort needed to make ideas work outside idealized settings.

The problem is that the same evaluation machinery often judges very different contribution types. A new algorithm, a carefully engineered system, a benchmark, a dataset, and an industrially relevant validation study should not all have to pretend to be the same kind of paper. Robotics may need clearer tracks for fundamental science, applications, infrastructure, benchmarks, and technical correctness.

Visibility is not value

Dongheui Lee addressed another vital part of the new publication ecosystem: arXiv, open-source releases, project pages, videos, blogs, and social media. These tools can broaden access, improve reproducibility, and help readers decide what to examine closely. But they also distort attention. Visibility can reflect institutional prestige, networks, speed, and self-promotion as much as scientific value.

Lee’s suggestion was pragmatic: keep expert peer review as a quality filter but use its signals better.…

News is gathered automatically from public robotics & AI feeds on a schedule.