Let Papers Flow: AI Conferences Should Embrace Submission Explosion via Autonomous Review Pipelines

1. Introduction

AI research has entered a regime of accelerated supply. Tools that assist with literature search, coding, experimentation, drafting, and even parts of scientific discovery are compressing research cycles and increasing the rate at which plausible submissions can be produced [1,2,3,4]. The institutional bottleneck is therefore shifting. The main scarcity is no longer how many PDFs the community can produce; it is how much trusted human attention the field can still devote to careful evaluation, calibration, appeals, and reading.

In Herbert Simon’s classic formulation, an information-rich world consumes attention [5]. At flagship AI-conference scale, that scarcity is now visible across the entire stack. ICLR 2025 reported 11,603 submissions and 18,325 reviewers, with 99.98% of papers receiving at least three reviews [6]. NeurIPS 2025 reported 21,575 valid main-track submissions handled by 20,518 reviewers, 1,663 ACs, and 199 SACs, and its program chairs explicitly described a process that is getting noisier and harder to calibrate at scale [7,8]. The scarce resource is not just reviewer time. It is also AC and SAC time for synthesis, PC time for governance, and reader time for deciding what the field will actually absorb. As Figure 1 later shows, the 2025 reviewer-and-chair stacks already run into the tens of thousands of named committee roles once these layers are summed, so the question is no longer only whether one more cycle can be staffed, but whether perpetual committee enlargement is itself a sustainable review architecture.

Venues are already experimenting with machine assistance. ICLR 2025 deployed a reviewer-feedback agent [9,10,11] and piloted a TMLR partnership [12]; ICML 2026 piloted Google’s Paper Assistant Tool (PAT) on the author side [13,14]; and AAAI-26 launched an official AI-assisted peer-review pilot in which the AI review gives no score or recommendation and all decisions remain human [15]. At the same time, broader norms are unstable. Nature has documented both growing unease around AI-mediated review and substantial undisclosed or policy-violating use, while prompt-injection attacks and illicit AI review have already become concrete governance problems [16,17,18,19].

We therefore use autonomous review pipeline to mean a machine-run first-pass control plane that produces structured evidence and routing signals, not a sovereign AI judge. The point is not to add one more reviewer bot. It is to let conferences officially adopt and govern machine-first review infrastructure so that routine evidentiary work scales with compute, human depth is reserved for escalation and certification, and the research loop speeds up rather than stalling at review. Table 1 defines the paper’s core vocabulary; Figure 1 sketches the architecture.

Position. To survive accelerated research, AI conferences must adopt autonomous review pipelines as their first-pass control plane. By scaling routine evidentiary work with compute, this machine-first spine reserves scarce human attention for high-stakes escalations. Ultimately, it decouples continuous technical certification from selective curation, accelerating scientific iteration while preserving a sustainable review commons.

2. Motivation: Suppressing Volume Is the Wrong Response

The submission explosion is the fundamental premise that forces a redesign of conference architecture. The usual instinct in a review crisis is to press down on this supply: stricter desk rejection, per-author caps, anti-overlap rules, or a thicker social norm that authors should “submit less.” Some abuse control is necessary, but volume suppression misdiagnoses the bottleneck. The real problem is that a human-first seasonal process still tries to do too many jobs at once: admissibility checks, novelty reconstruction, literature positioning, artifact inspection, conflict resolution, and visibility allocation.

We unpack this dynamic across three dimensions: Section 2.1 details how major venues already show this curve, Section 2.2 explains how abundance improves the scientific record, and Section 2.3 argues fairness must attach to due process rather than equal labor.

2.1. The Big-Three Venues Already Show the Curve

Figure 1 separates three trends: rising submission volume, the conservative floor for reviewer demand, and the escalating scale of annual committee recruitment. The top panel shows sustained, structural growth across venues, with ICLR growing fastest relatively and NeurIPS remaining largest absolutely.

The lower-left panel establishes a conservative lower bound for reviewer workload ($3N/R$), assuming a standard minimum of three reviews per paper before accounting for emergency backfills or second-round checks [6,20,21,22]. Maintaining this baseline requires continuous, massive reviewer expansion, which directly incurs quality-control costs. For instance, ICLR 2022 instituted a mentorship pipeline to promote inexperienced reviewers [23], and NeurIPS 2025 explicitly linked scale-driven recruitment to increased calibration noise [7]. Expanding the pool is therefore a quality-management challenge, not just a headcount problem.

The lower-right panel exposes the core sustainability threat: the current human-first system stabilizes only through perpetual committee enlargement. By 2025, the combined sum of named committee roles (reviewers, handling chairs, senior synthesis chairs, and PCs) reached roughly 19.2k at ICLR, 12.3k at ICML, and 22.4k at NeurIPS (Table A6). The institutional warning is clear: as AI lowers the cost of submissions, relying on annual, massive recruitment drives is an unsustainable control plane. While reviewer pools scale elastically, the synthesis and governance layers remain critically thin and hard to replenish (detailed historically in Appendix B, Table A4–Table A10 and Figure A1–Figure A2).

2.2. Abundance Can Improve the Scientific Record

Restriction-first responses also miss something scientifically valuable: abundance is not only a burden on review; it is evidence that the field has become more experimentally expressive. As AI tools accelerate research and lower the cost of drafting, more of the real search process can be externalized: replications, narrow but useful observations, artifact fixes, negative results, and domain-specific variants that previously lost out to a lab’s scarce “best shot.” Publication bias against negative or null results is already well documented [24,25]. A review architecture that can absorb more work can therefore make the record both broader and faster to correct.

Reader attention is the second bottleneck. Even if a venue could somehow review every paper carefully, the community would still need to decide what to read, cite, reproduce, and build on. Accepted-paper lists merely move overload downstream. This is why curation and interfaces belong inside the review architecture. Machine-first review can emit claim cards, cluster pages, artifact status, and contrastive summaries that help readers navigate abundance rather than drown in it.

2.3. Fairness Should Attach to Due Process, Not Equal Labor per PDF

At scale, fairness cannot mean giving every PDF the same quantity of human labor. That norm becomes gamable once submission cost falls and papers become highly correlated. A fairer principle is that every submission receives the same transparent rules, the same structured first pass, and the same accessible escalation path, while scarce human depth goes where uncertainty, disagreement, risk, or potential impact are greatest.

The current system already shows the strain. NeurIPS 2025 emphasized rising noise and calibration difficulty, explicitly linking larger-scale recruitment to a greater chance of less experienced reviewers and ACs [7]. ICLR 2025 launched a reviewer-feedback agent because review quality was harder to sustain under rapidly growing volume [9,10]. Review policies have become more operationally interventionist as well: ICLR warns that low-quality placeholder reviews can trigger desk rejection of the reviewers’ own papers, while ICML and ICLR still discuss emergency-review backfills when the three-review floor is not met [20,21,26]. The NeurIPS 2026 Position Track blog post also reported that roughly 35% of reviewers in the previous year’s position track were unresponsive, forcing emergency intervention and delays [27,28,29,30]. These are not marginal frictions. They are signs that the system is already rationing trusted attention and spending growing effort just to keep the human-first pipeline functional.

3. The Core Thesis: Autonomous Review Pipelines Should Be the Control Plane

To survive submission abundance, AI conferences must formally adopt autonomous review pipelines as their default control plane. By control plane, we mean a machine-run, venue-governed layer that ingests submissions, constructs structured evidence, triggers interventions, and routes scarce human attention [31]—not a sovereign AI judge.

This scalable first pass should systematically answer bounded questions before humans spend depth: Is the submission policy-compliant? What are its central claims and missing baselines? Are artifacts executable? Does the work belong to a larger portfolio of highly correlated submissions? Figure 1 sketches six core functions handling these checks. By universally applying these baseline diagnostics, the minimum floor of evidentiary depth rises, allowing human reviewers to enter the process with a richer dossier and concentrate exclusively on high-stakes escalations (e.g., difficult claims, disputed novelty, policy violations).

This architecture is multi-sided. Authors receive pre-submission diagnostics (missing controls, clarity checks); venues gain robust routing and cluster-detection tools; and readers inherit durable discovery assets like claim cards and contrastive summaries [10,32,33]. Crucially, this pipeline aligns with the accelerated tempo of AI research [2]. Treating incoming papers not as isolated PDFs, but as potentially correlated portfolios or rapid experimental iterations, allows the venue to group related submissions, eliminate duplicated manual reconstruction, and dramatically shorten the latency between idea, evidence, and critical feedback.

To replace opaque scoring with actionable evidence, the concrete output of this first pass must be a structured pipeline report exposing the same underlying data to authors, reviewers, and readers. At minimum, it should contain: (1) Policy summary (anonymity, formatting, disclosures); (2) Claim cards (extracted core contributions); (3) Literature positioning (retrieved baselines); (4) Artifact status (code execution results); (5) Cluster context (grouping metadata for parallel work); (6) Uncertainty flags (machine-generated risk notes); and (7) Escalation recommendation (explicit routing triggers). These structured objects make feedback legible enough for authors to repair issues while the work is still live [34].

To ground this architectural claim, Table 2 traces a hypothetical submission through this exact pipeline. Rather than a static “accept” or “reject” verdict, the machine handles routine validation and evidence structuring, producing an auditable report that intentionally routes the paper to a human expert only when novelty bounds or evaluation robustness become uncertain. An extended operational sketch of this pipeline is provided in Appendix C.

Ultimately, fairness in this regime attaches to uniform due process rather than equal manual labor per PDF. Every submission receives the same transparent rules and structured first pass, while human authority becomes more meaningful precisely because it is no longer diluted across routine screening.

4. Strict Standards Can Coexist with Faster Research Cycles

The primary objection to this architectural shift is that exploding submissions will inevitably degrade accepted quality. We argue no: if pipelines operate as certification stacks rather than synthetic reviewers, submission abundance widens intake without relaxing the evidentiary bar.

First, machine-first review raises both the floor and ceiling of rigor. It guarantees uniform baseline checks (anonymity, policy compliance, artifacts) that currently rely on reviewer goodwill, while redirecting expert human attention exclusively to high-impact, disputed, or uncertain edge cases (Appendix Table A2).

Second, a pipeline-first venue makes standards highly auditable. ML peer review is notoriously noisy and difficult to calibrate [35,36,37,38,39]. A governed pipeline mitigates this opacity by transparently logging module triggers, escalation flags, and human overrides.

Third, faster review cycles are essential for scientific health. Slower evaluation loops become increasingly costly when AI compresses the pace of proposing and executing research [1,2,40]. By accelerating the first pass, venues tighten the feedback loop, ensuring earlier error correction and better overall throughput [41].

Finally, structured first passes enable a crucial semantic shift toward typed certification. Rather than hiding judgment behind an opaque scalar verdict, venues can issue multidimensional certificates detailing policy compliance, artifact reproducibility, and claim support. This nuanced record is far more honest and actionable, successfully isolating technically reliable execution from empirically uncertain impact [33,36,38,42].

5. Human Roles, Governance, and the Economics of Attention

Autonomous review pipelines redeploy rather than eliminate human judgment: reviewers become escalation specialists, ACs/SACs focus on synthesis and calibration, PCs handle governance, and readers gain structured summaries (Appendix Table A3). However, explicit governance is critical to prevent correlated bias, where monocultural models might narrow collective scientific focus or over-favor standard paradigms [3]. Venues must ensure pluralistic backends, audit logs, and escalation policies designed to protect novel or out-of-paradigm work.

Simultaneously, the submission artifact must evolve beyond the transitional PDF wrapper [43]. Future submissions should integrate structured claim and executable artifact layers, enabling venues to directly verify dependencies and replication hooks, akin to a continuous-integration system for science [33,44].

Crucially, this redesign addresses the broken social contract of review. AI-accelerated research weakens or further erodes the historic symmetry between the cost of writing and reviewing a paper, deeply straining current incentive structures [45,46,47]. By reframing the economic tradeoff from “paid compute versus free volunteer labor” to “paid compute versus scarce human attention,” a machine control plane acts as an essential shock absorber. It offloads routine verification to compute, preserving expensive human depth for edge cases where it is actually needed [34]. Just as importantly, it offers a path away from a conference ecosystem that must re-recruit an ever larger human committee every cycle merely to keep the seasonal workflow intact. We formalize this dynamic in Section 5.1 and address failure modes in Section 5.2.

5.1. A Falsifiable Model of Institutional Attention

To transition from vision to testable institutional theory, we formalize the allocation of scarce attention. Let N represent the total submissions; $\omega$ the shadow price of one hour of trusted human reviewer time; $h_0$ the average human hours required for a traditional manual review; and $c_m$ the amortized computational cost per paper for the machine-first pass. In a pipeline architecture, the system escalates a fraction $p_e\in (0,1)$ of papers to human experts, requiring $h_e$ hours per escalated paper (where $h_e>h_0$ due to concentrated effort on difficult claims). The system incurs penalties for false clearances (rate ${\lambda }_{FC}$, cost $K_{FC}$) and false escalations (rate ${\lambda }_{FE}$, cost $K_{FE}$). Figure 1’s reviewer panel provides a conservative empirical proxy for the first-round human term: once a venue guarantees at least three reviews, reviewer demand cannot fall below $3N/R$ reviews per reviewer, before any emergency backfill or discussion-phase follow-up [6,20,22]. The figure’s committee-recruitment panel then adds the parallel institutional signal: even when the three-review floor is met, the human-first system does so by repeatedly enlarging the named committee stack.

Under traditional human-first review, expected attention cost is simply $C_{\mathrm{human}}=N{h}_0\omega$. A pipeline-first architecture, handling the first pass computationally and routing exceptions to humans, incurs a total expected cost of:

$$C_{\mathrm{pipe}}=N\left(c_m+p_e{h}_e\omega +{\lambda }_{FC}{K}_{FC}+{\lambda }_{FE}{K}_{FE}\right)$$

(1)

The pipeline dominates ($C_{\mathrm{pipe}}<C_{\mathrm{human}}$) when the required escalation rate satisfies:

$$p_e<{\displaystyle \frac{h_0}{h_e}}-{\displaystyle \frac{c_m+{\lambda }_{FC}{K}_{FC}+{\lambda }_{FE}{K}_{FE}}{h_e\omega }}$$

(2)

This inequality clearly specifies failure modes: excessive noise (${\lambda }_{FE}$), inability to clear routine work ($p_e$), or compute costs ($c_m$) eclipsing the value of human time. Crucially, as the shadow price of human attention spikes in the era of accelerated research ($\omega \to \infty$), the dominance condition approaches the asymptotic limit $p_e<h_0/h_e$. Provided the machine can safely handle enough routine work to offset the deeper human depth ($h_e$) spent on edge cases, conferences can sustain rigorous evaluation without exhausting the reviewer commons.

5.2. Failure Modes and Safeguards

Because autonomous review pipelines act as structural gatekeepers, they create new systemic risks that require explicit governance and safeguards:

Automation bias. Reviewers or chairs may over-trust fluent machine outputs. The safeguard is to make uncertainty legible, require explicit human sign-off on escalated decisions, and audit whether humans are simply copying automated recommendations.

Gaming and adversarial optimization. Authors may learn to target the surface cues of the control plane. Venues should therefore rotate checks, combine independent modules, maintain adversarial evaluation sets, and reserve sanctions for deliberate manipulation.

Cluster errors. Portfolio clustering can over-merge genuinely distinct work or miss related thin slices. Cluster detections should trigger human review or author clarification, not irreversible penalties.

Reader-interface distortions. Once machine-generated summaries shape discovery, poor summaries can bias reader attention just as badly as poor review scores. Reader-facing interfaces should be evaluated directly as part of the review system, not as a downstream cosmetic feature.

Unequal access to tooling. If some authors can privately use strong paper assistants while others cannot, hidden adoption amplifies inequality rather than reducing communal workload fairly. A pipeline architecture addresses this by making bounded diagnostic tools venue-provided and broadly accessible before submission [13].

6. Downstream Implication: From Seasonal Batches to Continuous Review

A pipeline-first control plane naturally weakens the logic of seasonal batch review, shifting the optimal architecture toward continuous intake, rolling certification, and periodic curation. Deadlines remain useful for synchronized visibility, but lose their status as the sole gateway to the technical record. Existing precedents—such as TMLR [42,48], ACL Rolling Review [49], the Journal-to-Conference track [50], and NeurIPS’s AC pilot [51], as well as distinctions between main tracks and dataset tracks [52,53]—already demonstrate the viability of decoupling baseline certification from seasonal curation.

A continuous regime aligns better with accelerated research by drastically reducing error-correction latency. Authors receive structured first-pass diagnostics early, enabling them to address missing controls or broken artifacts while code and experimental contexts remain fresh. This tightens the claim-criticism-repair loop without lowering standards [1,2,40]. Low-risk work seamlessly enters a searchable certified stream, while contested cases wait for deeper human review. Consequently, conference deadlines pivot to govern strictly curation (allocating scarce talks, posters, or awards) rather than existential admissibility.

This shift profoundly upgrades both the reader and reviewer experience. For readers navigating an accelerating literature, flat proceedings are insufficient; venues must provide claim cards, cluster pages, and typed certification states to direct attention intentionally. For reviewers, the burden shifts from synchronized seasonal bursts of thousands of volunteers to a smaller, continuously active pool of escalation experts, making their service more legible, intellectually valuable, and easier to sustain than annual committee inflation.

Ultimately, if agentic workflows generate drafts and ablations on a near-daily cadence, seasonal review spikes become structurally misaligned with the tempo of knowledge production [1,2,3]. Conferences face a stark choice: enforce harsh supply suppression, or build a governed shock absorber. A continuous autonomous review pipeline offers the sustainable path forward, absorbing research velocity while keeping final legitimacy, appeals, and accountability strictly human-governed [40].

7. A Staged Path to Venue Adoption

No major venue should jump to a fully automated control plane overnight. Moving toward a pipeline-first architecture requires bounded, auditable steps judged by institutional metrics—turnaround, reviewer burden, escalation quality, and reader utility—rather than mere accept/reject classification accuracy. The objective is to build infrastructure that matches the tempo of accelerated research while preserving human accountability.

Stage 1: Officialization of bounded assistance. Venues must replace hidden private usage with transparent, task-level policies specifying allowed AI use, required disclosures, and human-only actions. As recent pilots and violations show, formalizing access equalizes the playing field, preventing well-resourced labs from quietly leveraging superior private tools [15,26,54,55,56,57].

Stage 2: Author- and intake-side automation. The lowest-risk starting point is deploying PAT-style diagnostics, format checks, overlap screening, and basic claim extraction before human assignment [13,14,58]. This catches avoidable issues while code and experimental context are fresh, responsibly accelerating research by shifting criticism earlier in the loop [1,2].

Stage 3: Venue-side routing and portfolio escalation. Once first-pass reports are reliable, the infrastructure must integrate directly into conference workflows (e.g., OpenReview) [33,44,59,60,61]. Rather than asking models for free-form reviews, chairs use the control plane for cluster detection, uncertainty surfacing, and risk-based escalation, treating highly correlated submissions as joint portfolios rather than isolated PDFs.

Stage 4: Decoupling certification from curation. Building on existing rolling pathways [42,48,49,50,51], venues can finally separate continuous intake from seasonal visibility. Routine, sound work enters a searchable certified stream rapidly, while conferences reserve scarce talk slots, posters, and synthesis sessions strictly for high-attention curation.

Staging lowers institutional risk without ignoring the accelerating supply of papers. Conferences do not need to wait for a flawless synthetic judge; they only need governed infrastructure that speeds up routine evidence construction and sharper escalation, keeping final legitimacy and appeals safely in human hands [3,34,40]. Appendix G summarizes how existing pilots inform this path.

8. Alternative Views

The proposal to adopt autonomous review pipelines as a central control plane naturally invites comparisons to several alternative reform paradigms currently debated within the community.

• View 1: The labor deficit model.

This view treats the crisis as a mere staffing shortage, solvable by aggressively recruiting larger reviewer pools and improving incentives. Rebuttal: While better incentives help and Figure 1 shows recruitment can buffer the first-pass layer, this relies on repeated large-scale expansion, emergency backfilling, and continuous influxes of less experienced reviewers. The deeper failure is sustainability. A workflow demanding perpetual committee enlargement is not a stable control plane for machine-accelerated paper supply. The true bottleneck is not raw headcount, but the highly inelastic supply of trusted attention at the synthesis and governance layers needed to evaluate evidence, calibrate noise, and defend decisions.

• View 2: Supply suppression.

This perspective aims to protect review quality by artificially making submissions scarcer—enforcing strict per-author caps, aggressive desk-rejection quotas, or penalizing narrow extensions. Rebuttal: Artificial scarcity sacrifices the scientific benefits of accelerated research. Aggressive filtering disproportionately targets the exact artifacts an accelerated field needs to solidify its epistemic foundation: replications, negative results, specialized findings, and exploratory variants. We should build infrastructure that absorbs abundance, rather than punishing researchers for increased productivity.

• View 3: The egalitarian fallacy.

On this view, fairness dictates that every submitted PDF must receive roughly the same amount of human manual labor and depth. Rebuttal: In a high-volume regime, this baseline is easily gamed; it allows well-resourced labs to appropriate communal attention by flooding the queue with highly correlated variants. True equity lies in uniform due process—applying the exact same automated evidentiary standards and policy checks to all submissions—while reserving scarce human depth for papers that exhibit genuine uncertainty, novelty, or impact.

• View 4: The sovereign AI assessor.

The fully automated endpoint argues for transitioning directly to LLM judges that synthesize reviews and issue final accept/reject decisions. Rebuttal: This collapses helpful assistance into unaccountable sovereignty. Our architecture advocates for a control plane, not a synthetic judge. We must bound the machine’s authority to evidence construction, literature positioning, and routing, keeping legitimacy, policy calibration, and final accountability strictly in human hands.

• View 5: Fused certification and curation.

This view insists that maintaining the prestige of the field requires conference acceptance to simultaneously mean technical validation and high-visibility presentation. Rebuttal: This conflates establishing the scientific record with allocating community attention. Rolling venues (e.g., TMLR, ARR) already demonstrate that these functions can be decoupled. Once reader attention is recognized as the ultimate scarcity, conferences should certify broadly for technical soundness, but curate narrowly for visibility.

• View 6: Post-conference publication.

This argument treats the centralized conference model itself as obsolete, advocating a shift entirely to decentralized preprints (arXiv) and overlay journals. Rebuttal: While decentralized models remove the submission bottleneck, they drastically exacerbate the discovery bottleneck. Conferences remain vital as curators of scarce human attention and synthesizers of field-level agendas. They simply should not remain the sole, choked gateway through which all technically valid work must pass.

9. Research Agenda

The position outlined in this paper is normative, but it demands a concrete, falsifiable empirical agenda. Appendix H supports this agenda by mapping out relevant literature (Appendix H.1), systems (Appendix H.2), and implementation hooks (Appendix H.3). To operationalize the autonomous review pipeline, the community must address challenges across three distinct tracks:

• 1. Granular evaluation frameworks.

Modular benchmarks: We must move beyond monolithic “AI reviewer” datasets that optimize for matching a human’s final accept/reject score. The field needs specialized benchmarks for bounded evidentiary tasks: claim extraction fidelity, literature retrieval relevance, artifact execution success, and accurate cluster detection. Evaluating typed certification: Researchers should test whether multidimensional, typed certificates (e.g., separating policy compliance, empirical reproducibility, and theoretical novelty) provide better downstream utility to readers and meta-researchers than opaque scalar scores.

• 2. Institutional policy and governance.

Calibrated escalation policies: Formal evaluations must compare deterministic risk thresholds, learned routing policies, and human-in-the-loop triage to safely route contested, anomalous, or high-impact claims. Governance of attention: As human roles shift from routine screeners to escalation and certification specialists, empirical work must design new incentive structures, audit automation bias, and evaluate formalized appeal channels.

• 3. Systems and deployment.

Epistemic impact of reader interfaces: If conferences shift toward curation, the user interface becomes an epistemic tool. We must measure how AI-generated claim cards, cluster pages, and contrastive summaries alter citation spread, code adoption, and reading behavior compared to traditional flat proceedings. Live operational pilots: The decisive experiments must be in production. Venues should launch bounded, continuous-review pilots. These pilots must publicly track turnaround latency, queue lengths by risk tier, false-escalation rates, human workload relief, and the size and composition of the human committee they still require in order to move this architecture from theory to practice.

10. Conclusions

The submission explosion driven by AI-accelerated research is not a pathology to be suppressed, but a structural shift demanding a scalable response. By embracing autonomous review pipelines as a machine-first, human-governed control plane, conferences can systematically absorb this abundance without compromising rigor. This architecture accelerates critical feedback and automates routine evidentiary checks, ultimately redirecting scarce human attention away from basic screening and toward high-stakes escalation, continuous certification, and meaningful scientific curation. Just as importantly, it offers a path toward a sustainable review commons that does not depend on perpetual committee enlargement to keep seasonal review alive.

Appendix H.1. Representative Research Literature

Table A16. Representative literature relevant to autonomous review pipelines. The papers differ in what they automate; together they illustrate a design space broader than “LLM writes the final review.”

Work	Primary contribution	Why it matters for this paper
SEA [61]	Generates review-style feedback with standardization, evaluation, and analysis modules.	Shows early end-to-end automation of paper feedback, while stopping short of venue legitimacy.
Quality-checker framing [58]	Recasts LLMs as manuscript quality checkers rather than substitute reviewers.	Closely matches our argument that machines should gather evidence and flag problems.
Live review-feedback study [108]	Studies the ICLR 2025 reviewer-feedback deployment in a real review process.	Provides direct evidence that AI can assist review while creating governance tensions.
ReViewGraph [44]	Models automated paper reviewing through structured reviewer–author debate graphs.	Suggests richer representations for disagreement and escalation.
FactReview [33]	Combines claim extraction, literature positioning, and execution-based claim verification.	Exemplifies the evidence-grounded, claim-level style of autonomous review pipeline advocated here.
LLM-ASPR survey [109]	Synthesizes tasks, datasets, systems, and failure modes for automated scholarly paper review.	Helps position this paper’s argument in the broader technical and governance landscape.

Table A16. Representative literature relevant to autonomous review pipelines. The papers differ in what they automate; together they illustrate a design space broader than “LLM writes the final review.”

Work	Primary contribution	Why it matters for this paper
SEA [61]	Generates review-style feedback with standardization, evaluation, and analysis modules.	Shows early end-to-end automation of paper feedback, while stopping short of venue legitimacy.
Quality-checker framing [58]	Recasts LLMs as manuscript quality checkers rather than substitute reviewers.	Closely matches our argument that machines should gather evidence and flag problems.
Live review-feedback study [108]	Studies the ICLR 2025 reviewer-feedback deployment in a real review process.	Provides direct evidence that AI can assist review while creating governance tensions.
ReViewGraph [44]	Models automated paper reviewing through structured reviewer–author debate graphs.	Suggests richer representations for disagreement and escalation.
FactReview [33]	Combines claim extraction, literature positioning, and execution-based claim verification.	Exemplifies the evidence-grounded, claim-level style of autonomous review pipeline advocated here.
LLM-ASPR survey [109]	Synthesizes tasks, datasets, systems, and failure modes for automated scholarly paper review.	Helps position this paper’s argument in the broader technical and governance landscape.

Appendix H.2. Representative Systems, Platforms, and Repositories

Table A17. Selected systems and repositories that could form part of an autonomous review pipeline stack. This list is representative rather than exhaustive.

Category	Examples	Relevance
Venue workflow infrastructure	OpenReview platform and API [59,112,113]	Provides an existing open review platform, API layer, and venue workflow primitives for machine-mediated review.
Reviewer matching infrastructure	OpenReview expertise models [60]	Supplies paper–reviewer affinity tools that could feed escalation and expert routing.
Rolling or continuous review venues	TMLR, ACL Rolling Review, Journal-to-Conference [42,48,49,50]	Demonstrate that review, certification, and conference presentation can already be partially decoupled.
Document parsing and structuring	GROBID [114]	Converts scholarly PDFs into structured representations useful for claim extraction and policy checks.
Scholarly metadata and retrieval	OpenAlex and Semantic Scholar APIs [115,116]	Enable literature positioning, citation retrieval, cluster building, and reader-facing comparisons.
Venue-side AI assistance	ICLR review-feedback agent, ICML PAT [9,13,14]	Show that mainstream venues are already piloting machine assistance on both the reviewer side and the author side.

Table A17. Selected systems and repositories that could form part of an autonomous review pipeline stack. This list is representative rather than exhaustive.

Category	Examples	Relevance
Venue workflow infrastructure	OpenReview platform and API [59,112,113]	Provides an existing open review platform, API layer, and venue workflow primitives for machine-mediated review.
Reviewer matching infrastructure	OpenReview expertise models [60]	Supplies paper–reviewer affinity tools that could feed escalation and expert routing.
Rolling or continuous review venues	TMLR, ACL Rolling Review, Journal-to-Conference [42,48,49,50]	Demonstrate that review, certification, and conference presentation can already be partially decoupled.
Document parsing and structuring	GROBID [114]	Converts scholarly PDFs into structured representations useful for claim extraction and policy checks.
Scholarly metadata and retrieval	OpenAlex and Semantic Scholar APIs [115,116]	Enable literature positioning, citation retrieval, cluster building, and reader-facing comparisons.
Venue-side AI assistance	ICLR review-feedback agent, ICML PAT [9,13,14]	Show that mainstream venues are already piloting machine assistance on both the reviewer side and the author side.

Appendix H.3. Implementation Hooks and Repository Map

Table A18 makes the deployment story more concrete by mapping common autonomous review pipeline functions to reusable public infrastructure.

Table A18. Concrete repositories, APIs, and workflow components that can support an initial autonomous review pipeline deployment.

Function	Concrete resource	Typical use in an autonomous review pipeline pilot
Submission intake and state transitions	OpenReview platform plus API v2 and openreview-py [59,112,113]	Ingest submissions, store machine reports, attach decision metadata, and expose venue-specific workflow actions.
Reviewer or editor matching	OpenReview expertise models [60]	Compute affinity scores, seed escalation routing, and support specialist assignment once a paper leaves the automated queue.
PDF-to-structure conversion	GROBID [114]	Convert scholarly PDFs into structured sections, references, and metadata for claim extraction, overlap checks, and policy validation.
Literature retrieval and citation context	OpenAlex and Semantic Scholar APIs [115,116]	Build neighborhood graphs, retrieve likely comparisons, and generate reader-facing cluster or claim pages.
Venue-side author assistance	ICML PAT and reviewer-feedback agents [9,13,14]	Provide pre-submission diagnostics for authors and structured coaching for reviewers without delegating final legitimacy.
Rolling certification pathways	TMLR, ACL Rolling Review, and Journal-to-Conference [42,48,49,50]	Supply external certification channels that can feed conference curation after the machine-first control plane has done initial triage.

Table A18. Concrete repositories, APIs, and workflow components that can support an initial autonomous review pipeline deployment.

Function	Concrete resource	Typical use in an autonomous review pipeline pilot
Submission intake and state transitions	OpenReview platform plus API v2 and openreview-py [59,112,113]	Ingest submissions, store machine reports, attach decision metadata, and expose venue-specific workflow actions.
Reviewer or editor matching	OpenReview expertise models [60]	Compute affinity scores, seed escalation routing, and support specialist assignment once a paper leaves the automated queue.
PDF-to-structure conversion	GROBID [114]	Convert scholarly PDFs into structured sections, references, and metadata for claim extraction, overlap checks, and policy validation.
Literature retrieval and citation context	OpenAlex and Semantic Scholar APIs [115,116]	Build neighborhood graphs, retrieve likely comparisons, and generate reader-facing cluster or claim pages.
Venue-side author assistance	ICML PAT and reviewer-feedback agents [9,13,14]	Provide pre-submission diagnostics for authors and structured coaching for reviewers without delegating final legitimacy.
Rolling certification pathways	TMLR, ACL Rolling Review, and Journal-to-Conference [42,48,49,50]	Supply external certification channels that can feed conference curation after the machine-first control plane has done initial triage.

The main gap is therefore not whether building blocks exist, but whether the community is willing to combine them into a review system whose center of gravity is automated evidence construction plus human-governed escalation, rather than universal human-first screening.