1. Yoav Artzi
2. Log
3. Peter Pan Conferences: What If a Conference Refuses to Grow?

Peter Pan Conferences: What If a Conference Refuses to Grow?

The explosion in ICLR submissions is a sign of things to come. This writeup proposes that conferences should consider a hard cap on the number of submissions they can handle, thereby capping the load on both reviewing and size. This post outlines a partial mechanism, somewhat with the aim of soliciting suggestions.

Posted on: October 09, 2026

[While I am on the COLM board, this writeup does not represent the position of COLM or any planned or recommended policy. It is simply to air some ideas, and elicit thoughts.]

COLM 2026 had a significant increase in the number of submissions, but it was largely spared of the ICLR 2027 60k explosion.¹ This was simply a matter of timing. COLM’s deadline was March 31, Mythos was released to limited preview on April 7. NeurIPS, while seeing an increase, also avoided the deluge. ICLR is a sign of things to come.

The demand of reviewers, ACs, and even SACs has exhausted the reservoir of experience and attention the ML++ community can provide. While not wholly, much of the process has succumbed to being a procedural game of filling positions with warm bodies – individuals to sign their name on forms. Even before the ICLR explosion, because of randomness and scale, many have been wondering if acceptance carries any meaning. Recent trends only amplify this doubt.

COLM has not exploded yet, and still benefits from a size that allows for a meaningful process and a relatively high quality attendance experience. The crux of effectiveness of the COLM process is a relatively small and controlled pool of ACs, who are encouraged to exercise independent judgement and own their recommendations, even if that means overriding or down weighting low quality reviews. Naturally, the process is not perfect and remains noise. However, whatever effectiveness exists now, it is not designed to scale. I have heard many concerns about impending, and largely seen as inevitable scaling up of COLM.

This raises the question: what if a conference refuses to grow?

Consider a framework for a mechanism that enables this. The conference sets its reviewing capacity, the number of submissions the program committee considers it can review in a relatively meaningful way. This quantity is split into two: X% are reserved to the community set, and Y% are randomly distributed. For example, given a capacity of 1,000 and a 75/25 split, 750 may be allocated to the community set, and 250 will be distributed randomly. The community set is reserved to authors that are members of the community, for example authors who had accepted papers in the conference in the two prior years. The framework does not specify the exact mechanism of allocating submission credits, but dictates the credits sum up to the community share. The random split simply samples randomly from all papers. In addition, authors have a meaningful and strict maximum cap.²

In practice, this pushes some of the filtering process to PIs. Given limited quota, they may have to decide between competing papers in their lab. If the conference provides a better reviewing process, PIs will have to decide which of their submissions has the best chance, so as to use their allocation optimally. This can increase the overall quality of the submission pool, and may imply an increase in the acceptance rate. It is hard to speculate on long-term implications. A possible outcome in the long-run is a reduction in the number of papers, with the “mass” of each paper submitted increasing to improve its chances, as it consumes a costly submission slot.

This is not a free lunch. It carries meaningful costs. A core value of scientific conferences is that an impactful contribution can come from anywhere. This tenet of scientific publication is violated by the prioritization of incumbents and random pruning. This process attempts to balance between values that have become competing. Providing a meaningless, but equal process to all is pointless. It also likely much harder to apply this on the way down (i.e., shrinking an already gargantuan venue), than on the way up.

Critically, the exact mechanism of allocating the submission slots in the community pool is still not specified. I have been struggling with specifying it.

What about aggressive desk rejection or lowering the acceptance rate, two alternatives that came up recently in the chatter.

Aggressive desk rejection can help, but at significant cost and likely dubious process quality. While early desk rejection may protect the bulk of the reviewing process, current volumes will create a desk-rejection reviewing load that is already unaffordable. Unless automated, a desk rejection stage would require as many high quality decisions as the number of submissions. It will also require them fast. The quality is doubly important here, because the fate of each submission will be decided based on a single, quick decision (in practice, ICLR seem to require AC recommendation, then SAC recommendation, and then probably an automated PC decision). Even at 10k submissions, this load is high, especially when considering the relatively limited pool of reliable reviewers. Complete automation without any human intervention is more scalable, but carries fundamental problems.

Lowering the acceptance rate will not reduce the reviewing burden. If anything, it may amplify growth of pressure. While the noise at the top 10%, for example, is likely better than within the first 25%, even the 10% remain noisy. Reduction of the acceptance rate can reduce the convention size, which is a good outcome. However, across years, it will only amplify the already existing incentives to flood the system with submissions. If you throw more pasta at the wall, it is more likely that some of it will stick, even if the wall just became more slippery.