As promised, I'll explain a bit on the SODA 2012 PC work.

We received 523 submissions. As far as I know, that's a record number. Six submissions were withdrawn by the authors. Four submissions were rejected because they did not comply with the submission guidelines. Out of the remaining 513 submissions, we accepted 138 submissions.

The competition this year was fierce. We had to reject papers that received good reviews and are clearly SODA quality papers.

How were the papers accepted? - Each paper was handled by three committee members. (That amounted to 49 papers per PC member!!!). Some were reviewed by the PC members handling them. Others were "outsourced" to external reviewers. Roughly 1/3 of the accepted papers were obvious accepts, based on the initial reviews. This does not mean that they are necessarily better than the other papers. It means we got good initial reviews for various reasons (a better choice of reviewers, for instance). Another roughly 1/3 of the accepted papers became obvious accepts after discussing the initial reviews and sorting out the real or imaginary flaws that those reviews mentioned. The remaining accepted papers demanded most of the committee's work (I mean the discussion part, not the reading part which demanded even more work).

So how did we choose the last 1/3 of accepts? - We focused on controversial and problematic papers. Those were papers where different reviewers had widely differing opinions, or papers where the reviewers seem to have missed important points. After considerable discussion of these papers (including extra reviewing of some), each committee member chose a "pet paper" to accept. Before you turn on your buddy on the PC for neglecting to choose your paper, please recall that each PC member handled less than 10% of the papers, and the choices were subject to the scrutiny of the entire committee. Those choices accounted for most of the remaining accepts. It turned out that almost all pet papers were controversial due to differing opinions on the importance of the topic of the paper and not on its quality. So this step served to diversify the conference. The remaining few accepts were chosen after a detailed discussion on each and every paper with some issues still pending. (This could be as simple as a missing review, or contradicting reviews, or other issues.) We could not reach an agreement on three papers, so we voted on those. Incidentally, all three were rejected.

Our discussions included checking many details that were claimed about papers, involving PC members beyond the initial three, and sometimes last minute consultations with external reviewers (including having them comment on competing reviews). Serious "accusations" (errors, reproving known results) were presented to the authors to allow them to respond.

Why was my paper rejected? - Around 80 papers were obvious rejects. They received uniformly negative evaluations (not just the score, but also the contents of the review). At the other end were quite a few papers that were deemed decent submissions by their reviewers, but there simply wasn't room for them. There were around 30 that I think would have been accepted in a less competitive year, and many more that were borderline accept for SODA (i.e., mildly interesting). Our policy was NOT to reject papers based on poor writing (really extreme cases excluded). A few papers were rejected because of serious flaws or gaps in the proofs, or because the paper's claimed contribution had a large overlap with known results. We interpreted SODA's scope quite widely, and mostly avoided deliberate balancing between areas. (There was one area where we accepted 8 out of 13 submissions!!) We did compare specific papers to the accepted and rejected piles to strive for some uniformity in decisions.

Finally, I would like to take this opportunity to thank the members of the PC for agreeing to serve on the committee, for the hard work they put in, and for the pleasant atmosphere in which our discussions were conducted.
Shared publiclyView activity