CyberChair - An Online Submission and Reviewing System, or: A Program Chair's Best Friend

Richard R. van de Stadt
Department of Computer Science, University of Twente, The Netherlands
{stadt@cs.utwente.nl}


Introduction

Nowadays it is common practise to use electronic submissions for conferences. Usually, people are allowed to submit their paper by email. Compared to sending hardcopies by surface mail, this is a big step forward. However, it may still be quite cumbersome to maintain the administration. Moreover, collecting, maintaining and comparing reviews (and the comments they contain) by the Program Chair (PC) may become quite tedious, if not troublesome, due to the large amount of material, even when submitted by email.

In 1996, we decided to set up CyberChair [1] to automate the reviewing process for ECOOP1. This turned out to work very well, especially because the reviewers were able to look at each other's reviews prior to the PC meeting, so that they arrived well prepared. CyberChair has evolved and matured over the years and has now become a complete system that deals with all administrative tasks, such as the submission of abstracts, (camera-ready) papers and reviews. It also takes care of the notifications and proceedings. CyberChair is only one of many conference management systems [2].

So far, CyberChair has been used 4 times for ECOOP and was succesfully installed on other systems, for the ESEC/FSE'99, ICSE 2000, IWPC 2000, SAS 2000, and FSE 2000 conferences. NOTERE 2000, POPL 2001 and CSMR 2001 are also considering using CyberChair.

CyberChair and the Web

CyberChair uses the web for the submission and distribution of all data that eventually will result in the proceedings. The authors use webforms to submit papers and author information. The reviewers use the web to download papers and to submit2, update and read reviews. The PC is able to follow the process through the web, using several overviews that are periodically generated. This way, all people involved work towards the proceedings in a collaborative way.

The Process

CyberChair is more or less described in and based on the paper Identify the Champion by Oscar Nierstrasz [3], which describes the peer review process for technical contributions in computing science, using an organizational pattern language.

Some time before the paper submission, authors must submit an abstract of their paper, to give the reviewers time to indicate their preference for papers they would like to review. Several overviews are generated for this purpose. Abstracts are categorized by the conference topics that the authors indicated on the submission form. Based on the preferences of the reviewers and their expertise of the conference topics, CyberChair computes a paper assignment proposal for the PC. The preferences of the reviewers are honoured as much as possible, provided that all papers are assigned to an equal number of reviewers, and that all reviewers get as much as possible an equal number of papers.

Reviewers can download the papers they have to review from their personal, password protected webpage. After submission of a review, reviewers will be able to read the other reviewers' review. The reviewers' pages are regenerated frequently in order to reflect changes, such as newly submitted and conflicting reviews. The status of reviews is instantly indicated after submission of reviews, using a special coloring scheme, which immediately points out conflicting reviews, so that the reviewers and PC may give it special attention. Reviewers are encouraged and supported in discussing review conflicts by email, prior to the PC meeting. Reviews can be updated.

The Program Chair can monitor the reviewing process using several overviews, Papers are categorized using the highest and the lowest classification given by the reviewers (A to D, A=Accept, D=Reject), which helps to focus on the papers that have a fair chance of being accepted. The categories are described in the paper Identify the Champion. CyberChair generates overviews of reviews that can be used during the Program Committee meeting. After the meeting, CyberChair sends the (anonymized) comments to the authors. CyberChair also supports the preparation of the proceedings (using the format required by Springer Verlag, the publisher of the proceedings of ECOOP).

Conclusion

CyberChair significantly reduces the workload of the PC, by doing all necessary bookkeeping. It eases the submission of author information, papers and reviews instantly in electronic form, using the web. Further, it spreads the knowledge of reviews over all reviewers, resulting in a high-quality paper selection process. All the PC has to do is to watch the process, prepare the PC meeting, write an acceptance and rejection message and the preface of the proceedings.

References

  1. The CyberChair homepage, http://trese.cs.utwente.nl/CyberChair/
  2. Richard Snodgrass, Summary of Conference Management Software, http://www.acm.org/sigs/sgb/summary.html
  3. Oscar Nierstrasz, Identify the Champion, In: Pattern Languages of Program Design 4, N. Harrison, B. Foote, H. Rohnert (Ed.), vol. 4, Addison Wesley, 2000, pp. 539-556, http://iamwww.unibe.ch/~oscar/Champion/
  1. Mehmet Aksit and Satoshi Matsuoka were Program Co-Chairs of the 11th European Conference on Object-Oriented Programming, ECOOP'97
  2. Reviews may also be filled in off-line and emailed. They are converted by CyberChair and can be updated using the web.