[Editors’ note: This is the first in a series of upcoming posts on conferences and travel.]

It’s time to end the in-person PC meeting. On many fronts it no longer makes sense for 60+ people to travel to one location to each discuss a handful of papers. There seems to be an increasing sentiment that traveling to a PC meeting to participate in discussion of a very small number of papers (1-4) does not seem like the best use of everyone’s collective time. With the trend over the last several years towards the majority of papers being resolved pre-PC meeting via online discussion, each PC member is often left with few papers to discuss. We all want to make thoughtful, careful and fair decisions about the paper selection for our top conferences, but I fear we are holding on to a legacy process not because it works well but because it is what we have always done.

Note: Getting rid of PC meetings will also open up other opportunities such as rolling deadlines, major and minor revision options, journal first publication, etc.  The goal of this blog post is not to re-litigate the R2 discussion but rethinking the meeting model will allow rethinking of other aspects of our publication process. For now, I leave this for a separate discussion.

First, I lay out some major pros and cons of the in-person PC meeting, followed by a few possible alternatives. 

Pros: Ability to engage a wide cross-section of the community in the decision-making for papers. It’s unclear how fully engaged the majority of PC members are in the discussions of most papers but this is a primary goal of in-person meetings. The other benefit to PC members is the ability to network and socialize with colleagues as a result of the meeting.  However, we have other forums for networking so this should not be a primary reason to keep the physical PC meeting. PC meetings also provide mentorship to junior faculty and give them a platform to find their voice.  

Cons: The major downside is the travel and requirement on people’s time. Travel requirements are problematic on many fronts including carbon footprint (see SIGPLAN’s report on climate change and the need for action), the value placed on PC member’s time (which is voluntary), and travel restrictions (whether due to visa issues, the recent COVID-19 outbreak or company limitations on travel). The collective time spent on these meetings is significant. In the 2 months prior to the PC meeting, we have already volunteered our time to review papers. Attending the PC meeting represents approximately 3 days of additional time including travel.  We are not compensated for our time and academics pay for travel out of our own grant funds. Beyond that, I personally limit my travel for personal reasons; my current goal is to travel less than once per month. If I eliminated or reduced PC meeting travel it would free up time in my schedule for potentially more meaningful travel that might lead to research collaborations/funding. Beyond travel and time, it’s unclear if a full PC discussion does in fact lead to the best outcomes; PC members who have not read the paper may add noise to the discussion and are not as informed as reviewers when a PC wide vote is taken. The PC meeting amplifies the opinions of those in the room while potentially sidelining the external experts not there to speak for themselves. 

Alternatives to in-person PC meetings:

Here, I outline a range of possible options.  This list is not meant to be exhaustive but will hopefully spur some additional conversation and thought around how we could change our approach to in-person PC meetings.

1. Replace the physical PC meeting with a virtual PC meeting.  Pros: Eliminates travel. Video conference technologies such as Zoom have improved significantly in recent years. Cons: Still places a large burden on everyone’s time.  It’s not clear that a 60+ person virtual meeting would be easy to pull off effectively.  It would be even more difficult to hold participants’ attention in a virtual meeting than an in-person meeting. People will be more distracted by meetings and work obligations in their local environment. A 60+ person Zoom meeting for 10 straight hours does not sound like any fun. Can you imagine how many times someone will need to be reminded to mute their microphone or how often something will need to be repeated due to fluctuating bandwidth? 
2. A series of small-group virtual PC meetings. Some recent program chairs have done a “split PC”.  In this approach, program chairs divide the PC in half across two rooms or across two days. We could take a similar approach to virtual meetings. Sub-PCs could be formed to facilitate some wider feedback and discussion while avoiding the nightmare that would be a 60+ person video chat. The program chair would have to ensure that a consistent process and standard are applied across all such meetings. Just as every PC invite we receive tells us that we are expected to attend the PC in person, the PC chair could ask that everyone commit to 1.5-2 days for virtual meetings. Meetings could subsequently be scheduled in 2-hour chunks with a subset of the PC. While this format would not achieve engagement of the full PC in discussions, it would allow some broader engagement by PC members who did not review the paper. Creating non-conflicted subgroups may allow discussion across multiple similar papers and better calibration for those papers than is possible with a large group and disjoint conflict sets. A series of small group meetings would also allow for larger overall PCs. Currently PC size is largely dictated by what seems manageable for a single-room physical PC meeting. Pros: Larger program committees (lower reviewer load), greater ability to solicit extra expert opinions, no travel, shorter meetings allowing everyone to be more focused on discussions at hand. Cons: No engagement of the entire PC on papers (this can be good or bad), no calibration across the entire PC, potentially greater challenges in scheduling and assigning papers to facilitate small group discussion. Young researchers often learn how to be good PC members by observing senior members of the community during this process. Virtual PCs might make it more difficult to facilitate mentoring and training of more junior reviewers. 
3. Do all online discussion and decision making. Pros: We already decide nearly 50% of the papers through online discussion.  It’s not a stretch to push this to the limit. It also eliminates this ERC/PC dichotomy – all reviewers will have the same voice in the decision making process. Cons: Not all reviewers engage in online discussion. Anecdotally, active online discussion requires a lot of cajoling and nagging from the program chairs. Exploring ways to lower any barriers to discussion participation would be great. HotCRP is a great platform; however, perhaps we might wish to explore alternative models or features that will allow for more robust online discussion.  Finally, with all online discussion there may be no opportunity for non-reviewing PC members to make comments or ask questions about certain papers. Opening up the entire review site to all PC members over an extended discussion period may raise concerns about review process confidentiality or who has access to the material. This seems highly undesirable.
4. Full journal process. The PC chair or a set of delegated PC members (i.e., associate editors) read the reviews and make a judgment on the paper outcome without any discussion. Pros: Eliminates the meeting; eliminates the time sink of the online discussion period. Cons: Discussion is important to resolve differences of opinion and this puts too much burden on too few people and may lead to many complaints about the process.  

Of the above, some variant of option 2 seems most workable and achieves a balance of the pros and cons of in-person PC meetings. However, I welcome discussion and suggestions of other alternatives.  To facilitate discussion, you are welcome to fill out an informal poll on these options. Please note: this post and the options laid out represent the author’s own opinions and are not related to her service on the SIGARCH Executive Committee. 

We are not the first or only community to be thinking about doing away with the physical PC meeting.  Examples of other communities include: IEEE Security and Privacy (Oakland) which has experimented with a virtual PC meeting using Zoom.  POPL is another example of a conference that has switched to virtual program committee meetings in the last couple years. We can learn from our colleagues on how best to approach such a change and pitfalls to avoid. 

Finally, we need to proactively think about and explore alternatives to our current physical PC meetings so that we are not caught flat-footed by an extreme situation as with COVID-19.  To do this well, it will take a lot of discussion and planning. 

Disclaimer: Lest anyone think I am complaining about the running of any specific recent PC meeting, I’m not. I have the utmost respect for our program chairs – they do a tremendous service to our community and every PC meeting I’ve attended has been very well run.  I also appreciate the innovations that have been introduced over time to improve the process. However, I do not see incremental improvements as a viable path forward. It’s time for our community to embrace something new and different.  

About the author: Natalie Enright Jerger is the Canada Research Chair in Computer Architecture at the University of Toronto. 

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.