Guidelines for SIGARCH Sponsored Conferences

David A. Patterson
Computer Science Division /EECS Dept.
University of California, Berkeley
April 20, 1994

At the business meeting on April 20, 1994 it was voted that SIGARCH adopt this set of guidelines to aid program chairs on matters of policy at SIGARCH sponsored conferences (e.g., ISCA, ASPLOS, and so on).

SIGARCH Conference Guidelines

Amendments to the guidelines would be voted on at SIGARCH business meetings. There are several options in these guidelines, so the program chair will report which options were selected in the chair’s message that appears in the proceedings. To ensure that the guidelines are part of a tradition that is not forgotten, ACM will include them as part of the conference proceedings of the International Symposium on Computer Architecture (and thereby appear in Computer Architecture News once a year).

1. Quality vs. Balance
Papers will be judged on their scientific merit and anticipated interest to conference attendees. Balance means that the paper should be accepted for some other reasons than quality as judged by reviewers. Examples of non-quality issues that have risen are:

• Too many papers from one author;
• Too many papers from one institution;
• Too many papers from one country;
• Too many papers on one topic.

The policy of SIGARCH is that quality alone should be the guide on the first three issues. (It is understood that papers in new areas are likely to contain less quantitative evaluations and comparisons than those in more established areas.)

The last issue is a very sensitive one, for the program committee must be extremely careful in limiting the number of high quality submissions even if all are in a single area. In particular, the program committee should keep in mind that impact and relevance are an important component of quality. SIGARCH has played a vital role in improving the architecture of the commercial computer industry, and it is important to the field for that role to continue. Balance at the cost of impact and relevance must be avoided. (Discussions on statistics of balance must be delayed until after papers have been evaluated, if such statistics are discussed at all.)

One non-quality issue for a conference is papers likely to stimulate interesting and productive conversations at the conference. This is a perfectly acceptable reason to include a paper in the program.

2. Program Chair Selection

The program chair is key to the success of the technical papers, and hence the decision is critical to the success of the conference. Most SIGs separate the decision of the location/general chair of a conference (which needs to be made 3 or 4 years in advance) from the selection of the program chair (which needs to be selected no more than 2 years in advance). We recommend following this tradition, as career changes may make it difficult for program chair candidates to make or keep their commitments. (The decision should be made early enough so that the chair of the succeeding conference can be an unofficial member of the current program committee.)

3. Program Committee

The program chair has the dual goals of a high quality committee and finding representatives from several communities. Hence diversity is important in selection of the program committee (PC). This policy encourages submissions from many groups plus offers direct access to different groups of competent reviewers. The program chair should consider the following when selecting a PC:

• Personal qualities: judgement, reliability, ethics, standards;
• Coverage: technical areas, leading universities, leading companies;
• Balance: technical areas, geographic, academe vs. industry, youth vs. experience, and so on;
• Planning for the future: grooming future program chairs.

There should be significant turnover as well as some overlap between successive PC memberships.

Before picking a committee, the program chair should consult with prior program chairs for input along these four issues. It will be helpful if the program chair of the succeeding conference is invited as an ex-officio member of the program committee and participate at the PC meeting.

4. Reviewing Papers

At least two members of the PC must review each paper themselves in addition to external reviews. We encourage the recent tradition of blind reviews, asking the authors to blank out the names on self-identifying references in their submissions.

5. Program Committee Meeting

There must be a PC meeting where all PC members should be in attendance. Assurances should be made by the PC member that he or she will attend before agreeing to serve. If the makeup of the committee prevents the vast majority of the members being at a single meeting, then the recommendations of multiple meetings must be resolved at a single meeting containing many representatives of each meeting.

Members of the committee must not participate during the discussion of papers in which they have a conflict of interest. SIGARCH uses the National Science Foundation definition of conflict of interest, which will be distributed to the PC. (See NSF Policy on reviewer conflict of interest which recommends selecting reviewers who are not at the same institution, are not former PhD advisors/advisees or postdoc advisors/advisees, and are not known to be personal friends or antagonists of the authors.)

In consultation with the PC, the chair either:

1. Keeps the PC meeting blind, with papers unidentified; or
2. Identifies the authors so the PC meeting is not blind.

6. Papers by Program Committee Members

The first issue is the threshold of acceptance for program committee papers. The three policies are:

1. The quality of the PC papers must be at least as high as the threshold of the acceptance of regular papers. (This encourages participation of active members of the field in the PC while being fair to the non-committee papers.)
2. The quality of the PC papers must be much higher than the threshold of the acceptance of regular papers. (This reduces the chances of the appearance of bias.)
3. PC members cannot submit papers.

The program chair must pick the model and inform PC members before they are asked to join.

A second issue to address is the number of papers that a member of the PC can co-author. In return of the honor of being on the PC, the default policy should be no more than two papers submitted per PC member. (This addresses the appearance of impropriety of too many papers by a single PC author at the conference.)

The third issue is the method of acceptance PC papers. This is not a simple problem. The issues are being fair to PC papers, being fair (and looking fair) to papers that are not co-authored by the PC, and not having awkward situations or peer pressure at the PC meeting. The common points:

• The program chair and general chair are not authors or co-authors of any submissions.
• Submissions are divided into PC and non-PC papers.
• The whole committee decides the fate of the non-PC papers.

There are again three options:

1. “After regular papers” model: The PC authors are thanked and dismissed after the non-PC papers have been selected. The non-submitting members then decide the fate of the remaining (PC) papers.
2. “Before regular papers” model: The program chair sets up an informal committee, possibly chaired by a prior program chair or the general chair, and the fate of the PC papers is decided before the PC meeting. The meeting starts with the list of accepted PC papers. (This option makes it difficult to apply the non-PC threshold to PC papers.)
3. “Hot seat” model: Only the author is excused when a PC paper is evaluated by the PC. (This can be embarrassing, plus it is not blind.) A variation of this model is PC members are asked to leave the room whenever any paper from their institutions is discussed, which reduces embarrassment since it is unclear who the author. It also shields PC members from questions by colleagues about what happened to their paper.