I was recently talking to my UC Berkeley colleague Sophia Shao about the 50th International Symposium on Computer Architecture (ISCA) on June 17, and we discussed which papers are highly cited. The conversation inspired me to try to find the 50 most cited papers from the 50 ISCAs.

Alas, Google Scholar does not provide a simple query to answer this question, so I tried other means. My Google colleague Partha Ranganathan recently showed me the “ISCA Hall Of Fame” (HOF), which is a list of the most prolific authors of ISCA papers that was developed and is maintained by the good folk at the University of Wisconsin Madison. With the help of Google Scholar, I looked for highly cited ISCA papers from the 118 HOF authors who individually published 8 to 41 ISCA papers and who collectively wrote 1075 ISCA papers. I found many highly cited papers by these authors were published elsewhere (e.g., ASPLOS, HPCA, MICRO, SC), so keep that in mind. The IEEE Xplore repository has a table of contents for nearly all ISCAs from 1988 to 2022, so I used Google Scholar to see if papers with high Xplore citation counts are in the Top 50. I also checked the number of citations of the Influential ISCA Paper Award winners, which are given 15 years after a paper appears. It started in 2003, so it covers 1988–2007. Finally, very late I learned of an effort by Gaurang Upasani et al. to use an internal Google Scholar API to write a paper about the top ISCA citations per year of the conference. We compared notes to debug both of our lists.

The 2-part table lists the Top 50 most cited ISCA papers from following this process, showing the first author (underlined) and the HOF authors (in italics), if the paper won the ISCA award, and the type (see below). Google Scholar citation counts update hourly; this table is a snapshot of the citations on April 25. These 50 papers are ~2% of the 2134 ISCA papers from 1973 to 2023.

Table 1 partA (PDF version with live links to papers)

Table 1 partB (PDF version with live links to papers)

As a numbers guy I find the table fascinating, but keep in mind the cautionary comment from the HOF site, which I believe applies to citation ranking as well:

 “A real Hall of Fame should be determined by impact, not paper count.” 

Nevertheless, it’s hard not to notice that Norm Jouppi—also at Google—won 2 Influential ISCA Paper Awards and co-authored 2 of the Top 10 and 5 of the list of 50. Others who co-authored several top 50 papers are Joel Emer (5), Bill Dally (4), Anoop Gupta (4), Doug Burger (4), Kourosh Gharachorloo (3), John Hennessy (3), Onur Mutlu (3), and Dean Tullsen (3). These 9 of the 125 ISCA HOF architects co-authored 20 of the Top 50 papers. Put another way, of the top 35 HOF authors (≥15 papers), only 10 have multiple Top 50 papers, 15 have one paper, and 10 have none.

I then dove one level deeper into this data. I first divided the top 10 and all 50 papers by type and topic. My 3 types are:

1. Microarchitecture: Architecture techniques that could be used inside many computers;
2. Architecture: A description or proposal of a full computer architecture; and
3. Tools: Tools to help architects design computers, such as simulators or benchmarks.

Looking at the type mix in the pie charts below, it’s unsurprising now that tools that help architects are a larger fraction of the top 10 than of all 50, but that wasn’t obvious to me beforehand. In fact, the Top 50 list has only 5 tool papers, but 3 are in the top 5.

Distribution of Top ISCA papers by type

The architecture topics in the pie charts below are hopefully self-explanatory. As the chip design mantra today is power-performance-area, it is no surprise that parallelism and power are large slices. Machine learning (ML) accelerators have an unexpectedly large slice of the top 10 and of all 50, since the current excitement about deep neural networks started only a decade ago—the 9 ML papers in the table are from 2014 to 2017. ML papers in general are among the most highly cited in all of science and engineering, e.g., the ResNet ML paper from 2016 has 161,000+ citations. Thus, the huge popularity of ML today likely accelerates citations to ML accelerators.

Top ISCA papers by Topic

I showed the data to Thierry Tambe, who was visiting Berkeley. As a young architect, he wondered which papers were highly cited in the past five years. There is no easy way to check recency in Google Scholar¹Google’s h5 index is for papers published in the last 5 years, not the most citations in the last 5 years for papers of any year., so I manually adjusted the citations. Based on data I collected for the first table, as long as a paper has at least 700 citations, it is likely one of the most widely cited papers. The table below answers Thierry’s question.

As one might expect, the ML accelerated papers dominate the most recently cited papers, occupying the first 7 slots and 9 of the 15. Sophia Shao noticed that unlike the Top 50, most were architectures, presumably because it is easier for small groups of researchers to design new domain specific architectures than general purpose ones. These papers are also more recent than the Top 50: their median year is 2014 versus 2003 for the Top 50. My colleague Cliff Young noted the staying power of the SPLASH-2 paper, which appeared 12 years before the next oldest paper. The top paper in Table 2 added ~1200 citations since January 2022 vs ~120 for the top paper in Table 1, so might become the highest cited next year.

Finally, I looked at how Top 10 and Top 50 citations change over time. The histogram below plots the number of cited papers by year of the conference. It takes a few years after a paper is published before it can attract 1000+ citations, so it’s not shocking that the newest highly cited paper is from 2017. 

Distribution of Top ISCA papers by year of publication

The paucity of papers from the 1970s and 1980s is shocking. Speaking as an author of some of those papers, I have three observations that might explain what happened (other than that they were inconsequential!)

1. ISCA predates search engines and digital libraries. For example, ACM’s digital library started in 1998. Finding related work in the 1970s and 1980s was a lot of work—you had to go to libraries to read physical papers and to chase references—which reduced the number of references per paper. 
2. ISCA and ASPLOS changed policies on paper length for references in ~2015, hoping to increase the number of citations to computer architecture papers. Digital libraries plus no limits increased references. The 27 papers of the 1978 ISCA averaged 12 references (316 total), while 40 years later ISCA had 63 papers that averaged 58 citations or 5x more (3636 total). The ISCA 2018 total was ~12x more than 1978.
3. There are also many more outlets for papers today due to services like arxiv as well as more architecture conferences that accept more papers. David Wood pointed out that in 1990 ASPLOS was biennial so ISCA plus a half of ASPLOS equaled ~50 papers. ISCA, ASPLOS (now annual), and HPCA (which started in 1995) published ~240 papers in the past year, or ~5x more papers. Thus, there might be 5×5 or 25x more citations from refereed architecture papers published in 2023 than from those in 1990.

The table on citations since 2018 suggests that recent papers are more likely to cite newer papers than older papers. If so, then the three observations above explain why the 1970s and 1980s might be at a competitive disadvantage, but they were not necessarily unimportant. These hurdles make it all the more impressive that one paper from the first ISCA in 1973 made the Top 50 list (#43). Its HOF author was Jack Lipovski, who was also the first general chair of ISCA. 

It seems a proper tribute to leave the last words to Jack, whose observations from 50 years ago remain just as relevant today:

This symposium may well be, in the hind-sight of ten years from now, a marked turning point in Computer Architecture. With the dissolution of the Spring and Fall Joint Computer Conferences,²Starting in 1951, the SJCC and the FJCC conferences covered all of computer science and reached 15,000 attendees at their peak. The Federated Computing Research Conference (FCRC), founded by the Computing Research Association in 1993, is an attempt to recreate the benefits of large multidisciplinary meetings like the SJCC and the FJCC by colocating several single discipline conferences at the same venue and at the same time. ACM now holds FCRCs every 4 years. one of the major forums for Computer Architecture has been lost. So we have begun an annual symposium on Computer Architecture, to be rotated from year to year around the world. … 

The papers in the symposium indicate the growth of Computer Architecture as a science. Although it is difficult to explain the reasoning behind the decisions made in an architecture, in particular, the architecture of a practical machine, this reasoning is the basis of a science. … 

We hope that attendees will emphasize questions on the reasoning behind the architecture, and the authors will prepare for such questions. If this becomes a tradition in this annual symposium, it should orient authors toward the scientific explanation of their architectures for later symposia.

— Jack Liposvki, Preface of the Proceedings of the First Annual Symposium on Computer Architecture,

University of Florida, December 9–11, 1973

About the author: David Patterson is a UC Berkeley Pardee professor emeritus, a Google distinguished engineer, the RIOS Laboratory Director, and the RISC-V International BOD Vice-Chair. His most influential Berkeley projects likely were RISC and RAID. The most prominent of his seven books is Computer Architecture: A Quantitative Approach. ACM SIGARCH presented him with the Eckert-Mauchly Award and the Alan D. Berenbaum Distinguished Service Award. He and his book co-author John Hennessy shared the 2017 ACM A.M Turing Award and the 2022 NAE Charles Stark Draper Prize for Engineering.

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

• 1
  Google’s h5 index is for papers published in the last 5 years, not the most citations in the last 5 years for papers of any year.
• 2
  Starting in 1951, the SJCC and the FJCC conferences covered all of computer science and reached 15,000 attendees at their peak. The Federated Computing Research Conference (FCRC), founded by the Computing Research Association in 1993, is an attempt to recreate the benefits of large multidisciplinary meetings like the SJCC and the FJCC by colocating several single discipline conferences at the same venue and at the same time. ACM now holds FCRCs every 4 years.