The day is April 14, 2023. The MICRO deadline is just a few hours away, and micro-architects are skeptical about their MICRO submissions. Why? There are only a few sessions on microarchitecture from MICRO 2018 to MICRO 2022. Is this the end of microarchitecture research? Is this the beginning of one more wall, the microarchitecture research wall? This blog discusses this new wall in computer architecture and provides ways to break this wall.
CPU microarchitecture research RIP (Rest in Peace)? Let’s look at the conventional microarchitecture research: (i) branch predictors, just impossible to push its limits (more than a decade since a branch predictor paper appeared in ISCA); the research area for a few branch predictor ninjas, (ii) hardware prefetching for data, instructions, and translations getting increasingly difficult to improve the state-of-the-art by more than 2% courtesy recent data/instruction prefetching techniques, (iii) cache replacement policies, a recent work Mockingjay shows Belady’s pursuers have finally managed to make Belady’s policy possible without knowing the future, and (iv) value predictors, hard to beat the existing ones, even the 2nd CVP (Championship Value Prediction) co-located with HPCA 2021 was a no show as it was just impossible to improve on top of the leader of the CVP-1. Welcome to the microarchitecture research wall.
The microarchitecture research bias. The wall becomes unbreakable because of the “microarchitecture research bias.” Reviews like “it is a heavily mined area,” “this problem has been beaten to death by our community,” “marginal performance improvement of 2% on top of state-of-the-art, I was expecting more,” “incremental improvement,” “quantitative benefits are pretty small,” makes the microarchitecture research wall stronger and unbreakable. To emphasize the impact of a 2% improvement, let me quote Prof. Jiménez (recipient of the 2021 B. Rau Award), who mentioned that “a 1% speedup in the industry on the realistic simulator is a cause for celebration.” For the industry, 1% improvements on multiple microarchitecture ideas make a big difference in the final revenue. However, we know that reviewer-2 will not agree with this statement, with a defense that industry simulators are not noisy; given that academic simulators are noisy, the improvement should be significant for a reliable conclusion, which is extremely discouraging for graduate students. If this trend continues, the PC chair of MICRO 2032 will have a tough time finding “expert” reviewers to review an awesome paper on value prediction.
Breaking the microarchitecture wall.
(i) AI-assisted microarchitecture. It is time for AI-assisted microarchitecture. AI can help us understand the limits of non-AI-based microarchitecture techniques; what is the scope, and maybe we can use the insights from AI-based approaches and feed them into non-AI-based approaches. Recent works like Voyager and BranchNet attempt to push the limits keeping the practical issues aside. Usage of reinforcement learning is an effective way to capture the system’s dynamics for making microarchitecture techniques versatile like Pythia.
(ii) Post-silicon microarchitecture. Another exciting direction is the concept of post-silicon microarchitecture (PSM). The idea of post-silicon microarchitecture is that the microarchitecture decisions can be designed/enhanced, post-silicon, providing greater freedom for the microarchitecture community.
(iii) Questioning the existing microarchitecture ideas. Although microarchitecture research is crowded, there is still a scope to ask some fundamental questions like do we need the conventional cache hierarchy as discussed in CATCH? Can we design inclusive caches that do not create inclusion victims (Zero inclusion victim)? Our state-of-the-art hardware prefetchers, are they energy-efficient [CAL 2021]?
The renaissance of microarchitecture research. There are new challenges for the microarchitecture community on top of the existing challenges, which are as follows.
(i) Old decisions for new application domains. Application domains like Graph analytics and Machine learning pose new challenges for the old microarchitecture problems as the application behavior and demand are different.
(ii) Secure microarchitecture. After the disclosure of attacks like Spectre and Meltdown, all the microarchitecture units from the branch predictor to the on-chip DRAM controller must be re-examined for security (both offensive and defensive sides), performance, and energy. There is also scope for redefining microarchitecture and the ISA interface.
(iii) Microarchitecture-OS-compiler-programmer interface. As our community goes more and more toward the vertical approach, microarchitecture interactions with the upper layers of the computing system stack are becoming increasingly crucial. Ideas like polymorphic cache hierarchy open up interesting microarchitecture research. Similarly, ideas like REDUCT do what Hadoop did with software, with code moving towards data instead of data moving towards code. With virtualization, microarchitecture for Cloud must address interesting performance, power, and security challenges.
(iv) Microarchitecture for persistent memory hierarchy. Traditional microarchitecture decisions are based on the volatile nature of cache and memory. With the emergence of persistent memory and caches, it is time to rethink microarchitecture decisions, given the new challenges from the inter-play of latency and bandwidth.
What about the famous reviewer-2? Reviewer-2 will have less bias on the new microarchitecture topics. The reviewer can look for “insights” instead of 1% performance improvement for the conventional microarchitecture problems. Questions like “will this be implemented in the industry” can be replaced by “will this insight be useful for the community, both for academia and industry?” Simulators are becoming more realistic day by day, and artifacts and benchmark traces are available publicly; it is the best time to convince Reviewer-2.
So, after a bit of introspection, the micro-architects realized that microarchitecture research is alive and strong in new avatars. We, therefore, anticipate that MICRO 2023 will have many microarchitecture papers. Stay tuned.
About the Author: Biswa is an assistant professor at the Computer Science and Engineering Department, Indian Institute of Technology Bombay. His research interests span microarchitecture for performance and security.
Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.
