This blog is a part of a two-part series on the subject of architectural evaluation methodology. The first blog was a collection of opinions from experts on the question “When to Prototype? When to Simulate?”. For those who missed part-I, the picture below is a reasonable summary of the solicited opinions.
Paradigm compatibility is a good metric to determine when to invest efforts in prototyping. If the novel technology has paradigm compatibility (e.g. SMT), phase simulations are relatively low-effort but accurate. If the project requires invasive changes in the software stack, an FPGA prototype will provide the necessary speed to run the new software and hardware stack. If the technology is disruptive enough that today’s digital circuits can’t mimic it (e.g. Razor) then a prototype is a useful demonstration of the technology. Prototyping few incompatible components of your architecture and bridging the gap with simulations may be an effective way to reduce engineering efforts. These views are tainted from the perspective of academic research. The answer is likely to be quite different if you are trying to transfer the technology commercially through a start-up or with your industry partners.
This blog asks the difficult question: How does this impact our students?
Snippets in alphabetical order: Todd Austin, David Brooks, Doug Burger, Michael Taylor, David Wentzlaff.
Todd Austin “In terms of prototyping, I think that the challenge with respect to students is carving up the typically large prototyping effort, which requires many students’ effort, into multiple high-quality PhDs. In my opinion, this is an incredibly difficult thing to do with only PhD students, especially since the cost of prototyping often leads to a purposeful reduction in the prototype’s overall functionality. In my lab, I like to surround one or two PhD students with a gaggle of MS and BS students, that help with the low-level design details. This has been in my experience a win-win situation for all the students: the few PhD students on the project get a meaty thesis with a high visibility project backing it, and the MS and BS students get the incredibly valuable experience of bringing a large prototype to fruition, which is sure to land them some really great job offers.”
David Brooks “This is a tough question, especially in the context of faculty hiring where CVs are getting longer every year. There is often a trade-off between more papers and prototypes. A small chip can take 9 months of intense effort from RTL design through testing, and if there are design issues it can require another tape-out cycle. In the limited time available for a PhD, should a student spend 9 months writing another simulation-based ISCA paper or working on a prototype? This is a question that only hiring committees can answer.”
Doug Burger “While large projects require coordinated efforts between many students, I don’t believe there is a hero effect (i.e. one student stands out of the group). It’s possible and a good goal to carve out individual pieces of focus for students. Prototyping is a great way of training students and students need to develop diverse skills during their PhD. I would like to see more high-risk projects.”
Michael Taylor “The greatest strength of prototyping is that it trains the neural nets in you and your student’s brains to better model a certain class of system so that you come up with better ideas, and write more correct future simulators. I believe all architecture graduate students should play a part in prototyping a system as part of their path to PhD, so that they have good intuition for their future careers, as well as solid engineering skills.”
David Wentzlaff “Building hardware prototypes can significantly increase both the learning from and the impact of one’s research. Not only does it get students excited, but by prototyping you learn so much. In particular, I have found that through prototyping, we have learned the true implementation costs of a design that are typically ignored during simulation. In particular, many times, in simulation, the true design complexity, hardware size, or timing constraints of a hardware structure is ignored. For example, in a real chip, communication distance takes time (wire cost) often requiring a hardware structure to be replicated or add additional cycles, but this is often missed in simulation where communication is effectively free. I have even seen whole portions of a design that are missed in simulation (my own research included), but missing structures is clearly not possible when prototyping in hardware. I feel that one of the largest benefits of building hardware prototypes is enabling students to see the future! When building cutting edge prototypes, many times it is not even the idea that is being evaluated which is the largest benefit, but rather building provides the students the springboard of future research ideas from the pain points of the current design or by deeply understanding the parameters of advanced process design. Equally important, by building complex processors, you also learn what ideas not to pursue. Time and time again, I have seen research fads start based on flawed understanding of either implementation or device constraints, many times which would have been easy to steer clear of if the researchers would have built a simple prototype. Prototyping also increases the impact of the research. Creating a hardware prototype eases technology transfer to industry or the government because it effectively de-risks the design because a proof-of-concept has already been created. History has proven this out with many hardware research projects being transitioned to industry or being used as the starting point for a start-up company. The impact of prototype hardware can be even further multiplied by open sourcing your design. We did this with our OpenPiton project (http://www.openpiton.org) where we released the RTL of a fully configurable, cutting-edge, manycore processor that runs full stack Linux with a test suite of 8000+ tests, FPGA synthesis scripts, and ASIC backend scripts. Last, I would like to encourage researchers who have never prototyped to give prototyping a try! It is a great experience that gets students excited, gets funding agencies excited, and the type and quality of insight gathered can be difficult if not impossible to gather from other means such as analytical models or simulation.”
Parting Thoughts
It’s a tricky choice. On one hand, it’s an important skill to develop. On the other hand, it potentially limits the number of creative ideas students can explore during their PhDs. Maybe building a prototype as a culmination of related ideas is a good balance. A methodology which builds smaller sub-components as prototypes and bridges the gap with simulation is another option to balance. This approach is financially cheaper and less burdensome on students. Involving MS and BS students is a good idea to spread out the work. We definitely need more open source hardware to reduce the engineering efforts and promote high-risk projects in our community.
About the Author: Reetuparna Das is an Assistant Professor is University of Michigan. Feel free to contact her at reetudas@umich.edu
Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.
