I’ve always envied my Physics colleagues, because their field has a history rich with heroes and epic stories¹I have other reasons for Physics envy—it’s the hardest of the “hard sciences”, one step away from the pure rigor of Mathematics. I worry that CS needs “Science” in its name to bolster its weak science-y-ness. The French term Informatics is much better, as our discipline spans math, science, and engineering. . Newton had his apple (and also found the Calculus). Einstein believed that God did not play dice, and he wrote to Roosevelt about building an atomic bomb. Gauss was the last physicist to know everything. We computer architects might know Shockley, but Bardeen won two Nobels (for the transistor and superconductivity). Every physicist I know tells these stories, which are as much culture and lore of the field as they are part of its history. In contrast, Computer Science is a young field (less than 100 years!), so we have fewer such stories. Turing has gotten biographies and biopics, but The Soul of a New Machine and Fumbling the Future have yet to get their movie treatments. 

It’s time we in computer architecture did better. Let’s teach our students our history, not just the famous successes but also the ambitious failures. Beyond the purely technical history (many professors have wonderful reading lists associated with their graduate-level classes), let’s construct our own mythology, with our own set of heroes and their exploits. Along the way, let’s note why each success worked in its age, and what we think caused each failure. A tragic hero is still a hero. As a culture, we celebrate successes and don’t dwell on failures, but this means that we also don’t document failures—I wish there were the equivalent of NTSB accident reports about i860, Itanium, and Haswell’s TSX (to unjustly pick on Intel).²For an example of how to write such a report, see Colwell’s analysis of the Intel iAPX 432. Just reading section 1.1 is like the start of a horror movie.  The NOPE workshop is a good start. Such reports would let us better learn from the past—not just to avoid repeating past mistakes, but also to understand their tragic flaws, because a flaw from one technological era can transform into a virtue in the next. As a poor substitute, I’ll offer some of my (brief) opinions about the technical reasons for failure of various projects.   

I have a bigger agenda. My advisor, Michael D. Smith, taught me “There are no new ideas in computer architecture, only old ones whose time has come.” Mike’s maxim matches my experience: Anton used every parallelism technique I know save threading to show that special-purpose machines could deliver multiple orders of magnitude speedup; the first TPU combined systolic arrays, decoupled access/execute, and CISC, breathing new life into old (and let’s be honest, unpopular) ideas. I’ll bet that the next transformative machine also combines old ideas in surprising new ways. 

As this is a blog post, I can blithely ignore peer-reviewed rigor, adopt a more informal, subjective style, fail to remember important contributions, and make tart and most likely wrong comments along the way. But I have to keep things brief, so I’m going to present my history in bullet points and summarize huge, complicated areas with sentence fragments. Also, I’m going to focus on the machines and the people who built them more than particular techniques (although those also play a key role). With all these caveats, here’s my subjective, biased, and incomplete history of computer architecture, as a first draft for the lore and epic tales of our field. 

The 19th Century is mostly about industrialization, but the seeds of computer architecture slowly germinate: 

• In 1804, Joseph Marie Jacquard invents the Jacquard loom, an industrial weaving machine with the first stored program. Weaving transforms one-dimensional thread into two-dimensional fabric. Looms keep a set of threads (“the warp”) which can be selectively raised or lowered before a shuttle carrying an orthogonal thread (“the weft”) runs across. By selecting which threads to raise or lower, one can effectively set pixels in the resulting fabric. The programs are on punched cards, where each row of holes determines which threads to raise or lower. Cards were physically chained together to carry patterns, with a literal loop of cards giving rise to a repeating pattern in the fabric. It might be 130 more years before computers generalize and nest these “loops”; it’s 200 years before the terms are repurposed to describe GPUs. 
• In the 1820s, Charles Babbage builds the Difference Engine, the first automated engine that represents numbers. He hopes to calculate tables of mathematical functions automatically; his idea may have been crippled by the poor machining tolerances of the early Industrial Age. His Analytical Engine never works. As is perhaps typical in science and engineering, Ada Lovelace invents programming but gets a tiny fraction of the credit. 
• For the 1890 US census, Herman Hollerith invents punched cards and commissions tabulating machines. While early punch cards encode only numbers to be added by the tabulating machines, later cards recorded alphanumeric data³Look up the Hollerith Code, whose encoding is both informationally dense and works around the mechanical limits of paper, where two adjacent punches or too many punches in a column could cause a card to shred while processing. This is a great early example of HW/SW codesign. . Hollerith’s innovation spawns an industry; his company gets absorbed into what would become IBM, whose first golden age is in tabulation, not computers. Much later, punched cards become a mechanism for user input, program storage, and data storage for computers.⁴As a kid I hung around the UCLA Computer Science department and persuaded college kids to let me play Colossal Cave (Adventure) on the DEC-10. But there were still keypunch machines, and one could buy packs of blank cards from vending machines. I spent a bunch of time typing cards, but didn’t yet know how to program, so I just missed programming a computer using punched cards.  

Around WWII, a number of proto-computers are built and used for cryptography, artillery ballistics tables, and the design of nuclear weapons.⁵I find the principled stances of Computer Scientists who refuse government or military funding to be admirable, but I wonder if they know how martial the origins of our field are.  They include:

• Konrad Zuse’s Z1 mechanical computer never works but includes floating-point logic. 
• Alan Turing’s electro-mechanical bombe cracks the German Enigma machine. The cryptographic breakthroughs at the UK’s Bletchley Park and in the US Magic program speed the end of WWII, likely saving millions of lives. 
• Howard Aiken’s Harvard Mark I is an electromechanical computer; polymath genius John von Neumann uses it to calculate whether the “implosion” method used by the Fat Man nuclear bomb will work. 
• ENIAC is the first programmable, all-electronic computer. A key innovation is that the data and control memories are unified. Because ENIAC is entirely electronic, it runs 1000x faster than its electromechanical cousins. Von Neumann visits the project and grabs credit for inventing the modern computer by writing the First Draft of a Report on the EDVAC, which outlines the fundamental principles of all modern computers. But we computer architects celebrate J. Presper Eckert and John Mauchly, who built ENIAC, with our highest honor. 

In 1949, the Eckert-Mauchly computer company builds the BINAC, the first commercial computer. UNIVAC follows in 1951, with the US Census as a customer. The computer industry blooms during the 50s, but the Eckert-Mauchly computer company isn’t one of the big winners. 

That same year, EDSAC, designed by Maurice Wilkes at Cambridge, goes live. Wilkes is 36 years old; we computer architects honor Wilkes with our award for great impact early in their careers. EDSAC has 1024 locations of mercury delay-line memory⁶Presper Eckert used delay lines for analog radar decluttering before working on ENIAC. , a 17-bit word that can be chained into a 35-bit operand⁷I’ve always found EDSAC descriptions a bit inconsistent: was the 18th memory bit completely unusable? My guess is that there’s an explainable 17-bit machine inside an implementation with very rough edges. . Its initial programs are written in fixed-point arithmetic with programmers managing scaling constants in the margins of their code listings. As part of earning his PhD, David Wheeler invents subroutines; this allows a rich set of libraries, including support for division, floating-point, and complex numbers to be added. 

For a while, IBM runs the table: 

• Debuting in 1961, IBM’s 7030 Stretch aims to improve its predecessor’s performance by 100x. It’s judged a failure, reaching “only” a 30x improvement, in part because the expected compilation breakthroughs don’t materialize. It’s still the fastest computer in the world for 3 years, and the ideas from Stretch feed into System/360.
• IBM unifies its multiple computer lines into the System/360, which introduces the concept of a compatible Instruction Set Architecture, allowing multiple implementations of machines that run the same program. Gerrit Blauuw argues successfully for using an 8-bit word in the memory system. Fred Brooks manages the OS/360 operating system project, identifying both the software-engineering fallacy of The Mythical Man-Month and the pitfall of the Second System Effect. Running by 1968, the IBM System/360 Model 91 is the first out-of-order processor, implementing Tomasulo’s algorithm. Shipped in 1969, the IBM System/360 Model 85 is the first machine with a cache. Gene Amdahl, chief architect of the 360, goes on to found his own company which builds plug-compatible mainframes. You might think that a modern x86’s ability to run 8080 instructions is impressive, but IBM sells Z-series mainframes today that still run the 360 ISA. 
• IBM invents virtual machines in the 1970s. 
• Falling into the Innovator’s Dilemma pitfall, IBM misses the minicomputer boom, and Digital Equipment Corporation sells a generation of department-scale PDP-11 and VAX computers. With Raytheon and Lincoln Labs, DEC anchors the East-Coast technology hub near Boston called Route 128, which many of my younger colleagues have never heard of. 
• In the late 1970s, the 801 project led by John Cocke pioneers RISC computing. 801 shows that you can outperform a CISC machine through a stream of compiler-optimized RISC instructions. They invent the compiler technologies, including graph-coloring register allocation, at the same time. The Berkeley RISC and Stanford MIPS⁸Norm Jouppi is a summer intern with IBM’s 801 group for the summer just before he starts as John Hennessy’s student. Correlation is not causation, but it does make you wonder if there was something in the Eero Saarinen-designed 801 building’s water supply.  projects form the basis of the Sun SPARC and MIPS Technologies processors, and other RISC architectures come to market (including HP PA-RISC, IBM POWER/PowerPC, DEC Alpha, Motorola’s ill-fated 88000, and ARM). 
• IBM does well by introducing the IBM PC in 1981, choosing the Intel 8088 for its heart over the Motorola 68000 and TI TMS9900 (everyone agrees that the 68K was technically superior, but it is late to market; the 8088 is also cheaper). IBM soon loses control of the personal computer market that it legitimized to Intel and Microsoft. IBM’s attempt to reassert hardware and software control through the Micro Channel Architecture fails. 

Let me pause here, ending the first installment of this history with IBM largely in control, but with threats to their hegemony on the way. I’ll resume in Part 2 in a few days. 

About the Author: Cliff Young is a software engineer in Google Research, where he works on codesign for deep learning accelerators. He is one of the designers of Google’s Tensor Processing Unit (TPU) and one of the founders of the MLPerf benchmark. Previously, Cliff built special-purpose supercomputers for molecular dynamics at D. E. Shaw Research and was a Member of Technical Staff at Bell Labs. 

Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.

• 1
  I have other reasons for Physics envy—it’s the hardest of the “hard sciences”, one step away from the pure rigor of Mathematics. I worry that CS needs “Science” in its name to bolster its weak science-y-ness. The French term Informatics is much better, as our discipline spans math, science, and engineering. 
• 2
  For an example of how to write such a report, see Colwell’s analysis of the Intel iAPX 432. Just reading section 1.1 is like the start of a horror movie. 
• 3
  Look up the Hollerith Code, whose encoding is both informationally dense and works around the mechanical limits of paper, where two adjacent punches or too many punches in a column could cause a card to shred while processing. This is a great early example of HW/SW codesign. 
• 4
  As a kid I hung around the UCLA Computer Science department and persuaded college kids to let me play Colossal Cave (Adventure) on the DEC-10. But there were still keypunch machines, and one could buy packs of blank cards from vending machines. I spent a bunch of time typing cards, but didn’t yet know how to program, so I just missed programming a computer using punched cards. 
• 5
  I find the principled stances of Computer Scientists who refuse government or military funding to be admirable, but I wonder if they know how martial the origins of our field are. 
• 6
  Presper Eckert used delay lines for analog radar decluttering before working on ENIAC.
• 7
  I’ve always found EDSAC descriptions a bit inconsistent: was the 18th memory bit completely unusable? My guess is that there’s an explainable 17-bit machine inside an implementation with very rough edges. 
• 8
  Norm Jouppi is a summer intern with IBM’s 801 group for the summer just before he starts as John Hennessy’s student. Correlation is not causation, but it does make you wonder if there was something in the Eero Saarinen-designed 801 building’s water supply.