On July 22-23rd, UC San Diego’s Non-Volatile Systems Laboratory and the Storage Networking Industry Association (SNIA) hosted the first Persistent Programming In Real Life (PIRL). PIRL is a new meeting devoted to gathering and sharing real-world, practical expertise in using persistent memory (e.g., NVDIMMs and Intel’s new 3DXPoint DIMMs). The event took place a few yards from the beach at Scripps Forum in La Jolla, California.
The meeting featured five keynotes and seventeen presentations. Nearly all the presentations from PIRL are online in both video and slide form. You can download/watch them at the PIRL website or the PIRL YouTube channel.
I helped organize the event, and below, I recount a few highlights from PIRL that provide a look at persistent programming past, present, and future.
Linux Development: A View From Within
Intel’s Dan Williams (who leads Intel’s persistent memory software efforts for Linux) provided a glimpse into the inner workings of Linux kernel development. He told the tale of getting support for DAX-style mmap() into ext4 and xfs. It’s a story filled with interesting characters, good engineering, and…ponies.
The History of PMDK
Intel’s Persistent Memory Development Kit (PMDK) is (I think) the oldest and most widely-used library for programming with persistent memory. Piotr Balcer is one of the lead developers of PMDK, and he gave us a brief history of PMDK’s evolution.
Piotr starting by discussing the first C-based version (circa 2014) of PMDK inspired by the NV-Heaps and Mnemosyne projects from UCSD and the University of Wisconsin, Madison, respectively. Then, he traced the maturation of PMDK for C into version 1.0 in 2016 and, then, the devolvement of a version for C++ (circa 2016) that leveraged its type system to improve programmability.
He also described his work implementing an easy-to-use key-value library called pmemkv that let the programmer avoid the complexity of low-level manipulation of persistent memory. The talk was great, but Piotr didn’t get through all his slides. Fortunately, the whole deck is online.
Using Persistent Memory in Animation Production
Scott Miller from Dreamworks Animation described his vision for how persistent memory could help improve the workflow that Dreamworks uses to produce feature-length animated films (e.g., “How To Train Your Dragon”). Dreamworks runs a remarkably data-intensive enterprise working on as many as 20 animated movies, television shows, and theme park attractions at any time.
According to Miller, a single film generates about 750 TB of data across 500M files and consumes 150 M core-hours on server farm/network-of-workstations containing 80,000 cores. Overall, their storage infrastructure hosts 15 billion files totaling 20PB. They do about 400k IOPs 24-7.
Scott and his team are exploring how persistent memory could improve productivity reducing the need to serialize/deserialize large data sets as animation components flow between animator workstations.
Compute Express Link
Stephen Bates from Eideticom gave an electrifying introduction to Computer Express Link (CLX), a (sort of) new competitor for the memory and IO interconnect fabric in next-generation computer systems. CLX will join the fray along with Gen-Z, CCIX, and OpenCAPI to decide how processors, memory, and co-processors will communicate.
Stephen pointed out a few key features of CLX that make it particularly attractive:
- It is electrically and mechanically compatible with PCIe 5.0, allowing CLX devices to leverage existing chassis and form factors. This will also, in principle, allow the system to decide at boot time whether expansion slots should be PCIe or CLX.
- CLX support three modes of operation: CLX.io — which carries PCIe packets, allowing legacy PCIe hardware and software to work over CLX; CLX.cache — which allows co-processors on a CLX device to take cache-coherent ownership of memory regions; and CLX.mem which allows a CLX device to expose on-board memory as part of the CPU’s physical address space. A single device can implement any subset of these modes.
- The CLX standards body includes AMD, Arm, and Intel, suggesting that it will have deeper market penetration than any of the competing protocols.
From a persistent memory perspective, CLX.mem is the most interesting, since it would allow the creation of memory modules that are physically larger and consume more power than DRAM DIMMs while still providing memory-like access to their contents.
It’s easy to see how existing peripherals like SSDs, NICs, and GPUs fit into such a model, but it will also enable a range of new devices, the likes of which we have not seen before.
PIRL 2020
Planning is already underway for PIRL 2020, and we have some other plans in store as well, including a programming contest and a blog devoted to real-world experience with persistent memory (Contributions welcome, contact swanson@cs.ucsd.edu). We’ll also be introducing some new content throughout the year. Follow @pirlconf on twitter (or watch the web site and blog) for details.
About the Author: Steven Swanson is a professor of Computer Science and Engineering at the University of California, San Diego and director of the Non-Volatile Systems Laboratory.
Disclaimer: These posts are written by individual contributors to share their thoughts on the Computer Architecture Today blog for the benefit of the community. Any views or opinions represented in this blog are personal, belong solely to the blog author and do not represent those of ACM SIGARCH or its parent organization, ACM.
