ASPLOS 2008 CUDA Tutorial

CUDA: A Heterogeneous Parallel Programming Model for Manycore Computing

Abstract

Modern GPUs provide a level of massively parallel computation that was once the preserve of supercomputers like the MasPar and Connection Machine. NVIDIA’s Tesla architecture for GPU Computing provides a fully programmable, massively multithreaded chip with up to 128 scalar processor cores and capable of delivering hundreds of billions of operations per second. Researchers across many scientific and engineering disciplines are using this platform to accelerate important computations by up to 2 orders of magnitude.

In this tutorial, we will provide an overview of the Tesla architecture and explore the transition it represents in massively parallel computing: from the domain of supercomputers to that of commodity “manycore” hardware available to all. We will also introduce CUDA, a scalable parallel programming model and software environment for parallel programming. By providing a small set of readily understood extensions to the C/C++ languages, CUDA allows programmers to focus on writing efficient parallel algorithms without the burden of learning a multitude of new programming constructs. Finally, as the GPU is the only widely available commodity “manycore” chip available today, we will explore its importance as a research platform for exploring important issues in parallel programming and architecture.

Course Notes

Welcome to the course notes for the half-day ASPLOS 2008 CUDA Tutorial!

The tutorial was held at ASPLOS 2008 on Sunday, March 2, 2008.

Course Organizer

David Luebke, NVIDIA Corporation

Course Speakers

Tutorial Outline

  • Introduction/Overview (“GPU Computing: The Democratization of Parallel Computing”) (Luebke) (PDF)
  • Trends in Multicore Architecture (Skadron) (PDF)
  • Parallel Computing in CUDA (programming model and language) (Garland) (PDF)
  • GPU Architecture & Applications (Luebke) (PDF)
  • Advanced Data-Parallel Algorithms Programming: Data Structures and Algorithms (Owens) (PDF)
  • Wrapup and Open Issues (Skadron) (PDF)