SANTA CLARA, CA — (Marketwire) — 02/28/2011 – NVIDIA today announced the latest version of the NVIDIA® CUDA® Toolkit for developing parallel applications using NVIDIA GPUs.

The NVIDIA CUDA 4.0 Toolkit was designed to make parallel programming easier, and enable more developers to port their applications to GPUs. This has resulted in three main features:

• NVIDIA GPUDirect™ 2.0 Technology – Offers support for peer-to-peer communication among GPUs within a single server or workstation. This enables easier and faster multi-GPU programming and application performance.
• Unified Virtual Addressing (UVA) – Provides a single merged-memory address space for the main system memory and the GPU memories, enabling quicker and easier parallel programming.
• Thrust C++ Template Performance Primitives Libraries – Provides a collection of powerful open source C++ parallel algorithms and data structures that ease programming for C++ developers. With Thrust, routines such as parallel sorting are 5X to 100X faster than with Standard Template Library (STL) and Threading Building Blocks (TBB).

“Unified virtual addressing and faster GPU-to-GPU communication makes it easier for developers to take advantage of the parallel computing capability of GPUs,” said John Stone, senior research programmer, University of Illinois, Urbana-Champaign.

“Having access to GPU computing through the standard template interface greatly increases productivity for a wide range of tasks, from simple cashflow generation to complex computations with Libor market models, variable annuities or CVA adjustments,” said Peter Decrem, director of Rates Products at Quantifi. ”The Thrust C++ library has lowered the barrier of entry significantly by taking care of low-level functionality like memory access and allocation, allowing the financial engineer to focus on algorithm development in a GPU-enhanced environment.”

The CUDA 4.0 architecture release includes a number of other key features and capabilities, including:

• MPI Integration with CUDA Applications – Modified MPI implementations automatically move data from and to the GPU memory over Infiniband when an application does an MPI send or receive call.
• Multi-thread Sharing of GPUs – Multiple CPU host threads can share contexts on a single GPU, making it easier to share a single GPU by multi-threaded applications.
• Multi-GPU Sharing by Single CPU Thread – A single CPU host thread can access all GPUs in a system. Developers can easily coordinate work across multiple GPUs for tasks such as “halo” exchange in applications.
• New NPP Image and Computer Vision Library – A rich set of image transformation operations that enable rapid development of imaging and computer vision applications.
• New and Improved Capabilities
  • Auto performance analysis in the Visual Profiler
  • New features in cuda-gdb and added support for MacOS
  • Added support for C++ features like new/delete and virtual functions
  • New GPU binary disassembler

A release candidate of CUDA Toolkit 4.0 will be available free of charge beginning March 4, 2011, by enrolling in the CUDA Registered Developer Program at: www.nvidia.com/paralleldeveloper. The CUDA Registered Developer Program provides a wealth of tools, resources, and information for parallel application developers to maximize the potential of CUDA.

For more information on the features and capabilities of the CUDA Toolkit and on GPGPU applications, please visit:www.nvidia.com/cuda.

Version 1.3 adds several new features, including:

• a state-of-the-art sorting implementation, recently featured on Slashdot.
• performance improvements to stream compaction and reduction
• robust error reporting and failure detection
• support for CUDA 3.2 and gf104-based GPUs
• search algorithms
• and more!

Get started with Thrust today! First download Thrust v1.3 and then follow the online quick-start guide. Refer to the online documentation for a complete list of features. Many concrete examples and a set of introductory slides are also available.

Thrust is open-source software distributed under the OSI-approved Apache License v2.0.

Acknowledgments
• Thanks to Duane Merrill for contributing a fast radix sort implementation
• Thanks to Erich Elsen for contributing an implementation of find_if
• Thanks to Andrew Corrigan for contributing changes which enable OpenMP in the absence of nvcc
• Thanks to Andrew Corrigan, Cliff Woolley, David Coeurjolly, Janick Martinez Esturo, John Bowers, Maxim Naumov, Michael Garland, and Ryuta Suzuki for bug reports
• Thanks to Cliff Woolley for help with testing

We have previously suggested mixed precision iterative solvers specifically tailored to the iterative solution of sparse linear equation systems as they typically arise in the finite element discretization of partial differential equations. These schemes have been evaluated for a number of hardware platforms, in particular single precision GPUs as accelerators to the general purpose CPU. This paper reevaluates the situation with new mixed precision solvers that run entirely on the GPU: We demonstrate that mixed precision schemes constitute a significant performance gain over native double precision. Moreover, we present a new implementation of cyclic reduction for the parallel solution of tridiagonal systems and employ this scheme as a line relaxation smoother in our GPU-based multigrid solver. With an alternating direction implicit variant of this advanced smoother we can extend the applicability of the GPU multigrid solvers to very ill-conditioned systems arising from the discretization on anisotropic meshes, that previously had to be solved on the CPU. The resulting mixed precision schemes are always faster than double precision alone, and outperform tuned CPU solvers consistently by almost an order of magnitude.

(Dominik Göddeke and Robert Strzodka: “Cyclic Reduction Tridiagonal Solvers on GPUs Applied to Mixed Precision Multigrid” , accepted in: IEEE Transactions on Parallel and Distributed Systems, Special Issue: High Performance Computing with Accelerators, Mar. 2010. Link.)

• fancy iterators
• binary search algorithms
• pair and tuple types
• segmented scan (experimental)
• pinned memory support (experimental)
• and more!

To get started with Thrust, first download Thrust and then follow the online tutorial.  Refer to the online documentation for a complete list of features.  Many concrete examples and a set of introductory slides are also available. As the following code example shows, Thrust programs are concise and readable.

#include <thrust/host_vector.h>
#include <thrust/device_vector.h>
#include <thrust/generate.h>
#include <thrust/sort.h>
#include <cstdlib>

int main(void)
{
    // generate twenty random numbers on the host
    thrust::host_vector<int> h_vec(20);
    thrust::generate(h_vec.begin(), h_vec.end(), rand);

    // transfer data to the device
    thrust::device_vector<int> d_vec = h_vec;

    // sort data on the device
    thrust::sort(d_vec.begin(), d_vec.end());

    return 0;
}

Abstract:

Scan and segmented scan algorithms are crucial building blocks for a great many data-parallel algorithms. Segmented scan and related primitives also provide the necessary support for the flattening transform, which allows for nested data-parallel programs to be compiled into flat data-parallel languages. In this paper, we describe the design of efficient scan and segmented scan parallel primitives in CUDA for execution on GPUs. Our algorithms are designed using a divide-and-conquer approach that builds all scan primitives on top of a set of primitive intra-warp scan routines. We demonstrate that this design methodology results in routines that are simple, highly efficient, and free of irregular access patterns that lead to memory bank conflicts. These algorithms form the basis for current and upcoming releases of the widely used CUDPP library.

(S. Sengupta, M. Harris, and M. Garland. Efficient parallel scan algorithms for GPUs. NVIDIA Technical Report NVR-2008-003, December 2008)

General purpose programming on graphics processing units (GPGPU) has received a lot of attention in the parallel computing community as it promises to offer the highest performance per dollar. While GPUs are usually used to tackle regular problems that can be easily parallelized, we describe two implementations of List Ranking—a traditional irregular algorithm that is difficult to parallelize on such massively multi-threaded hardware. In our best implementation, we introduce a GPU-optimized, recursive version of the Helman-JaJa algorithm. Our implementation can rank a random list of 8 million elements in just over 100 milliseconds, and achieves a speedup of about 8-9 over a CPU implementation as well as a speedup of 3-4 over the best reported implementation on the Cell Broadband Engine. We also discuss some practical issues that come to the fore when working with massively multi-threaded architectures, especially for algorithms with highly irregular memory access patterns. (M. Suhail Rehman, K. Kothapalli, P.J. Narayanan. Fast and Scalable List Ranking on the GPU. 23rd International Conference on Supercomputing (ICS). New York, USA, June 2009. (To Appear))