High performance CUTLASS template abstractions support matrix multiply operations (GEMM), Convolution AI, and improved Strided-DGrad.
NVIDIA continues to enhance CUTLASS to provide extensive support for mixed-precision computations, providing specialized data-movement, and multiply-accumulate abstractions. Today, NVIDIA is announcing the availability of CUTLASS version 2.8.
Download the free CUTLASS v2.8 software.
What’s new
- Emulated single-precision GEMM and Convolution (up to 48TFLOPs)
- Grouped GEMM concept
- Improved Strided-DGrad
See the CUTLASS Release Notes for more information.
About CUTLASS
CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-multiplication (GEMM) at all levels, and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS.
CUTLASS decomposes these “moving parts” into reusable and modular software components abstracted by C++ template classes. These thread-wide, warp-wide, block-wide, and device-wide primitives can be specialized and tuned via custom tiling sizes, data types, and other algorithmic policy. The resulting flexibility simplifies their use as building blocks within custom kernels and applications.
To support a wide variety of applications, CUTLASS provides extensive support for mixed-precision computations, providing specialized data-movement, and multiply-accumulate abstractions for:
- Half-precision floating point (
FP16), BFloat16 (BF16), and Tensor Float 32 (TF32) data types. - Single-precision floating point (
FP32) data type. - Double-precision floating point (
FP64) data type. - Integer data types (
4band8b). - Binary data types (
1b).
Furthermore, CUTLASS demonstrates warp-synchronous matrix multiply operations targeting the programmable, high-throughput Tensor Cores implemented on NVIDIA Volta, Turing, and Ampere architectures.
CUTLASS implements high-performance convolution (implicit GEMM). Implicit GEMM is the formulation of a convolution operation as a GEMM. This allows CUTLASS to build convolutions by reusing highly optimized warp-wide GEMM components and below.
Learn more
- A Deep Dive into the Latest HPC Software (GTC: A31050)
- Accelerating Convolution with Tensor Cores in CUTLASS (GTC: S31883)
- Recent Developments in NVIDIA Math Libraries (GTC: S31754)
- A Deep Dive into the Latest HPC Software (GTC: S31286)
- Tensor Core-Accelerated Math Libraries for Dense and Sparse Linear Algebra in AI and HPC (GTC: CWES1098)
- CUTLASS Product Documentation