The NVIDIA CUDA Deep Neural Network library (cuDNN) is a
GPU-accelerated library of primitives for deep neural networks. cuDNN
provides highly tuned implementations for standard routines such
as forward and backward convolution, pooling, normalization, and
activation layers. cuDNN is part of the NVIDIA Deep Learning SDK.

You will need to register for NVIDIA developer account to download the
source.

Make sure you create the CUDA_HOME environment variable and add it to
your ~/.bashrc. The default should be

CUDA_HOME=/usr/share/cuda