Installing NVIDIA Container Toolkit offline

Issue

If your cluster nodes cannot access the internet, you cannot install NVIDIA Container Toolkit online using a package manager. Even if the NVIDIA graphics driver is already installed on the nodes, Docker will still be unable to use GPU resources.

Solution

Download the NVIDIA Container Toolkit offline installation package from the NVIDIA GitHub Releases page, transfer it to the target node, then manually install it and configure the Docker runtime.

Prerequisites

Confirm that the target node meets the following requirements:

NVIDIA graphics driver is installed.
Docker is installed.

Step 1: Download the offline installation package

In an environment with internet access, visit the libnvidia-container Releases page.
Download the corresponding installation package based on the operating system and CPU architecture of the target node:
@tab CentOS/RHEL/Rocky Linux
Download the following RPM packages:
- libnvidia-container1-<version>.rpm
- libnvidia-container-tools-<version>.rpm
- nvidia-container-toolkit-base-<version>.rpm
- nvidia-container-toolkit-<version>.rpm
@tab Ubuntu/Debian
Download the following DEB packages:
- libnvidia-container1_<version>.deb
- libnvidia-container-tools_<version>.deb
- nvidia-container-toolkit-base_<version>.deb
- nvidia-container-toolkit_<version>.deb
Transfer the downloaded installation packages to the target node.

Step 2: Install NVIDIA Container Toolkit

Run the installation command on the target node:

@tab CentOS/RHEL/Rocky Linux

sudo rpm -ivh libnvidia-container1-*.rpm \
  libnvidia-container-tools-*.rpm \
  nvidia-container-toolkit-base-*.rpm \
  nvidia-container-toolkit-*.rpm

@tab Ubuntu/Debian

sudo dpkg -i libnvidia-container1_*.deb \
  libnvidia-container-tools_*.deb \
  nvidia-container-toolkit-base_*.deb \
  nvidia-container-toolkit_*.deb

Step 3: Configure the Docker runtime

Run the following command to configure NVIDIA Container Runtime as the default Docker runtime:
Terminal window
```
sudo nvidia-ctk runtime configure --runtime=docker
```
Restart the Docker service for the configuration to take effect:
Terminal window
```
sudo systemctl restart docker
```

Step 4: Verify the installation

Run the following command to verify that NVIDIA Container Runtime has been registered with Docker:

docker info | grep -i nvidia

If the output contains runtime information related to nvidia, the installation and configuration are successful.