Installing NVIDIA Container Toolkit offline

Issue

If your cluster nodes cannot access the internet, you cannot install NVIDIA Container Toolkit online using a package manager. Even if the NVIDIA graphics driver is already installed on the nodes, Docker will still be unable to use GPU resources.

Solution

Download the NVIDIA Container Toolkit offline installation package from the NVIDIA GitHub Releases page, transfer it to the target node, then manually install it and configure the Docker runtime.

Prerequisites

Confirm that the target node meets the following requirements:

• NVIDIA graphics driver is installed.
• Docker is installed.

Step 1: Download the offline installation package

1. In an environment with internet access, visit the libnvidia-container Releases page.

2. Download the corresponding installation package based on the operating system and CPU architecture of the target node:

   CentOS/RHEL/Rocky Linux

   Download the following RPM packages:

   • libnvidia-container1-<version>.rpm
   • libnvidia-container-tools-<version>.rpm
   • nvidia-container-toolkit-base-<version>.rpm
   • nvidia-container-toolkit-<version>.rpm
   Ubuntu/Debian

   Download the following DEB packages:

   • libnvidia-container1_<version>.deb
   • libnvidia-container-tools_<version>.deb
   • nvidia-container-toolkit-base_<version>.deb
   • nvidia-container-toolkit_<version>.deb

3. Transfer the downloaded installation packages to the target node.

Step 2: Install NVIDIA Container Toolkit

Run the installation command on the target node:

CentOS/RHEL/Rocky Linux

sudo rpm -ivh libnvidia-container1-*.rpm \
  libnvidia-container-tools-*.rpm \
  nvidia-container-toolkit-base-*.rpm \
  nvidia-container-toolkit-*.rpm

Ubuntu/Debian

sudo dpkg -i libnvidia-container1_*.deb \
  libnvidia-container-tools_*.deb \
  nvidia-container-toolkit-base_*.deb \
  nvidia-container-toolkit_*.deb

Step 3: Configure the Docker runtime

1. Run the following command to configure NVIDIA Container Runtime as the default Docker runtime:

   sudo nvidia-ctk runtime configure --runtime=docker

2. Restart the Docker service for the configuration to take effect:

   sudo systemctl restart docker

Step 4: Verify the installation

Run the following command to verify that NVIDIA Container Runtime has been registered with Docker:

docker info | grep -i nvidia

If the output contains runtime information related to nvidia, the installation and configuration are successful.