Managing engines

Neutree provides built-in engines, and also supports creating custom engines via the CLI tool, adding new versions to built-in engines, and deleting engine versions.

Current built-in engines:

Name	Version	Description
vllm	v0.8.5	vLLM community version v0.8.5.
	v0.11.2	vLLM community version v0.11.2.
	v0.17.1	vLLM community version v0.17.1. This is the default version for both Kubernetes clusters and static node clusters.
llama-cpp	v0.3.7	Llama-cpp Python high-level implementation (llama-cpp commit: 794fe23f29fb40104975c91fe19f23798f7c726e).

Viewing engines

Log in to the Neutree management interface. Click Engines in the left navigation pane. The engine list on the right shows all built-in engines. Click an engine name to enter the details page and view the task types and parameters supported by the engine.

Importing an engine

You can create an engine via the CLI tool, or add a new version to an existing engine.

Procedure

1. Create an API key and save it securely.

2. Download version 1.0.1 of the Neutree CLI tool and the specified engine image for your server’s CPU architecture.

3. Import the engine image using the CLI tool:

   Pull from Docker Hub

   When imageRegistry is configured as Docker Hub, you only need to import the engine image metadata — the image itself does not need to be uploaded. Run the following command to automatically pull the engine image from Docker Hub.

   ./neutree-cli-<arch> import engine --skip-image-push \
   --package <engine_version_package> \
   --api-key <api_key> \
   --server-url <server_url>

   Parameter	Description
   <arch>	Replace with the server’s CPU architecture: amd64 or aarch64.
   <engine_version_package>	The engine image package name, for example vllm-v0.8.5.tar.gz.
   <api_key>	The API key created in step 1.
   <server_url>	The address of the control plane, for example http://localhost:3000.

   Pull from remote registry

   ./neutree-cli-<arch> import engine [--skip-image-push] \
   --package <engine_version_package> \
   --mirror-registry <mirror_registry> \
   [--registry-project <registry_project>] \
   --registry-username <registry_username> \
   --registry-password <registry_password> \
   --api-key <api_key> \
   --server-url <server_url>

   Parameter	Description
   [--skip-image-push]	Optional. Use this parameter if the image already exists in the target registry and you only need to upload the version metadata to the platform.
   <arch>	Replace with the server’s CPU architecture: amd64 or aarch64.
   <engine_version_package>	The engine image package name, for example vllm-v0.8.5.tar.gz.
   <mirror_registry>	The registry address. Must match the registry address used when uploading images with the CLI tool during Neutree management plane deployment. Enter an OCI-compatible registry address without the https:// prefix.
   [--registry-project <registry_project>]	Optional. The project name in the registry. The project must be created in advance. For example, specifying --registry-project neutree-ai will push the image to <mirror_registry>/neutree-ai/vllm/vllm-openai:v0.XX.XX.
   <registry_username>	The registry username. Must have permission to push images.
   <registry_password>	The registry user’s login password or access token (for example, a token).
   <api_key>	The API key created in step 1.
   <server_url>	The address of the control plane, for example http://localhost:3000.

   Import via standalone manifest file

   If the engine image already exists in Docker Hub or a remote registry, you can use a standalone manifest.yaml file to import only the engine image metadata without downloading the full engine image.

   ./neutree-cli-<arch> import engine --skip-image-push \
   --package <manifest.yaml> \
   --api-key <api_key> \
   --server-url <server_url>

   Parameter	Description
   <arch>	Replace with the server’s CPU architecture: amd64 or aarch64.
   <manifest.yaml>	The path to the manifest.yaml file, which contains the engine image metadata.
   <api_key>	The API key created in step 1.
   <server_url>	The address of the control plane, for example http://localhost:3000.

4. After importing, log in to the Neutree management interface. Click Engines in the left navigation pane and confirm that the new engine or engine version appears in the engine list.

Deleting an engine version

You can delete engine versions using the CLI tool. When only one version remains, use the --force parameter to force-delete that version and remove the entire engine at the same time.

Prerequisites

Confirm that the engine version is not currently in use by any endpoint.

Procedure

Run the following command:

./neutree-cli-<arch> engine remove-version \
  --name <engine_name> \
  --version <engine_version> \
  --api-key <api_key> \
  --server-url <server_url>

Parameter descriptions:

Parameter	Description
<arch>	Replace with the server’s CPU architecture: amd64 or aarch64.
<engine_name>	The name of the engine, for example vllm.
<engine_version>	The version to delete, for example v0.8.5.
<api_key>	The API key created in the prerequisites.
<server_url>	The address of the control plane, for example http://localhost:3000.
--force	Optional. Required when only one version remains — force-deletes that version and removes the entire engine.

Note

If you delete a built-in engine, the system automatically rebuilds it.