Configuring the startup timeout for Kubernetes endpoints

Issue

When you create an endpoint in a Kubernetes cluster, the endpoint may fail to start if model loading or service startup takes longer than the default wait time.

Solution

The Neutree management interface does not currently support configuring the endpoint startup timeout. To extend the startup wait time for Kubernetes endpoints, configure spec.deployment_options.startup_timeout_seconds through YAML or the API.

The default value of startup_timeout_seconds is 1200, in seconds. This value must be a positive integer.

The following example uses the CLI to update the startup timeout for an existing endpoint:

Export the existing endpoint configuration:

neutree-cli-<arch> get Endpoint <endpoint-name> \
  -w <workspace-name> \
  -o yaml \
  --api-key <api_key> \
  --server-url <server_url> \
  > endpoint.yaml

Edit endpoint.yaml, add or modify startup_timeout_seconds in spec.deployment_options, and retain the other existing endpoint configurations:
```
spec:
  deployment_options:
    startup_timeout_seconds: 1800
```

Apply the updated endpoint configuration:

neutree-cli-<arch> apply -f endpoint.yaml \
  --api-key <api_key> \
  --server-url <server_url> \
  --force-update

Neutree redeploys the endpoint with the new startup timeout.