Managing model usage

Model usage helps you analyze call trends by API key, model, and type, and view prompt, completion, and total token usage. Model usage statistics are stored in the PostgreSQL database of the Neutree control plane. The model gateway extracts usage information from model call responses and continuously aggregates and displays it as daily usage statistics visible to users. Aggregated usage statistics are continuously retained in the PostgreSQL database until the related API key is deleted or database data is cleaned up.

Viewing model usage

Raw usage records written by the model gateway are periodically aggregated into daily usage data and displayed on the page.

By default, you can view model usage generated by API keys that you created. To view aggregated usage for all API keys in a workspace, make sure that you have the Workspaces:Usage Read permission. When All workspaces is selected, the page aggregates visible usage data of the current user across accessible workspaces.

Procedure

Log in to the Neutree management interface. In the left navigation pane, click Model Usage.
Select a time range. The page displays data from the last 30 days by default. You can also switch to Last 7 days or Last 90 days.
Set filter conditions as needed.
- API key: Select All API keys or a specific API key. After you select a single API key, the trend chart displays usage by model.
- Type: Select All types, Internal, or External.
- Model: Enter a model name to filter usage data by model name.
View the trend chart.
- If no API key is specified, the trend chart displays daily token usage by API key.
- After an API key is specified, the trend chart displays daily token usage by model and automatically switches to Bar chart.
- You can switch between Line chart and Bar chart.
- Click a day in the trend chart to narrow Daily detail to that date. To cancel date filtering, click the close button of the date filter item.
View summary information.

The page provides two summary sections, By API key and By model, which display Prompt, Completion, and Total token usage within the current filter range.
View Daily detail.

Daily detail displays usage records in descending order by date, including Date, API key, Type, Endpoint, Model, Prompt, Completion, and Total.

Exporting model usage

To archive or analyze model usage offline, you can use the Neutree CLI to export model usage data.

Preparation

Create an API key and store it securely.

Procedure

Run the following command:

neutree-cli-<arch> export model-usage \
  -w [workspace] \
  --format [csv|json|jsonl] \
  --file [model-usage.csv] \
  --since [YYYY-MM-DD] \
  --until [YYYY-MM-DD] \
  --api-key-id [api_key_id] \
  --server-url <server_url> \
  --api-key <api_key>

The parameters are described as follows:

Parameter	Description
`[workspace]`	Workspace name. If this optional parameter is left blank, the `default` workspace is used.
`--all-workspaces` or `-A`	Optional parameter. Exports model usage data from all workspaces that the current API key has permission to read. This parameter cannot be used together with `-w` or `--workspace`.
`[csv\|json\|jsonl]`	Export format. This optional parameter can be set to `csv`, `json`, or `jsonl`. If not specified, the default format is `csv`.
`[model-usage.csv]`	Export file path. If this optional parameter is left blank, the exported content is printed to standard output.
`[YYYY-MM-DD]`	Export date range. `--since` indicates the start date, and `--until` indicates the end date. If not specified, data from the last 30 days is exported by default.
`[api_key_id]`	API key ID. This optional parameter filters model usage by API key.
`<api_key>`	The API key created during preparation. This API key must have the model access viewing permission.
`<server_url>`	Control plane access address, for example, `http://localhost:3000`.

You can also filter model usage by endpoint name, endpoint type, and model name:

neutree-cli-<arch> export model-usage \
  --all-workspaces \
  --endpoint-type external-endpoint \
  --model e2e-model \
  --format jsonl \
  --server-url <server_url> \
  --api-key <api_key>