Models and Provider Backends

Use moi-cli model and moi-cli provider-backend to query visible routing models and workspace-declared models, inspect or configure llm, embedding, and parser provider backends, manage endpoints and router policies, and probe upstream model lists. Complete Configuration and Authentication before running commands; omit --workspace-id when a default workspace is already saved in your local configuration.

Distinguish routing model lists and workspace backend models

model provides two distinct sets of model listing commands with different visibility scopes:

# 1. List visible routing models (workspace backend declarations plus permitted system models)
moi-cli -auth pat -o model list --workspace-id "$WORKSPACE_ID"
moi-cli -auth pat -o model list --workspace-id "$WORKSPACE_ID" --query-param type=embedding
moi-cli -auth pat -o model list-embedding-models --workspace-id "$WORKSPACE_ID"

# 2. List only LLM models declared by readable backends in the current workspace
moi-cli -auth pat -o model list-workspace-llm-models --workspace-id "$WORKSPACE_ID"

Subcommand

Underlying API path

Returned scope and purpose

model list

/models

Lists visible routing models for the workspace (accepts --query-param type=embedding and other filters), combining workspace-configured backends with permitted system routing models.

model list-embedding-models

/embedding/models

Lists embedding routing models available to the workspace.

model list-workspace-llm-models

/workspaces/{workspace_id}/llm/models

Lists only model names and backend metadata declared by readable LLM backends in that workspace, excluding system routing models not declared on a workspace backend; see List available models for the HTTP contract.

Seeing a model in these lists confirms configuration visibility only and does not test model capabilities or live upstream connectivity. To run a direct workspace chat completion check from the CLI, use model workspace-chat-completion or model workspace-chat-completion-stream --output (inspect request fields via moi-cli schema model workspace-chat-completion).

Inspect and manage backends, endpoints, and routers

provider-backend selects the backend category via --kind (llm by default, embedding, or parser). Every subcommand operates within the selected workspace (the underlying /llm/backends route is workspace-scoped rather than a cluster-wide admin endpoint):

# List workspace backends, read one backend, list endpoints, and inspect router settings
moi-cli -auth pat -o provider-backend list --workspace-id "$WORKSPACE_ID" --kind llm
moi-cli -auth pat -o provider-backend get --workspace-id "$WORKSPACE_ID" --kind llm --id "$BACKEND_ID"
moi-cli -auth pat -o provider-backend list-endpoints --workspace-id "$WORKSPACE_ID" --kind llm --backend-id "$BACKEND_ID"
moi-cli -auth pat -o provider-backend router --workspace-id "$WORKSPACE_ID" --kind llm

Add an upstream endpoint to a backend or change an endpoint’s availability status:

# Create an upstream endpoint and record its returned id
moi-cli -auth pat -o provider-backend create-endpoint \
  --workspace-id "$WORKSPACE_ID" \
  --kind llm \
  --backend-id "$BACKEND_ID" \
  --address 'https://llm-upstream.example.com/v1'

# Explicitly set endpoint status to ONLINE or OFFLINE
moi-cli -auth pat -o provider-backend set-endpoint-status \
  --workspace-id "$WORKSPACE_ID" \
  --kind llm \
  --backend-id "$BACKEND_ID" \
  --endpoint-id "$ENDPOINT_ID" \
  --status OFFLINE

Warning

Review endpoint visibility and status normalization rules before changing endpoint status:

  • For llm and parser, list-endpoints and get return ONLINE endpoints only—save ENDPOINT_ID when creating the endpoint before setting it OFFLINE, because an empty read result does not mean the endpoint was deleted. embedding endpoint reads return both ONLINE and OFFLINE endpoints.

  • set-endpoint-status requires an explicit ONLINE or OFFLINE string: omitting or passing an empty --status fails client-side validation, while unrecognized non-empty values fall back to ONLINE on the HTTP API (parser trims surrounding whitespace and uppercases first). Read back after updating to confirm.

Configure backends, update routers, and probe upstream models

In a deployment-matched moi-cli build, store the upstream provider API key in a restricted local file (such as chmod 600 provider.key) and configure backends, routers, or model probes through flags or JSON subcommands:

# Create an LLM backend (global -api-key authenticates MOI requests only; pass upstream key via --api-key-file)
moi-cli -auth pat -o provider-backend create \
  --workspace-id "$WORKSPACE_ID" \
  --kind llm \
  --name 'openai-compatible-prod' \
  --type OPENAI \
  --address 'https://llm-upstream.example.com/v1' \
  --models 'qwen3.6-plus,deepseek-v3' \
  --api-key-file ./provider.key

# Update router strategy and health-check parameters
moi-cli -auth pat -o provider-backend update-router \
  --workspace-id "$WORKSPACE_ID" \
  --kind llm \
  --strategy round_robin \
  --health-check-interval 30 \
  --max-retries 2 \
  --session-affinity=false

# Probe models exposed by an upstream address (queries the provider without saving or enabling a backend)
moi-cli -auth pat -o model probe-llm-models \
  --workspace-id "$WORKSPACE_ID" \
  --address 'https://llm-upstream.example.com/v1' \
  --api-key-file ./provider.key

Configuration and probing options across subcommands:

Subcommand

Key flags

Description

provider-backend create / model setup-llm-backend

--name, --type, --address, --models (comma-separated), --api-key-file, --timeout-seconds, --reasoning-control-protocol, --supported-mime-types

Create a backend and declare served models; --supported-mime-types applies to parser backends.

provider-backend update

--id, --name, --models, --api-key-file, --timeout-seconds, --reasoning-control-protocol, --supported-mime-types

Update explicitly supplied backend fields (passing an empty file to --api-key-file sends an explicit empty key).

provider-backend update-router

--kind, --strategy, --health-check-interval, --max-retries, --session-affinity

Update router selection and health-check settings for the specified backend kind.

provider-backend probe-backend-models, model probe-llm-models, model probe-embedding-models

--address (or --id), --api-key-file

Fetch the model list directly from the upstream provider; parser does not support probe-backend-models.

model *-workspace-llm-backend, model *-workspace-embedding-backend, model update-workspace-*-router

--json / --json-file

Submit typed JSON payloads directly (inspect schemas via moi-cli schema model <subcommand>).

Warning

Run moi-cli provider-backend <subcommand> --help to check the flags supported by your installed binary before running write or probe commands. In earlier 0.1.0 offline packages, provider-backend create, update, update-router, probe-backend-models, and model setup-llm-backend omit --api-key-file, --models, --type, and router flags: running provider-backend update on that older binary sends {} without changing settings, and running provider-backend update-router sends {} which resets router strategy, health-check interval, max retries, and session affinity to server defaults. If your binary lacks these flags, obtain a deployment-matched installer from your deployment administrator, or use the --json-file subcommands or Product HTTP API.

Parser backend defaults and multi-key pool rules

When managing --kind parser backends or backends configured with multi-key credential pools, note the following server defaults and client adaptations:

moi-cli -auth pat -o provider-backend list --workspace-id "$WORKSPACE_ID" --kind parser
moi-cli -auth pat -o provider-backend list-endpoints --workspace-id "$WORKSPACE_ID" --kind parser --backend-id "$BACKEND_ID"
  • Default type and timeout ceiling: When a parser backend is created without an explicit type or timeout, the server defaults to PARSER_MINERU and a 3600-second timeout (up to a maximum of 86400 seconds).

  • Client response adaptation: Parser HTTP list endpoints return a raw data array (null when empty), which the CLI normalizes into backends or endpoints collections; delete and set-endpoint-status return HTTP data: null, which the CLI reports as success: true.

  • Multi-key pool (api_keys_encrypted) rules: When a backend already uses a multi-key pool, updating only the single-key api_key field (such as via --api-key-file) is rejected by the server; --api-key-file manages single-key credentials only and does not replace a full key pool. Submitting an explicit empty array api_keys_encrypted: [] through the underlying API switches back to single-key mode while retaining the prior single key or first pool entry unless a new non-empty api_key is provided at the same time.

Limitations and notes

  • The global -api-key option authenticates MOI Product API requests only and never populates the upstream provider’s api_key.

  • Saving backend or router configurations, listing ONLINE endpoints, or probing upstream model lists does not verify live inference or document parsing; run an end-to-end workload check after configuration changes.

Last updated on