Models and Provider Backends¶
Use moi-cli model and moi-cli provider-backend to query visible routing models and workspace-declared models, inspect or configure llm, embedding, and parser provider backends, manage endpoints and router policies, and probe upstream model lists. Complete Configuration and Authentication before running commands; omit --workspace-id when a default workspace is already saved in your local configuration.
Distinguish routing model lists and workspace backend models¶
model provides two distinct sets of model listing commands with different visibility scopes:
# 1. List visible routing models (workspace backend declarations plus permitted system models)
moi-cli -auth pat -o model list --workspace-id "$WORKSPACE_ID"
moi-cli -auth pat -o model list --workspace-id "$WORKSPACE_ID" --query-param type=embedding
moi-cli -auth pat -o model list-embedding-models --workspace-id "$WORKSPACE_ID"
# 2. List only LLM models declared by readable backends in the current workspace
moi-cli -auth pat -o model list-workspace-llm-models --workspace-id "$WORKSPACE_ID"
Subcommand |
Underlying API path |
Returned scope and purpose |
|---|---|---|
|
|
Lists visible routing models for the workspace (accepts |
|
|
Lists embedding routing models available to the workspace. |
|
|
Lists only model names and backend metadata declared by readable LLM backends in that workspace, excluding system routing models not declared on a workspace backend; see List available models for the HTTP contract. |
Seeing a model in these lists confirms configuration visibility only and does not test model capabilities or live upstream connectivity. To run a direct workspace chat completion check from the CLI, use model workspace-chat-completion or model workspace-chat-completion-stream --output (inspect request fields via moi-cli schema model workspace-chat-completion).
Inspect and manage backends, endpoints, and routers¶
provider-backend selects the backend category via --kind (llm by default, embedding, or parser). Every subcommand operates within the selected workspace (the underlying /llm/backends route is workspace-scoped rather than a cluster-wide admin endpoint):
# List workspace backends, read one backend, list endpoints, and inspect router settings
moi-cli -auth pat -o provider-backend list --workspace-id "$WORKSPACE_ID" --kind llm
moi-cli -auth pat -o provider-backend get --workspace-id "$WORKSPACE_ID" --kind llm --id "$BACKEND_ID"
moi-cli -auth pat -o provider-backend list-endpoints --workspace-id "$WORKSPACE_ID" --kind llm --backend-id "$BACKEND_ID"
moi-cli -auth pat -o provider-backend router --workspace-id "$WORKSPACE_ID" --kind llm
Add an upstream endpoint to a backend or change an endpoint’s availability status:
# Create an upstream endpoint and record its returned id
moi-cli -auth pat -o provider-backend create-endpoint \
--workspace-id "$WORKSPACE_ID" \
--kind llm \
--backend-id "$BACKEND_ID" \
--address 'https://llm-upstream.example.com/v1'
# Explicitly set endpoint status to ONLINE or OFFLINE
moi-cli -auth pat -o provider-backend set-endpoint-status \
--workspace-id "$WORKSPACE_ID" \
--kind llm \
--backend-id "$BACKEND_ID" \
--endpoint-id "$ENDPOINT_ID" \
--status OFFLINE
Warning
Review endpoint visibility and status normalization rules before changing endpoint status:
For
llmandparser,list-endpointsandgetreturnONLINEendpoints only—saveENDPOINT_IDwhen creating the endpoint before setting itOFFLINE, because an empty read result does not mean the endpoint was deleted.embeddingendpoint reads return bothONLINEandOFFLINEendpoints.set-endpoint-statusrequires an explicitONLINEorOFFLINEstring: omitting or passing an empty--statusfails client-side validation, while unrecognized non-empty values fall back toONLINEon the HTTP API (parsertrims surrounding whitespace and uppercases first). Read back after updating to confirm.
Configure backends, update routers, and probe upstream models¶
In a deployment-matched moi-cli build, store the upstream provider API key in a restricted local file (such as chmod 600 provider.key) and configure backends, routers, or model probes through flags or JSON subcommands:
# Create an LLM backend (global -api-key authenticates MOI requests only; pass upstream key via --api-key-file)
moi-cli -auth pat -o provider-backend create \
--workspace-id "$WORKSPACE_ID" \
--kind llm \
--name 'openai-compatible-prod' \
--type OPENAI \
--address 'https://llm-upstream.example.com/v1' \
--models 'qwen3.6-plus,deepseek-v3' \
--api-key-file ./provider.key
# Update router strategy and health-check parameters
moi-cli -auth pat -o provider-backend update-router \
--workspace-id "$WORKSPACE_ID" \
--kind llm \
--strategy round_robin \
--health-check-interval 30 \
--max-retries 2 \
--session-affinity=false
# Probe models exposed by an upstream address (queries the provider without saving or enabling a backend)
moi-cli -auth pat -o model probe-llm-models \
--workspace-id "$WORKSPACE_ID" \
--address 'https://llm-upstream.example.com/v1' \
--api-key-file ./provider.key
Configuration and probing options across subcommands:
Subcommand |
Key flags |
Description |
|---|---|---|
|
|
Create a backend and declare served models; |
|
|
Update explicitly supplied backend fields (passing an empty file to |
|
|
Update router selection and health-check settings for the specified backend kind. |
|
|
Fetch the model list directly from the upstream provider; |
|
|
Submit typed JSON payloads directly (inspect schemas via |
Warning
Run moi-cli provider-backend <subcommand> --help to check the flags supported by your installed binary before running write or probe commands. In earlier 0.1.0 offline packages, provider-backend create, update, update-router, probe-backend-models, and model setup-llm-backend omit --api-key-file, --models, --type, and router flags: running provider-backend update on that older binary sends {} without changing settings, and running provider-backend update-router sends {} which resets router strategy, health-check interval, max retries, and session affinity to server defaults. If your binary lacks these flags, obtain a deployment-matched installer from your deployment administrator, or use the --json-file subcommands or Product HTTP API.
Parser backend defaults and multi-key pool rules¶
When managing --kind parser backends or backends configured with multi-key credential pools, note the following server defaults and client adaptations:
moi-cli -auth pat -o provider-backend list --workspace-id "$WORKSPACE_ID" --kind parser
moi-cli -auth pat -o provider-backend list-endpoints --workspace-id "$WORKSPACE_ID" --kind parser --backend-id "$BACKEND_ID"
Default type and timeout ceiling: When a
parserbackend is created without an explicit type or timeout, the server defaults toPARSER_MINERUand a3600-second timeout (up to a maximum of86400seconds).Client response adaptation: Parser HTTP list endpoints return a raw
dataarray (nullwhen empty), which the CLI normalizes intobackendsorendpointscollections;deleteandset-endpoint-statusreturn HTTPdata: null, which the CLI reports assuccess: true.Multi-key pool (
api_keys_encrypted) rules: When a backend already uses a multi-key pool, updating only the single-keyapi_keyfield (such as via--api-key-file) is rejected by the server;--api-key-filemanages single-key credentials only and does not replace a full key pool. Submitting an explicit empty arrayapi_keys_encrypted: []through the underlying API switches back to single-key mode while retaining the prior single key or first pool entry unless a new non-emptyapi_keyis provided at the same time.
Limitations and notes¶
The global
-api-keyoption authenticates MOI Product API requests only and never populates the upstream provider’sapi_key.Saving backend or router configurations, listing
ONLINEendpoints, or probing upstream model lists does not verify live inference or document parsing; run an end-to-end workload check after configuration changes.