Inspect Model Backends and Endpoints¶
List the LLM backends configured in a workspace and inspect the endpoint addresses and online status for a selected backend with the SDK.
Task flow¶
List workspace LLM backends ──▶ Select target backend ──▶ Inspect backend endpoints and status
Prerequisites¶
Configure the client, personal access token, and workspace ID (
PRODUCT_API_BASE_URL,PRODUCT_API_KEY,WORKSPACE_ID) by following Use the SDK.
1. List LLM backends¶
Initialize the client and workspace model service, then list the workspace’s LLM backends to obtain the total count, backend IDs, names, and declared model names.
import os
import moi_product_sdk as sdk
client = sdk.new_with_personal_access_token(
os.environ["PRODUCT_API_BASE_URL"], os.environ["PRODUCT_API_KEY"]
)
workspace = client.workspace(os.environ["WORKSPACE_ID"])
models = workspace.models()
page = models.list_workspace_llm_backends()
print("Total backends:", page.total)
for backend in page.backends:
print(backend.id, backend.name, list(backend.models))
package main
import (
"context"
"fmt"
"os"
sdk "github.com/matrixorigin/matrixflow/sdk/go-sdk"
)
func main() {
ctx := context.Background()
client, err := sdk.NewWithPersonalAccessToken(
os.Getenv("PRODUCT_API_BASE_URL"), os.Getenv("PRODUCT_API_KEY"),
)
if err != nil {
panic(err)
}
workspace, err := client.Workspace(os.Getenv("WORKSPACE_ID"))
if err != nil {
panic(err)
}
models := workspace.Models()
page, err := models.ListWorkspaceLLMBackends(ctx)
if err != nil {
panic(err)
}
fmt.Println("Total backends:", page.GetTotal())
for _, backend := range page.GetBackends() {
fmt.Println(backend.GetId(), backend.GetName(), backend.GetModels())
}
}
2. Read backend endpoints¶
Select a target backend ID from the returned list and query its endpoint addresses and status values.
if not page.backends:
raise SystemExit("no LLM backends configured in this workspace")
backend_id = page.backends[0].id
for endpoint in models.list_workspace_llm_endpoints(backend_id):
print("Endpoint:", endpoint.id, endpoint.address, "Status:", endpoint.status)
if len(page.GetBackends()) == 0 {
fmt.Println("no LLM backends configured in this workspace")
return
}
backendID := page.GetBackends()[0].GetId()
endpoints, err := models.ListWorkspaceLLMEndpoints(ctx, backendID)
if err != nil {
panic(err)
}
for _, endpoint := range endpoints {
fmt.Println("Endpoint:", endpoint.GetId(), endpoint.GetAddress(), "Status:", endpoint.GetStatus())
}
Limitations and notes¶
Endpoint status values: Endpoint status is encoded as an integer where
0means online (the SDK also defaults to0when the server omits the field) and1means offline. Having a backend record or a0status indicates only that the endpoint is enabled in configuration; actual reachability and vision or reasoning capabilities still require live invocation checks.Retaining offline endpoint IDs: LLM backend details and LLM endpoint queries return online endpoints only, so an empty list can also mean that all endpoints are currently offline. Save the endpoint ID returned at creation before setting an LLM endpoint offline so you can bring it back online later. Embedding model endpoint queries return both online and offline endpoints.
Scope differences across model lists: The workspace model service provides both a list of model names declared on workspace backends and a list of routable models permitted by the current access scope (which can include system models). Because the two lists serve different purposes, their model sets do not necessarily match.
Next steps¶
Discover and Select Agents: Find available agents in the workspace and invoke them.
Read Resource Inventories: Review other resource configurations in the current workspace.