Models and Providers

MOI model configuration is scoped to a workspace. A model request is backed by a provider family, one or more backends and endpoints, and a router configuration:

Workspace
└── Provider (LLM / Embedding / Parser)
    ├── Backend
    │   ├── Models or supported MIME types
    │   └── Endpoint (one or more service addresses)
    └── Router (configuration for this provider family)

  • A provider is a capability-specific entry point, not a resource that you create. The Product SDK exposes LLM, Embedding, and Parser providers.

  • A backend stores the configuration of a model service, such as its name, protocol type, timeout, model names, or supported MIME types.

  • An endpoint is a service address owned by a backend. A backend can have more than one endpoint.

  • A router controls selection within a provider family, together with health-check, retry, and session-affinity settings.

These objects are part of the model-resource control plane used by workflows, knowledge bases, and other product features. They are distinct from the Genesis Model Inference API, which applications use to send inference requests.

SDK scope

The examples in this section target the MOI Product SDK under matrixflow/sdk:

  • The Python package imports as moi_product_sdk and exposes ModelProviderHandle, ProviderBackendHandle, and ProviderRouterHandle.

  • The Go module is github.com/matrixorigin/matrixflow/sdk/go-sdk and exposes the corresponding Go types.

The standalone moi-python-sdk and moi-go-sdk repositories currently cover existing services such as Catalog; they do not expose the provider, backend, endpoint, or router configuration APIs described here. Do not reproduce these calls through guessed routes when using a standalone SDK. Configure model resources in Resource Center > Model Resources, or use an environment that includes the Product SDK.

Choosing an entry point

Once you have a WorkspaceHandle, select the provider family that matches the resource:

Purpose

Python

Go

LLM

workspace.llm_provider()

workspace.LLMProvider()

Embedding

workspace.embedding_provider()

workspace.EmbeddingProvider()

Document parsing

workspace.parser_provider()

workspace.ParserProvider()

A practical configuration sequence is:

  1. Select the provider family and, for LLM or Embedding, probe the remote address before creating a backend.

  2. Create the backend with an explicit name and the options required by that provider family.

  3. Keep the returned backend ID, then add or manage endpoints as needed.

  4. Read the current router configuration and update only values confirmed for the deployment.

  5. Read the backend, endpoint list, and router again to verify the final state.

Continue with:

Before changing or deleting a model resource, check that workflows, knowledge bases, and other workspace resources no longer depend on it. Model-resource operations are also subject to workspace permissions.