Workspace inference request fields

Chat requests

Field

Type

Required

Description

model

string

Yes

Nonempty model name without leading/trailing whitespace. Use source routing lists to choose a name; they are not Genesis model ID lists.

messages

object (array of objects)

Provider-dependent

Ordinary role/content chat messages. OPENAI-compatible paths may pass through provider extensions. Gateway parsing is not complete provider schema validation.

stream

boolean

No

true requests SSE; defaults to false when omitted.

temperature、top_p

number

No

Sampling parameters. Ranges and zero semantics depend on the model. Explicit 0 may differ from omission.

max_tokens、max_completion_tokens

integer

No

Output reservation/limit according to adapter and provider protocols. Context budgeting requires a positive output reservation; max_completion_tokens takes precedence when both are supplied.

tools

object (array of objects)

No

Tool declarations, depending on model and adapter.

tool_choice

string or object

No

Tool selection constraint, depending on provider protocol.

reasoning_effort

string

No

Depends on the model’s supported reasoning_efforts and configured reasoning control protocol. type=reasoning alone does not establish effectiveness.

context_budget

string

No

required or if_supported; nonstreaming only. Empty strings/null are not omission.

Other provider extension fields

Provider-dependent

No

OPENAI-compatible paths retain unconsumed extensions. Qianwen/Ollama and other adapter conversions do not guarantee passing every extension through.

The current gateway does not fully validate message roles/content. The type table does not guarantee provider acceptance. The gateway consumes and removes context budget fields rather than sending them upstream. required returns unavailable for adapters without measurement support; if_supported may return unknown. The current measurable branch is VLLM, requiring available tokenize capability, fully measurable input, and a positive output reservation. Unmeasurable nonempty extensions may fail the check. Exceeding the budget prevents sending a generation request; see the response protocol.

Although backend_id exists in the current SDK request model, this HTTP handler does not copy it into InvokeChatCompletionCommand.BackendID. It is not a verified fixed-backend selector. This alignment issue is recorded in SDK differences. Tests of internal calls with explicit command.BackendID do not establish HTTP behavior.

Embedding requests

Field

Type

Required

Description

model

string

Recommended

Explicitly specify an available embedding model name. HTTP parsing allows omission but rejects null. Routing with an omitted model does not imply upstream acceptance.

input

string or array (strings or objects)

Yes

Omission, null, and numbers are invalid. Integer arrays of token IDs are unsupported. Object content follows the model protocol; this API does not establish support for arbitrary multimodal structures.

backend_id

integer

No

Optional backend selector. 0/omission uses automatic routing. Use identifiers from the list; platform backend IDs may be negative. null is invalid.

type

string

No

Provider/model-specific type marker; not proof of a verified capability.

dimensions, encoding_format, and other extensions

Provider-dependent

No

Handled according to the backend service protocol. The batch merge path uses numeric vectors; compatibility with all encoding formats is not guaranteed.

Body limit 4 MiB; arrays allow at most 4096 items. Empty strings/arrays may pass initial parsing but be rejected by real providers. The backend must be OPENAI, have a positive configured timeout, and an online endpoint. backend_id and proxy are not sent upstream; mock_response is unsupported. Input defaults to batches of 64 and may shrink for upstream limits. At most 128 upstream calls and a cumulative 64 MiB upstream response-read budget are allowed; reading exactly the remaining budget also fails. One request’s timeout and usage attribution apply. Batches are not independent successful retries.

Last updated on