Workspace inference request fields¶
Chat requests¶
Field |
Type |
Required |
Description |
|---|---|---|---|
model |
string |
Yes |
Nonempty model name without leading/trailing whitespace. Use source routing lists to choose a name; they are not Genesis model ID lists. |
messages |
object (array of objects) |
Provider-dependent |
Ordinary role/content chat messages. OPENAI-compatible paths may pass through provider extensions. Gateway parsing is not complete provider schema validation. |
stream |
boolean |
No |
true requests SSE; defaults to false when omitted. |
temperature、top_p |
number |
No |
Sampling parameters. Ranges and zero semantics depend on the model. Explicit 0 may differ from omission. |
max_tokens、max_completion_tokens |
integer |
No |
Output reservation/limit according to adapter and provider protocols. Context budgeting requires a positive output reservation; max_completion_tokens takes precedence when both are supplied. |
tools |
object (array of objects) |
No |
Tool declarations, depending on model and adapter. |
tool_choice |
string or object |
No |
Tool selection constraint, depending on provider protocol. |
reasoning_effort |
string |
No |
Depends on the model’s supported reasoning_efforts and configured reasoning control protocol. type=reasoning alone does not establish effectiveness. |
context_budget |
string |
No |
required or if_supported; nonstreaming only. Empty strings/null are not omission. |
Other provider extension fields |
Provider-dependent |
No |
OPENAI-compatible paths retain unconsumed extensions. Qianwen/Ollama and other adapter conversions do not guarantee passing every extension through. |
The current gateway does not fully validate message roles/content. The type table does not guarantee provider acceptance. The gateway consumes and removes context budget fields rather than sending them upstream. required returns unavailable for adapters without measurement support; if_supported may return unknown. The current measurable branch is VLLM, requiring available tokenize capability, fully measurable input, and a positive output reservation. Unmeasurable nonempty extensions may fail the check. Exceeding the budget prevents sending a generation request; see the response protocol.
Although backend_id exists in the current SDK request model, this HTTP handler does not copy it into InvokeChatCompletionCommand.BackendID. It is not a verified fixed-backend selector. This alignment issue is recorded in SDK differences. Tests of internal calls with explicit command.BackendID do not establish HTTP behavior.
Embedding requests¶
Field |
Type |
Required |
Description |
|---|---|---|---|
model |
string |
Recommended |
Explicitly specify an available embedding model name. HTTP parsing allows omission but rejects null. Routing with an omitted model does not imply upstream acceptance. |
input |
string or array (strings or objects) |
Yes |
Omission, null, and numbers are invalid. Integer arrays of token IDs are unsupported. Object content follows the model protocol; this API does not establish support for arbitrary multimodal structures. |
backend_id |
integer |
No |
Optional backend selector. 0/omission uses automatic routing. Use identifiers from the list; platform backend IDs may be negative. null is invalid. |
type |
string |
No |
Provider/model-specific type marker; not proof of a verified capability. |
dimensions, encoding_format, and other extensions |
Provider-dependent |
No |
Handled according to the backend service protocol. The batch merge path uses numeric vectors; compatibility with all encoding formats is not guaranteed. |
Body limit 4 MiB; arrays allow at most 4096 items. Empty strings/arrays may pass initial parsing but be rejected by real providers. The backend must be OPENAI, have a positive configured timeout, and an online endpoint. backend_id and proxy are not sent upstream; mock_response is unsupported. Input defaults to batches of 64 and may shrink for upstream limits. At most 128 upstream calls and a cumulative 64 MiB upstream response-read budget are allowed; reading exactly the remaining budget also fails. One request’s timeout and usage attribution apply. Batches are not independent successful retries.