Shared information for structured sources and import control¶
Confirm source probing, sampling, task creation, run triggering, and destination writes separately. Successful sampling or triggering does not establish completed import into the destination table.
Identity and variables¶
Examples use the public /v5 URL, AI_STUDIO_API_KEY, and WORKSPACE_ID. Prepare the target Workspace’s resource privileges; all connectors, tasks, and destination data must belong to the same environment. Source APIs also accept validated short-lived task identities, but runtime authorization, roles, and task identifiers cannot be copied between environments.
Obtain CONNECTOR_ID from the connector list. Kafka uses an actually readable KAFKA_TOPIC. Table capability queries use SOURCE_DATABASE and SOURCE_TABLE, adding source.schema when needed. TASK_ID comes from the Workspace import task list or creation result; do not use a run ID.
API enums and source restrictions¶
Task |
Currently allowed sources |
Parameter and result boundaries |
|---|---|---|
Sampling |
|
Count 1–100. single requires one topic, multiple a nonempty topic list, and pattern a nonempty pattern. Do not supply database, table, Schema, or collection. |
Table capabilities |
|
Must match the connector’s actual type and specify database and table. Assess Doris source reading and CDC capabilities separately. |
Trigger immediately |
Periodic structured tasks for Kafka, Doris, MySQL, or REST API |
Must be active, in created or uploading state, and have a server-side periodic workflow task. MySQL has no independent compute. |
One-time full refresh |
Periodic incremental Doris or MySQL tasks |
Must be active and in created or uploading state, with run_mode=periodic and sync_strategy=incremental. MySQL has no independent compute. |
The API uses SOURCE_TYPE_*; lowercase Worker runtime types are a different input contract. SDKs and CLI do not automatically correct lowercase types here. Kafka sampling returns source-defined next_position; the current input has no position/cursor parameter. Do not interpret it as a generic pagination protocol.
Confirm run results¶
Successful run control returns code: OK, data: null, without a run ID or completion report. Query task runs and details, correlate time, task references, and errors for this execution, then confirm destination data. For unknown outcomes, query state first rather than issuing new requests to determine whether the previous one succeeded.
Full refresh sets full_refresh for one execution without modifying the periodic task’s sync strategy. It is not a universal retry entry point for ordinary files, Kafka, REST API, or every failure state. Resolve paused tasks or unavailable dependencies first.
Errors and environments¶
Actions |
Verified business error behavior |
Handling |
|---|---|---|
Source sampling and capabilities |
Invalid source type/scope, disabled connector, or unsupported import returns HTTP 400 |
Retain connector and environment context; check enums, matching types, topics, and counts. |
Structured capability not enabled |
Feature-switch errors may return HTTP 200 with business code |
Read the business code. HTTP 200 does not mean the capability is enabled. |
Run control |
An ineligible task state/configuration may return HTTP 200 |
Read the business code, then query the task and run records. Authorization and missing-resource errors may return other statuses beforehand. |
Worker or source read |
May include sanitized source diagnostics and internal service errors |
Retain request information and safe error fields. Do not invent stable public fields from internal exceptions. |
Genesis /v1 and Billing are separate services.
The old Reference’s five source browsing, Schema, and backfill parsing operations, plus three missing REST draft/saved probe/preview operations, still require separate field-level review. These four API pages do not cover all source or import contracts.