Add data source

Append data sources to existing knowledge base. A successful request does not mean that the source has completed parsing or indexing.

POST https://moi.matrixorigin.cn/newmoi/semantic-models/{model_id}/sources

Preparation before calling

First check whether the data source has been added and remove duplicates; when selecting in batches by database or volume, first preview the data source to be added]. Prepare a personal access token and target workspace ID that has access to the target workspace.

The example below uses:

  • $AI_STUDIO_API_KEY: The actual personal access token, passed through the X-API-Key Header.

  • $WORKSPACE_ID: Target workspace ID, passed through X-Workspace-ID Header.

  • $MODEL_ID: The knowledge base ID of the data source to be appended.

For direct sources, provide fields by source type:

Source type

Required fields

catalog_table

table_id

catalog_file

file_id, volume_id

local_file

The file ID returned by Upload local file and the original file_name

Range selection supports Catalog Databases and Volumes only. Do not use source_selections to add data sources outside Catalog.

Request example

curl -X POST "https://moi.matrixorigin.cn/newmoi/semantic-models/$MODEL_ID/sources" \
  -H "X-API-Key: $AI_STUDIO_API_KEY" \
  -H "X-Workspace-ID: $WORKSPACE_ID" \
  -H 'Content-Type: application/json' \
  -d '{
    "sources": [
      {
        "source_type": "catalog_file",
        "file_id": "$FILE_ID",
        "volume_id": $VOLUME_ID
      }
    ]
  }'

Path parameters

Parameters

Type

Description

model_id

integer

The knowledge base ID of the data source to be appended.

Request body

Field

Type

Required

Description

sources

array

Yes

The source to add.

sources[].source_type

string

Yes

Source type, such as catalog_table, catalog_file, local_file.

sources[].table_id

integer

Conditionally required

catalog_table The table ID of the source.

sources[].file_id

string

Conditionally required

The file ID of the file source.

sources[].file_name

string

Conditionally required

local_file The original file name of the source.

sources[].volume_id

integer

Conditionally required

catalog_file The authoritative volume ID of the source.

source_selections

object[]

No

Rules for selecting sources by database or volume.

source_selections[].kind

string

Conditionally required

database_tables or volume_files.

source_selections[].database_id

integer

Conditionally required

Catalog Database ID of database_tables.

source_selections[].volume_id

integer

Conditionally required

Catalog Volume ID of volume_files.

source_selections[].all_selected

boolean

Yes

true selects all objects in the current range; false must provide the corresponding explicit selection ID.

source_selections[].selected_table_ids

array

Conditionally required

Catalog table IDs explicitly selected when all_selected is false.

source_selections[].selected_file_ids

array

Conditionally required

Catalog file IDs explicitly selected when all_selected is false.

source_selections[].excluded_table_ids

array

No

Catalog table IDs to exclude when selecting all.

source_selections[].excluded_file_ids

array

No

Catalog file IDs to exclude when selecting all.

In field paths, [] means each item in an array. For example, sources[].source_type is the source_type field of each item in sources.

Successful response

Returns 200 on success. Save the row_id for each source and confirm the actual processing results with the processing task status.

{
  "code": "OK",
  "msg": "OK",
  "data": {
    "data_domain": {
      "model_id": 401,
      "catalog_id": 10,
      "database_id": 20,
      "raw_volume_id": 30,
      "processed_volume_id": 31,
      "ensure_status": "ready"
    },
    "sources": [
      {
        "row_id": "src_01",
        "source_type": "file",
        "model_id": 401,
        "resource_id": "file_01",
        "ingest_status": "pending",
        "effective_enabled": true
      }
    ],
    "jobs": [
      {
        "job_id": "job_01",
        "source_id": "src_01",
        "model_id": 401,
        "job_type": "ingest",
        "job_status": "pending",
        "idempotency_key": "idem_01",
        "retry_count": 0
      }
    ]
  }
}

The response fields are as follows.

Field

Type

Description

code

string

OK on success.

msg

string

OK on success.

data.data_domain

object

The data field of the knowledge base and its guaranteed status.

data.sources

object[]

Created source record.

data.sources[].row_id

string

Source record ID.

data.sources[].source_type

string

Source type.

data.sources[].resource_id

string

Source resource ID.

data.sources[].ingest_status

string

Processing status; pending does not mean that it can be retrieved.

data.sources[].effective_enabled

boolean

Current actual effective status.

data.jobs

object[]

New processing task.

data.jobs[].job_id

string

Task ID.

data.jobs[].source_id

string

Associated source ID.

data.jobs[].job_type

string

Task type.

data.jobs[].job_status

string

Current task status.

data.jobs[].idempotency_key

string

Task idempotent key.

data.jobs[].retry_count

integer

Number of recorded retries.

In field paths, [] means each item in an array. For example, data.sources[].row_id is the row_id field of each item in data.sources.

Error response

{
  "code": "ErrParamInvalid",
  "msg": "invalid source request",
  "data": null
}

Common HTTP errors

HTTP status code

error code

Common causes

Recommended actions

400

ErrParamInvalid

Invalid source type, file/table ID, or volume ID.

Try again after providing the required fields by source type.

401

ErrUnauthorized

The API Key is invalid or has expired.

Check API Key.

403

ErrForbidden

The caller does not have permission to update the knowledge base or read the source object.

Check workspace and object authorization.

404

ErrNotFound

The knowledge base or source object does not exist or is not visible to the current caller.

Reconfirm ID.

409

ErrConflict

Service denied conflicting source write.

Adjust requests after checking for existing sources.

500

ErrServer

The service failed to create the source or process the task.

Keep the desensitized response information and try again.

Follow-up operations

Use data.jobs[].job_id Query data processing tasks to confirm that source processing is completed.

Last updated on