Upload locally and create an import task¶
Upload local files and create import tasks. This interface uses multipart/form-data; unlike the connector source’s JSON creation interface. A successful response returns the task ID but does not indicate that all files have been written to the target location or that all parsing has been completed.
POST https://moi.matrixorigin.cn/newmoi/connectors/upload
Before you call¶
First select the data form and target:
Unstructured files are written to the data volume. Prepare a writable data volume ID and specify it via
VolumeID.Structured File is written to the data table. Currently, only CSV, XLS or XLSX can be selected for the creation page. First obtain
conn_file_idthrough Upload File, and then specify an existing target table or create a new target table throughtable_config. The data rows of the structured file are written to the table, not to the data volume specified byVolumeID.
The local file selection restrictions for the currently created page are as follows:
Unstructured files: select up to 20 files at a time, with a maximum size of 200 MiB for a single file. You can select files or folders; when you select a folder, each file in the folder counts toward the quantity.
Structured files: Only 1 file can be selected at a time, with a maximum size of 200 MiB per file; currently only CSV, XLS or XLSX are allowed.
The example below uses:
$AI_STUDIO_API_KEY: The actual personal access token, passed through theX-API-KeyHeader.$WORKSPACE_ID: The workspace ID of the task to be created, passed through theX-Workspace-IDHeader.
Request example¶
Import unstructured files into data volumes¶
The following example uploads local files to the <VOLUME_ID> data volume. Path regularization operates on meta.path, so you can use it to exclude unnecessary files; it does not control the target directory within the data volume.
curl -X POST "https://moi.matrixorigin.cn/newmoi/connectors/upload" \
-H "X-API-Key: $AI_STUDIO_API_KEY" \
-H "X-Workspace-ID: $WORKSPACE_ID" \
-F 'file=@./manuals/product.pdf' \
-F 'file=@./manuals/guide.docx' \
-F 'VolumeID=<VOLUME_ID>' \
-F 'meta=[{"filename":"product.pdf","path":"manuals/product.pdf"},{"filename":"guide.docx","path":"manuals/guide.docx"}]' \
-F 'path_regex=^manuals/.*' \
-F 'unzip_keep_structure=false' \
-F 'dedup={"by":["name","md5"],"strategy":"skip"}'
Import local CSV into existing data table¶
First upload file obtains <CONN_FILE_ID>, and then submits the table configuration. The example uses row 1 as the table header, starts reading data from row 2, and maps the file columns to the existing target table.
curl -X POST "https://moi.matrixorigin.cn/newmoi/connectors/upload" \
-H "X-API-Key: $AI_STUDIO_API_KEY" \
-H "X-Workspace-ID: $WORKSPACE_ID" \
-F 'meta=[{"filename":"orders.csv","path":"orders.csv"}]' \
-F 'table_config={"new_table":false,"table_id":<TARGET_TABLE_ID>,"conn_file_ids":["<CONN_FILE_ID>"],"isColumnName":true,"columnNameRow":1,"rowStart":2,"csv":{"separator":","},"existed_table":[{"tableColumn":"<TARGET_COLUMN>","column":"<FILE_COLUMN>","col_num_in_file":1}],"existed_table_opts":{"method":"append"}}'
Form fields¶
Field |
Type |
Is it required |
Description |
|---|---|---|---|
|
file[] |
Condition required |
Upload one or more files during unstructured local import. Each file is passed using the field |
|
JSON array |
Yes |
Source information corresponding to the order of uploaded files. Each item contains |
|
string |
Conditional required |
The target data volume ID for unstructured import. The request does not have a target directory field, and the file is written to this data volume; structured file import does not fill it out. |
|
JSON integer array |
No |
File type codes allowed to be processed. The code is the same as |
|
string |
No |
Only process files whose |
|
boolean string |
No |
Whether to keep the directory structure when decompressing, for example |
|
JSON object |
No |
Duplicate file handling settings. Object contains |
|
JSON object |
Condition required |
Target table settings for structured file import. Can contain |
Successful response¶
The task has been accepted when the code in the response is OK.
{
"code": "OK",
"msg": "OK",
"data": {
"task_id": "task_01",
"file_ids": ["file_01"],
"success": true,
"message": "OK",
"results": [
{"success": true, "message": "OK"}
]
}
}
The response fields are as follows.
Field |
Type |
Description |
|---|---|---|
|
string |
|
|
string |
|
|
string |
ID of the created import task. |
|
string[] |
This newly uploaded and successfully processed file ID; may be empty in the structured temporary file process. |
|
boolean |
Whether the request was created successfully. |
|
string |
Processing result description. |
|
object[] |
Processing results returned by file. Each item contains |
[] after a type denotes an array. [] in a field path denotes each item in an array.
Error response¶
{
"code": "ErrParamInvalid",
"msg": "invalid upload parameters",
"data": null
}
Common HTTP errors¶
HTTP status code |
error code |
Common causes |
Recommended actions |
|---|---|---|---|
|
|
The multipart form is invalid, or |
Check field names, JSON strings, target resource IDs, and regular expressions. |
|
|
The caller does not have permission to write data to the target data volume or target table. |
Check workspace, data volumes and data table authorizations. |
|
|
The service failed to create or schedule the task. |
Check HTTP status and |
Follow-up operations¶
Log data.task_id. Use the ID to Query task details to confirm the target, status and processing statistics; if necessary, Query task files or Query running records.