Import tasks

Choose an import method by data source and shape, then inspect task status, file results, or run records.

Choose a task

Task

Page

Description

Find import tasks

Query task list

Lists the import tasks in the current workspace that you have read permission for.

Inspect an import task

Query task details

Read the definition and current status of an import task.

Understand import-task fields

Import task object fields

Describe task fields and their conditions in import list and detail responses.

Import connector files

Create task

Import connector files or directories into a volume, or load structured file data into a target table.

Upload and import local files

Upload locally and create an import task

Import local files into a volume, or load structured file data into a target table.

Import structured database data

Create a database structured loading task

Read structured data from a table, collection, or other supporting object in a database connector and write it to the target data table.

Resolve a backfill position

Resolve a structured-load backfill start

Resolve the backfill start for a structured-load task.

Change a paused import task

Update task

Update a paused import task.

Remove an import task

Delete task

Delete the task record without rolling back data already written to the destination.

Request an import pause

Pause task

Request a pause for a running import task; success indicates acceptance.

Resume a paused import

Resume task

Request resumption of a paused import task; success indicates acceptance.

Inspect import file results

Query task file

List the files and processing results in an import task.

Inspect import runs

Query running records

Lists the running record of an import task to confirm the status and processing scope of each execution of the task.

Retry failed import files

Retry failed files

Request a retry for selected or all failed files; success indicates acceptance.

Confirm where data will be imported

  • Unstructured imports: The selected data volume is the final write location. Connector-based imports specify it with volume_id; local imports specify it with the multipart VolumeID field. The current request has no target-directory field: files are written to the selected data volume.

  • Structured-file imports: Data is written to the target table in table_config. For an existing table, provide table_id; to create a table, provide database_id, new_table: true, and create_table. A data volume is not the target location for data rows in this task type.

  • Structured database loads: Data is written to the target table in structured_load_config.target. The request cannot include volume_id and has no target-directory field.

A connector file or directory source path is specified in source_config.common_file_task_config.uris. In local uploads, the path in meta describes and filters the uploaded source path; it is not a target directory in the data volume. The current creation page does not offer Keep directory structure, so compressed packages are handled in a flattened form.

Check connector file types

Before creating a task, you can call List files. data.files[].type in the response is the file type recognized by the connector, and data.files[].uri can be used as the source URI for connector-file imports.

  • The current structured file picker allows only CSV (7), XLS (24), and XLSX (25).

  • The current unstructured file filter offers document, audio, video, image, web page, email, CAD, and other categories. Use a type value returned by the file list in file_filter_config.file_types or the local-upload file_types field.

File types indicate only that the type can be recognized from a file name. Whether a task completes still depends on the source file, connector, and import configuration. After submitting a task, get task details, task files, or run records to confirm the actual result.

Local-upload count and size limits

The following are limits of the current creation page’s local file picker. They are not a general promise about server-side processing for direct API calls.

Import type

Maximum selection per attempt

Single-file size

Notes

Unstructured files

Up to 20 files

Up to 200 MiB

You can select local files or a folder. Each file in a selected folder counts toward the 20-file limit.

Structured files

1 file

Up to 200 MiB

The current file picker allows only CSV, XLS, or XLSX.

Configurable options for unstructured imports

After selecting a data volume, unstructured imports can use the following options as needed.

Page feature

Request field

Effect

Connector source / local upload

/task JSON request or /connectors/upload multipart request

Determines whether files are read from a connector or uploaded from the local computer.

One-time / scheduled import

load_mode_config

A connector source can run once or scan a directory every minute, hour, or day. Scheduled tasks can select only directories.

File-type filter

file_filter_config.file_types or file_types

Select document, audio, video, image, web page, email, CAD, and other types. When Other is selected, files are not filtered by type.

Path regular expression

file_filter_config.path_regex or path_regex

Processes only files whose source paths match the regular expression.

Decompression strategy

unzip_keep_structure

The current page handles compressed packages in a flattened form.

Deduplication strategy

dedup.by, dedup.strategy

Detects duplicate files by file name and/or MD5, then skips or overwrites them.
Last updated on