Data export

Data export sends processed files or table data from Catalog to external systems. It is suitable for delivering model-training data, backing up to object storage, or allowing external databases and applications to consume processing results. Export does not replace original assets in Catalog; it reads the selected data and writes it in the format required by the destination type.

Prepare for export

  1. Confirm that the content to export is already in Catalog and that you have permission to read the files or tables.

  2. Create a connector for the destination system and make sure its purpose includes “Export.”

  3. Confirm that the destination account has write access to the bucket, path, database, table, or model-platform dataset.

  4. For large or scheduled exports, evaluate task resources, network traffic, and destination write limits in advance.

Go to Data Connection > Data Export and start creating a task by selecting a destination type.

Export destinations

The current export entry groups destinations as follows. Actual options depend on the connectors and product capabilities available in the workspace.

Group

Destination

Common use

Model / dataset platforms

Hugging Face, LLaMA Factory

Deliver processed datasets in the format required by a training or fine-tuning workflow

Object storage

Alibaba Cloud OSS, standard S3

Backup, archiving, cross-system delivery, or downstream file processing

Database

MatrixOne

Publish table data or file-related results to a specified database and table for external queries or consumption

Note You do not need to “export to a knowledge base.” A knowledge base can directly select files or tables in Catalog as its data source. To build one, go to Knowledge bases.

Create an export task

  1. On the export page, select a destination such as Standard S3, MatrixOne, or a target model/dataset platform.

  2. Enter a task name that preferably indicates the data, destination system, and scheduling purpose.

  3. Select the content to export in the Catalog file tree. Select unstructured files and structured tables according to the scope actually supported by the task.

  4. Select a destination connector. The list shows only connectors whose purpose includes “Export.” If none is available, create or update a destination connector first.

  5. Complete destination-specific settings.

  6. Select manual triggering or a schedule, then configure compute resources and runtime priority.

  7. Click Start export, then confirm the result in the task list.

Destination-specific settings

Destination

Key settings

OSS / standard S3

Select the destination connector and write location; choose Gzip compression or no compression as needed

MatrixOne

Select the destination connector and database, then select an existing table or create a destination table

Hugging Face / LLaMA Factory

Select the corresponding platform connector; the system pushes content using the dataset organization required by the platform

For the first export to a new destination, use a small dataset to verify the path, format, compression, destination-table schema, and platform-side visibility before creating a large or scheduled task.

Scheduling and runtime resources

Exports can run manually or on a schedule. The required resources depend on whether files or table data are exported:

Export type

Compute resources

Processing

Unstructured export

Task compute resources

Read files from a volume and write them to the external destination

Structured export

Task compute resources + query compute resources

Read data from an MOI table, then write it to the external destination

You can start with the platform’s default elastic resources, then select dedicated resources and high, medium, or low priority according to timeliness, throughput, and isolation requirements. Writing over the public internet incurs network traffic; evaluate cost and destination throttling before bulk exports.

Task management

The task list at the bottom of the export page summarizes all export tasks and shows their names, content summaries, destinations, statuses, creation/completion times, and details.

Action

Description

View details and edit

View source, destination, schedule, resource, and runtime information; change settings within the scope allowed by the UI

Pause / resume

Control a running or scheduled task

Retry failure

Start again after confirming that destination permissions, network issues, and format problems have been fixed

Delete

Remove the export task and its schedule; results already written to the external system are not automatically deleted

On failure, first check the destination connector test, destination write permissions, path or table name, destination limits, and the format requirements of the exported content. If the task reads a structured table, also check query compute resources and table read permissions.

Export, load, and sharing

Requirement

Recommendation

Bring external data into MOI as a reusable asset

Use data loading

Deliver MOI processing results to an external system

Use data export

Allow a knowledge base or agent in the workspace to use existing data

Bind it directly from Catalog; no export is required