Catalog¶

Catalog is the workspace’s unified data-asset center. Raw data loaded through connectors, results written by workflows, and accumulated models and operators are all browsed, managed, and reused here. It is the destination for import tasks and the upstream asset source for SQL, workflows, knowledge bases, sharing, and export.

Entry point: Resource Center → Catalog.

How Catalog works with other features¶

Catalog receives loaded and processed data and makes it available for queries, knowledge bases, sharing, and export.

Connectors / load → Catalog → Workflows · SQL · Knowledge bases · Sharing/subscriptions · Export
                                  ↑__________________|
                              (results can be written back)

What you want to do	Where to go
Bring in external data	Data connections: connectors, loading, and load tasks
Prepare compute for SQL, workflows, or loading	Compute resources
Process tables and write back results	Data processing: workflows, running, and debugging
Query tables in Catalog	SQL editor
Turn data into searchable, conversational knowledge	Knowledge bases
Make data available to agents	Agents (configure a knowledge base first)
Let another workspace use the data	Data sharing, Data subscriptions
Export externally	Data export
Control who can manage these objects	User permissions
Check task completion and alerts	Monitoring and alerts
View storage and other costs	Cost and billing

Cross-workspace sharing, external-database catalogs, and similar capabilities depend on console availability. Follow the procedures below when their entries are enabled.

Object-tree structure¶

From top to bottom:

Workspace → Catalog → Database → Volume / Table / Model / Operator

Level	Meaning
Catalog	Top-level group: an MOI catalog whose data is stored in the platform, or an external database catalog whose data remains at the source while the platform registers metadata
Database	Container for a group of related objects
Table	Structured data
Volume	File-based, unstructured data
Model	Model-service integration
Operator	Reusable processing capability

Catalog also contains read-only subscribed libraries created through data subscriptions and built-in read-only system catalogs.

Browse and preview¶

Expand the object tree from the workspace root: catalog → database → volume/table/model/operator.
Search by name or filter by type, such as volume, table, operator, or model.
Select a node; the details pane and breadcrumbs change accordingly.

External catalogs, subscribed libraries, and built-in system objects generally use different icons or labels such as “System Default.”

Common details-page content¶

Object	Main details
Catalog	Database list and size information; external catalogs also show connection and synchronization status
Database	Volume, table, operator, and model lists
Table	Row count and size; columns, statistics, SQL, data sampling, lineage, and other tabs
Volume	File list; upload, download, file lineage, and more
Model	Category, version, and integration method such as API or HuggingFace
Operator	Basic information; an API Service tab if an API has been published

Data sampling for a table is available only when processing has completed or the object is otherwise ready.

Manage objects¶

Object	Common operations	Example creation entry
Catalog	Create / rename / delete	“Create Catalog” in Catalog
Database	Create / rename / delete	“Create Database” in catalog details
Table	Create / rename / delete	“Create → Table” in a database
Volume	Create; upload/download files	“Create → Volume” in a database
Model	Create / manage	“Create → Model” in a database
Operator	Create / manage; publish as an API	“Create → Operator” in a database

Before deletion, confirm that downstream workflows, knowledge bases, export tasks, and other consumers have no hard dependency. For impact analysis, first inspect the Lineage tab in table details.

Create a catalog¶

Choose one of two types:

	MOI catalog	External database catalog
Data location	Inside the platform	Remains in the source database; the platform registers metadata
Creation method	Create databases, tables, and volumes	Select an existing database connector and synchronize its hierarchy
Writability	Supports create, delete, and update	Generally read-only; structure follows the upstream source
Query	Uses platform storage and compute	Pushes SQL down to the source and does not require a platform-side data copy

To create an external catalog when the entry is available:

Create a catalog and select External Database Catalog.
Select a configured database connector. Maintain connection information under Connectors, not here.
Enter a catalog name and description.
After creation, synchronize database, table, schema, and other metadata.

Hierarchy terms differ among source databases: PostgreSQL includes schemas, while MongoDB uses collections, for example. Federated queries do not mount platform storage but may still depend on source connectivity. For high-stability requirements, load the data into an MOI catalog instead.

Create a table¶

Define columns individually in the UI, or paste/upload CREATE TABLE SQL.

Volumes and files¶

Uploading creates a load task that the platform uses to parse and ingest data.
Compressed archives generally cannot be uploaded directly; extract them first.
Files with the same name as an existing volume file may be skipped.
The file list can download original files or parsed artifacts, subject to the UI.

Models¶

Models wrap LLM, embedding, and similar integrations for use by workflow nodes, knowledge bases, and other features. Common integration methods include:

API integration: provider, base address, model ID, API Key, and other settings; connection testing is supported
HuggingFace: model ID, token, and other settings

Also specify a model category, such as large language model, vision, OCR, text embedding, or reranking. For the billing boundary among platform Credit, Genesis, and BYOM, see Credit and billable items.

Operators¶

An operator is an atomic “input + parameters → processing → output” capability that can be reused as a custom node in a workflow.

When creating one, configure a Chinese name and English identifier, language such as Python or SQL, input/output descriptions, parameter list, implementation code, and more. A code node in a workflow can also be frozen as an operator and added to Catalog.

Publish as an API service (from operator details)¶

Both system and custom operators can publish an API service from operator details for synchronous calls by external systems: callers submit data and receive an immediate result. Prefer the workflow job mechanism for long-running batch processing.

Publishing generally configures:

Item	Description
Operator version	Pin a version so platform upgrades do not change behavior
Runtime resource	Non-model operators use a task compute resource; model operators may not require one and can use a model service
Call-rate limit	Maximum calls per unit of time
Authentication	API Key / OAuth / public access (use public access cautiously in production)
Timeout and concurrency	Runtime settings

After publication, an API Service tab may appear in details for enabling/disabling the service and viewing endpoint documentation. For development-side calls, see Development.

System catalogs¶

System operator libraries, system model libraries, sample libraries, metadata libraries, and similar assets are read-only and cannot be created, deleted, or modified. System operators and model versions used by workflows often come from here. Create custom assets in an ordinary MOI catalog.

Subscribed libraries¶

These read-only libraries are created when another workspace publishes and this workspace subscribes. They are placed at the Catalog location selected during subscription. Their contents update with the source and become inaccessible when the subscription is canceled or publication is revoked.

Data lineage¶

Lineage answers “where did this data come from, and who uses it?” The platform derives it automatically from declared processing relationships such as loads, workflow reads/writes, exports, and knowledge-base associations; manual registration is generally unnecessary.

Structured tables (Table details · Lineage)¶

There are three levels, from coarse to fine:

Level	What it shows
Table-level graph	Upstream sources and downstream consumers of the current table
Column-level mapping	Which upstream columns and logic produced each column
Row-level trace	Source rows that contributed to a result, traced by grouping/association keys when applicable

Common graph nodes include external sources, processing nodes such as SQL, Python, and loading, Catalog tables, and consumers such as knowledge bases and export tasks. Switch between “Upstream · How it was produced” and “Downstream · Who uses it.” Before changing or deleting a table, inspect downstream impact. A table not involved in platform processing may show no lineage.

Column mappings may label pass-through, expression, aggregation, JOIN-key, and other transformation types. A processing node can link to its workflow.

Unstructured files (Volume · File lineage)¶

Documents, images, audio, and video use page-level or source-unit-level forward tracking. Select a page or segment and follow artifacts through parsing → cleaning → chunking → embedding → knowledge base and other steps. Open Lineage from the file list in volume details.

Typical minimum units are page/block for documents, whole image/region for images, and time slice for audio/video. Select a chunk to compare its originating layout block and inspect processing parameters at each step. This corresponds to knowledge-base chunk maintenance; see Chunks and semantics.

Difference from job monitoring¶

Lineage describes data dependencies and origins. To determine whether a job succeeded, see jobs and logs under Monitoring and alerts. For workflow runtime debugging, see Run and debug.

Versions and connection strings¶

Capability	Description
Data versions	Some objects support historical versions, comparisons, and rollback under version management in object details
SQL connection string	Host, port, database/table path, and authentication for direct BI/client/application connections

Connection-string queries consume query compute resources. For sustained workloads, specify a dedicated query resource; see Compute resources. For development-side connection instructions, see Development.

Storage and costs¶

Raw data, processing results, vectors, and other data in Catalog generally count toward Credit billable items such as data storage. For usage and bills, see View cost and usage and Credit and billable items.