Workflow¶

A workflow organizes multiple processing steps into a savable, executable, and traceable data processing flow. It is especially suitable for tasks that must run reliably and repeatedly, such as multimodal parsing, cleaning, chunking, embedding, structured extraction, SQL transformation, and writing results back.

Three Facts About Workflows¶

You must understand the following three facts to use the canvas and AI orchestration assistant correctly.

Fact	Meaning	How it affects you
The saved DSL is the source of truth for execution	At runtime, a workflow uses the saved `dsl_yaml`	Changing only the canvas without saving does not change subsequent formal executions
The canvas is an editable representation	Node positions, connections, and configurations make the process understandable and editable	The canvas should express the logic clearly, but it is not an independent execution source
An AI candidate is not a formal process	Copilot generates or modifies a candidate DSL	You must review, apply, test, and save it first; a candidate does not start a task on its own

These three layers allow conversational and configuration-driven approaches to work together: you can quickly express your intent in natural language, then confirm critical details one by one on the canvas and in configuration panels.

When to Create a Workflow¶

Workflows are suitable when the processing logic will run again, not only for one-off temporary analysis.

Scenario	Why use a workflow
Newly arriving documents must be parsed, cleaned, and indexed for retrieval on an ongoing basis	Inputs, parsing policies, and output locations can be fixed and executed repeatedly through a data volume trigger or schedule
Raw tables must be cleaned and transformed regularly, then written to target tables	SQL and non-SQL steps can be orchestrated into one asset production chain
Structured information must be extracted from contracts, résumés, drawings, or multimedia	Parsing, model, field extraction, and persistence steps can be linked into a reviewable job
Multiple teams need to reuse the same processing logic	Explicit input, output, and permission boundaries reduce the cost of verbal handoffs or copied scripts

If you only want to confirm a table’s fields, validate a filter condition, or quickly compare aggregation results, start with the SQL editor.

Preparation Before Creation¶

Before opening the workflow editor, clarify the business objective, inputs, and outputs. The more complete the preparation, the easier it is to accurately validate a process generated from a template or by the AI assistant.

Preparation item	Question to answer
Input assets	Which table, volume, files, or configured data source will be processed?
Target result	Which table, volume, knowledge base, or governed consumption interface should receive the result?
Processing rules	Which fields must be retained, cleaned, extracted, chunked, or aggregated? Which quality rules must not be violated?
Resources and models	Which resources and models are required for SQL, parsing, embedding, or model steps?
Execution strategy	Will it run only manually, be triggered by new files, or run on a schedule?
Permissions and responsibility	Who can edit, run, view results, and handle failure alerts?

Prefer named and permission-controlled objects in Catalog for inputs and outputs. Temporary paths or personal naming may speed up experimentation, but they make later collaboration, lineage tracing, and reruns difficult.

Entry and Creation Methods¶

Open “Workflow” in AI Studio’s data processing area, then open an existing process from the list or create a new one. When creating a workflow, choose one of the following methods according to the maturity of the task.

Creation method	Suitable situation	Next action
Blank canvas	You already understand the nodes, dependencies, and output design	Manually add nodes, connections, and parameters
Template	Your requirements are close to a typical processing pattern provided by the platform	Replace the inputs, outputs, models, and runtime parameters, then test
DSL import	The team already maintains a reusable workflow DSL	Import the YAML and check the canvas projection, resource references, and parameter form
AI orchestration assistant	You can describe the goal clearly but do not want to start from a blank canvas	Have the assistant generate a candidate, then review, apply, and save it

Templates and AI candidates are starting points, not configurations that are already adapted to your data. In particular, data-source paths, target tables, models, sensitive fields, and scheduling policies must be changed to the actual configuration of the current workspace.

Organizing a DAG on the Canvas¶

The workflow editor uses a visual canvas to represent processing relationships. Ordinary processing nodes are organized as a directed acyclic graph: one node’s output can flow to multiple downstream nodes, and multiple upstream nodes can converge on the same processing step when the node’s input constraints are satisfied.

Every new workflow has fixed start and end boundaries. A start node cannot accept upstream connections, and an end node cannot have outgoing connections. The canvas also prevents a node from connecting to itself or forming a direct loop through ordinary connections.

When you need parallelism, conditional routing, or repeated execution, use process control nodes instead of drawing loop edges manually. This allows the runtime to interpret branches, joins, and loops correctly.

Recommended Sequence for Manual Creation¶

Select a read node from the node panel and place a Catalog table, volume, or other authorized input on the canvas.
Add processing nodes based on the data’s state, such as parsing, cleaning, chunking, embedding, structured extraction, or SQL transformation.
Configure the data source, model, parameters, and output constraints for each node; do not connect nodes without configuring them.
Use output nodes to write results to explicit Catalog targets, rather than leaving critical results in a temporary runtime context.
Connect the nodes and verify that every edge represents a real dependency, rather than a placement that merely “looks sequential.”
Name the workflow and describe its purpose so other members viewing the list can determine what data it processes and what results it produces.
Run a test first, then save it as a formally executable workflow. For details, see Run and Debug.

When the canvas contains many nodes, use automatic layout and zoom controls to improve readability. The purpose of the layout is not visual decoration, but to make inputs, transformations, branches, outputs, and error-handling paths immediately clear.

Using the AI Orchestration Assistant to Generate a Candidate¶

The workflow editor provides a conversational AI orchestration assistant. It is suitable for quickly turning a business objective into an editable process draft, and for requesting specific changes to an existing canvas.

When describing your requirements, clearly state these five types of information together: what the input data is, what result you want, which processing steps are required, where the output should go, and which quality or business constraints must not be ignored.

For example, do not merely say “process PDFs.” Specify which volume contains the PDFs, that they should be parsed page by page and cleaned, that they should be chunked by section and have contract numbers and amounts extracted, which table should receive the results, and how failed files should be retained.

Safe Sequence for AI Orchestration¶

Describe the processing objective in the conversation area, adding input objects, target structure, and constraints as needed.
When the assistant presents choices or follow-up questions, confirm them against actual resources in the current workspace.
Review the generated candidate process and confirm that its nodes, connections, inputs, outputs, and runtime parameters meet expectations.
Apply the candidate DSL to project it onto the canvas, then make detailed adjustments in the node configuration panels.
Use a test run to validate parameters and results, then save after everything is correct.

The assistant can accelerate design, but it does not replace the data owner’s judgment. Generating a candidate DSL does not automatically execute the workflow, nor does it mean that access to data, models, or external services has been authorized.

Node Configuration and Interface Contracts¶

Click a node on the canvas to open its configuration area. Fields vary by node, but every node should be reviewed from the perspectives of input, processing, and output.

Review aspect	What to check
Input	Source object, format, fields, file scope, parameters, and whether read permission exists
Processing	Parsing policy, SQL, model, prompt, cleaning rules, timeout, or other node-specific parameters
Output	Result type, target object, write method, field/document structure, and downstream compatibility

The most common issue in a processing chain is not that a node is unavailable, but that the data shape produced upstream does not match what the downstream node expects. Therefore, after modifying a node, validate its output structure before adding further downstream nodes.

For available nodes and ways to combine them, see Workflow nodes.

Saving, Viewing, and Routine Management¶

Saving writes the formal DSL and the canvas restoration state to the workflow definition. Only after it has been saved successfully can the workflow serve as a stable basis for subsequent execution, scheduling, job tracing, and lineage analysis.

The workflow list supports locating objects by name or status. Depending on the current user’s permissions and the workflow’s status, the interface provides actions such as view details, edit, enable or disable, stop, restart, and delete. Not every user sees the same actions for every workflow.

Before deletion, confirm that there are no running or paused executions and no collaborative tasks that still depend on the process. The platform provides protective feedback for unsafe deletion, but the better practice is to confirm the impact in jobs and lineage first.

Relationship to Catalog, Compute Resources, and Permissions¶

A workflow does not replace data assets, computing power, or access control. It combines the three into an executable processing contract.

Capability	How a workflow depends on it	Learn more
Catalog	Provides inputs, outputs, models, operators, and traceable asset locations	Catalog
Compute resources	Provide query or task compute for SQL or non-SQL nodes	Compute resources
User permissions	Control the ability to view, create, modify, run, stop, and delete workflows	User permissions
Jobs and logs	Record how an execution started, which nodes it passed through, and why it succeeded or failed	Run and Debug

Next Steps¶

To choose the right nodes for a process, read Workflow nodes.
To test, run, schedule, or troubleshoot failures, read Run and Debug.
To learn a complete design from raw data to a knowledge base or structured results, read Processing patterns.