Run and Debug

The goal of running and debugging is not to “click Run once and see whether an error appears,” but to verify that a saved data processing definition can reliably produce the expected results with real resources, real data, and real permissions.

A workflow typically moves from design to production through the following stages: configure inputs and outputs → test run → save the formal definition → select a trigger method → inspect the job → revise and rerun based on evidence. Each stage retains a different type of evidence.

Execution Chain

flowchart LR
    A[Saved Workflow DSL] --> B[Runtime parameters and compute resources]
    B --> C[Workflow execution]
    C --> D[Nodes, workers, and runtime status]
    D --> E[Catalog result assets]
    D --> F[Job topology, inputs, outputs, and errors]
    E --> G[Lineage, knowledge bases, and downstream consumption]
    F --> H[Monitoring, logs, and rerun decisions]

The saved DSL is the formal execution source. Canvas layouts and candidate plans from the AI assistant enter this execution chain only after they have been applied and saved.

Pre-Run Checks

Complete the following checks before submitting a test or formal execution. Most failures can be prevented before a run.

Check	What to confirm	Common consequence
Workflow definition	Node parameters, connections, inputs, outputs, and target objects have all been saved	An old definition runs, or the process is incomplete
Input data	The table, volume, file, or parameter actually exists and is readable	Empty input, incorrect path, or missing object permission
Output target	The table/volume location, fields, and write mode meet expectations	The wrong object is overwritten, duplicate data is appended, or downstream systems cannot identify it
Compute resources	SQL and non-SQL steps can use the correct query/task resources respectively	The run cannot start, resources are insufficient, or the execution type is incompatible
Models and external dependencies	Required models, credentials, APIs, and network conditions are available	Parsing, embedding, extraction, or API calls fail
Permissions	The current user has view, run, and required object-access permissions	Submission is rejected, or assets cannot be accessed during execution
Trigger method	The manual, data volume, or schedule policy matches the business frequency	Unexpected duplicate runs, missed new data, or poor traceability

If processing results will affect a downstream knowledge base, business table, or shared data, first select a small sample as input. Expand the scope or enable automatic triggers only after the sample passes validation.

Test Runs and Formal Runs

A test run is used to verify whether the candidate or current editing state meets expectations. It is not the same action as saving the formal workflow, scheduling it for the long term, or writing production data.

Type	Purpose	When to use it
Test run	Fill in the current runtime parameters and validate nodes, outputs, and errors	After creating a process, changing key parameters, switching models, or adjusting data sources
One-time formal run	Run a saved workflow once with the current parameters	After confirming the process and when a specific data processing task must be completed
Data volume trigger	Start a run automatically when a new file appears in a data volume	When files are continually added and need prompt, consistent processing
Scheduled run	Execute with default parameters on a fixed schedule	Daily, weekly, or other periodic refreshes, cleaning, and aggregation

A test run does not create or replace the formal workflow definition. After a satisfactory test, you must still click Save to formally store the reviewed DSL and canvas state.

Recommended Testing Steps

1. Start a “test run” from the editor and first fill in the required parameters for the current workflow.

2. Choose a controlled input scope for the test, such as one file, a small number of records, or a single business partition.

3. Wait for the test to complete, then inspect the inputs, outputs, status, and errors of every key node.

4. Verify the result schema, record count, source location, and business fields in the target table/volume.

5. Test again after modifying the problematic node. Once everything is correct, save the workflow and start a formal run.

Testing does not mean output inspection can be skipped. Especially for model extraction, cleaning, chunking, and append-write scenarios, confirm that “the result is correct,” not merely that “the task succeeded.”

Configuring Runtime Parameters

A workflow can define a runtime form. The form explicitly exposes inputs that vary between executions, such as input objects, date ranges, business conditions, model selections, or target parameters.

Parameter design principle	Reason
Make values that change between runs parameters	The same logic can be reused for different data batches without repeatedly modifying the DSL
Keep stable business rules in node configurations	Reduce manual choices and mistakes on each run
Give required fields business-oriented names and descriptions	Runners understand why a value is required instead of guessing what a field means
Validate resource, model, and object parameters	When old parameters are copied, the target resource may no longer exist or may be inaccessible

Even if the interface says that the current workflow requires no user input, confirm that its default data sources, resources, and output objects match the purpose of this run.

Three Trigger Methods

One-Time Run

A one-time run is suitable for validation, backfilling, temporary recomputation, or one-off processing. It uses the runtime parameters entered for this run as input and creates an independently traceable execution record.

For business tasks that may need replay, record the date range, file set, or business batch used for this run. Then, after viewing the result in job details, the team can identify exactly what this execution processed.

Data Volume Trigger

A data volume trigger is suitable for scenarios where “a new file should be processed as soon as it arrives.” After configuration, the platform monitors the selected data volume. Whenever a new file appears, it starts a workflow execution with that file’s information as input.

Before using a data volume trigger, you must first select a monitorable data volume in the runtime parameters. Do not enable the trigger without an explicit data-volume parameter, because the runtime would not know which input location to monitor.

Examples suitable for a data volume trigger include automatically extracting fields after a new contract is uploaded, automatically transcribing a new recording, and parsing newly added daily images for retrieval preparation.

Scheduled Run

A scheduled run is suitable for refreshing data, aggregating metrics, or rebuilding results at a fixed frequency. You can configure it with interval, hourly, daily, weekly, or monthly expressions provided by the interface, or enter an advanced Cron expression when needed.

Advanced Cron supports five- or six-field expressions: configure it by minute, hour, day, month, and weekday, and add a seconds field when second-level granularity is required. After saving, the platform schedules executions according to the expression; manually clicking Complete does not immediately trigger an extra scheduled run.

Scheduled tasks should be idempotent whenever possible. If the same period may be rerun, the target table’s create, replace, or append policy must prevent duplicate data and inexplicable overwrites.

How Compute Resources Participate in Execution

SQL and non-SQL steps in a workflow require different resource types. Resources are part of the runtime control plane, not part of the data assets.

Step type	Resource generally used	Examples
SQL queries, structured reads/writes, and SQL nodes	Query compute resources	Aggregation, joins, cleaning, and result-table materialization
Parsing, embedding, extraction, file processing, and other non-SQL nodes	Task compute resources	Document parsing, audio/video processing, model calls, and file writes

Paused, faulty, or undersized resources directly affect workflows. Before a run, confirm that resources are available. For resource-related failures, first check status, capacity, and associated workloads in Compute resources.

Observing an Execution Through Its Job

After a formal run, open Jobs to inspect the execution record. The Jobs page answers: “Did this run succeed? Which node failed? What data did it process? Can it be recovered or rerun?”

Job information	Question it answers
Workflow, execution ID, trigger method	Which process is this, and which execution was started by whom or by what policy?
Status, start and end times, duration	Is it running, completed, paused, failed, or canceled? Is the duration abnormal?
Workflow topology	Which node succeeded, failed, was skipped, or is still waiting?
Processed files and data	What content did this run actually consume or produce?
Node details	What configuration did the node use, what input did it receive, and what result or error did it output?

When troubleshooting, do not look only at “Failed” in the list. Open the details, locate the first critical failing node, then determine the root cause from its inputs, outputs, parameters, and upstream dependencies.

A Repeatable Troubleshooting Path

Symptom	Where to look first	Next step
The task does not start	Workflow save status, runtime form, role permissions, compute resources	Verify required parameters, resource status, and run permission
Data cannot be read	Node input, Catalog object, and object permissions	Check the table/volume location, file scope, connection, and read permission
A parsing or model node fails	Node details, model configuration, input sample	Confirm model availability, supported formats, and parameters, then retry with a small sample
A SQL node fails or produces unexpected results	Node SQL, target-table policy, SQL editor	Reproduce the SQL in a workbook first, and check fields, permissions, and write mode
Output table data is duplicated or overwritten	Target and write mode of the write node or SQL node	Confirm whether this run should append, replace, or create, and design idempotent rerun logic
An external API call fails	Node request parameters, credentials, and Operation Logs	Check authorization, response errors, and timeouts, then decide whether to retry
The job takes too long	Job topology, input size, resource specifications, SQL history	Locate the bottleneck node first, then optimize the data scope, SQL, concurrency structure, or resources

For SQL-related issues, SQL History provides additional query text, results, and execution-plan information. For API or permission issues, operation logs are better suited to examining request paths, status codes, and error context.

Pause, Resume, Retry, and Rerun from a Node

Job details display available actions according to the current status and what the backend permits. Pause, resume, retry, or rerun from a node only when the interface actually provides the corresponding action.

Action	Suitable situation	What to assess first
Pause	The current execution can be paused safely	Whether downstream systems will be affected by the delay or resource release
Resume	A paused execution still meets the conditions to continue	Whether inputs, resources, credentials, and dependencies remain valid
Retry	A temporary resource, network, or external-service fluctuation caused the failure	Whether the failure is transient rather than a configuration or data error
Rerun from a node	A processing step has been identified and the downstream path needs to run again after a fix	Whether upstream results are reliable and target writes can be repeated safely

Retrying is not “taking another chance.” For incorrect parameters, insufficient permissions, wrong data sources, or incorrect write modes, fix the root cause before starting a new execution so that the result remains explainable.

Principles for Stopping and Deleting

When terminating a workflow, distinguish between “stopping this execution” and “deleting the workflow definition.” Stopping affects the current job; deleting affects future editing, execution, reuse, and collaboration.

Before deletion, stop or wait for running and paused executions to complete, and confirm that no other workflow, knowledge asset, or team task still depends on the process. The server validates unsafe deletions, but protective errors should not be treated as a routine governance mechanism.

Making Execution Results Trustworthy

A successful job does not automatically prove data quality. For critical processes, standardize the following four types of validation as team checks.

1. Completeness: Do the input count, output count, and exception count match expectations?

2. Correctness: Can business samples verify key fields, chunks, extracted results, or SQL aggregations?

3. Traceability: Can the result be traced back to the source file, table, processing node, and this execution record?

4. Rerunnability: If rerun with the same inputs and parameters, will it produce an explainable result without unexpected duplication or overwrites?

These checks put “Control Your Data” into practice: data should not only be processable, but should also explain where it came from, how it was processed, and why downstream systems can trust it.

Next Steps

• To learn about node inputs, outputs, and composition, read Workflow nodes.

• To diagnose issues using the three types of records—jobs, SQL history, and logs—read Monitoring and Alerts.

• To design a tested process as a structured-data or knowledge-base production chain, read Processing patterns.