Lineage and Execution Artifacts

A workflow job answers whether processing for a source file completed. Lineage answers where the result came from and who consumes it. The official SDKs currently cover only a small part of the former: they list job state but do not expose node outputs, logs, or lineage queries.

Data available on a job

A WorkflowJob or Python job dictionary contains:

Field

Meaning

job_id / JobID

Job ID; the raw service field is id

workflow_id / WorkflowID

Workflow that created the job

source_file_id / SourceFileID

Source-file filter supplied by the caller; without one, the SDK may derive it from triggerTaskID in the description

status / Status

0 unknown, 1 running, 2 completed, 3 failed

start_time / StartTime

Start time

end_time / EndTime

End time; it can be empty while running

The list API does not return node inputs or outputs, error logs, result-file IDs, or lineage edges. A completed state also does not prove that the output passed business validation.

Read the artifact

The document-processing helper writes to the target volume supplied during creation. After the job completes, list files in that volume with the Catalog/File APIs and then use get_file()/GetFile() or a download method to read the target object. Retain:

  • workflow ID;

  • source-file ID;

  • job ID;

  • target-volume ID;

  • discovered result-file ID;

  • business batch or request ID.

Do not retain only a temporary download URL.

References are not a lineage graph

Both SDKs provide Catalog reference helpers, including:

  • Python: get_volume_ref_list(), get_table_ref_list(), add_volume_workflow_ref(), and remove_volume_workflow_ref();

  • Go: GetVolumeRefList, GetTableRefList, AddVolumeWorkflowRef, and RemoveVolumeWorkflowRef.

These methods inspect or manage Catalog object references. They are not APIs for a complete table-, column-, row-, or file-processing lineage graph. A reference list cannot establish that a job succeeded or reconstruct its node execution path.

Query boundary

The official SDKs do not currently publish lineage query methods or types. Use the Catalog UI or a Product API explicitly documented by the target instance for upstream/downstream graphs, column mappings, or unstructured-file processing trails. Inspect downstream dependencies before changing or deleting an asset; use the job APIs in Inspecting, Pausing, and Canceling Runs to determine job state.