Source governance and processing results

Confirm governance settings, source processing, current segments, and re-embedding separately. One success response does not establish the entire processing chain.

Governance settings

File sources can save tags, expiration, and enablement settings before processing completes. Omitted fields retain values. tags: [] clears tags; expires_at: null clears expiration. Current binding treats tags: null as absent, not a clear operation.

enabled: false also clears forced enablement after expiration. Setting enabled: true on an expired source that was not effectively enabled derives forced enablement after expiration. Read enabled, expired, force_enabled_after_expiry, and effective_enabled from the response rather than inferring final state from submitted values.

Catalog table sources accept only expiration and enablement fields, not explicit tags/force-enabled fields. The vector table’s disabled flag synchronizes only when governance actually changes enablement on a successfully processed file source. Saving settings on an unprocessed source does not establish index availability.

Source processing view

The source processing list aggregates underlying jobs by candidate source, selecting at most 32 sources per call. total counts candidate sources, not underlying jobs or complete history. No page, offset, or cursor parameters exist currently.

job_id identifies the source summary view, not an application or run ID accepted by workflow queries. Top-level reconcile_required may indicate historical source relationships need reconstruction. If a list is empty but this flag is true, inspect source relationships before assuming a retryable job exists.

Ordinary reconciliation observes run state and advances eligible associations, results, or follow-up processing. Explicit retries additionally require a source and matching failed job. RAG jobs can retry. Catalog file copying and local file loading retry according to their types. Failures associated with structured import tasks do not restart through this path.

Current segments and conditional writes

Document GET returns 20 segments by default, at most 100. Continue using segment_status.next_cursor. Conditional-write etag represents the entire current segment set, independent of the page window. It does not cover source governance fields or retrieval counts; changes do not prove vectors were regenerated.

Create, update, enable/disable, delete, and re-embed return complete segment snapshots. Use the returned ETag for subsequent conditional writes. Re-embedding may return the same ETag when segment facts do not change. On write conflicts, reread and confirm intent rather than automatically overwriting. At least one segment must remain after deletion.

Current creation uses chunk. Although doc/section pass preliminary hierarchy-type checks, they lack full persistence support and are not usable creation modes. Parsing supplies image and page-image identities on creation; requests cannot override them.

Confirm the result

updated: true only means reconciliation was handled and may occur even without candidates. Determine processing success from source ingest_status, associations, and document segment_status, not an emptied candidate list.

Source processing states use pending/succeeded/failed and related source enums. Storage preparation ensure_status uses a different set, including ready. Ready storage does not mean successful source processing; governance enablement does not mean parsed or retrievable content.

This work set retains unpublished retry field extensions.

Last updated on