Index, external engine, branch and inference lifecycles¶
Baseline: v4.2.5. This chapter connects interfaces with kernel lifecycles. SQL syntax does not establish multi-CN, GPU or external-system certification.
Index plugins and search¶
AlgoPlugin integrates Catalog, Compile, Plan and Idxcron hooks per algorithm. Vector/fulltext classification also uses capabilities; registration alone does not make an index vector-based.
CREATE INDEX → metadata/storage → build/background updates → Plan selection → load/query
└→ REINDEX / DROP / schema change / cache and resource release
Record experimental flags, algorithm, distance, quantization/encoding, scale and index state. Compare approximate results against exact search. Validate transaction visibility, index refresh and cross-CN caches separately. Rebuild/drop/cancellation must handle metadata and resources. GPU also requires compatible build, driver, device and native resources.
External engines, CDC and ecosystem commits¶
Native tables, file externals, writable Stages, Iceberg and remote SQL have different read/write/commit domains. Identify ownership, format, Catalog/credentials and write path before analyzing pushdown, batches and types.
File output and database metadata are not one atomic commit domain; ROLLBACK/DROP TABLE do not imply file cleanup. Iceberg snapshots do not automatically establish native primary-key uniqueness. Bad-row reporting does not promise byte-identical CSV reconstruction.
CDC/connectors need explicit source-position → destination-commit → checkpoint/offset ordering. Validate duplicates, deletes, schema changes, DDL, NULL/Decimal/time types and rebalances separately. Driver connectivity or SQL readback does not certify a connector’s recovery protocol. Main tutorials and historical branches are independently managed.
Data Branch and history¶
Lineage/DAG, common-ancestor reads and diff classification organize changes. Copy-on-Write does not make retained history free; branch lifetime, protection and GC determine storage. Validate cross-account permissions, conflicts, outputs and ancestor protection with SQL usage.
ONNX inference¶
Model bytes/DATALINK → load → shape/type validation → native runtime → JSON output. NULL examples only test NULL propagation. Actual inference needs model hash, inputs, dtype/shape, tolerances, runtime and device.
The SQL interface does not establish model limits, caching, native memory/concurrency or cancellation release. Database transactions do not roll back model files or control every native/GPU allocation.
Invariants and verification¶
Each extension must define metadata/external-effect commit domains, freshness, ownership and cleanup. Observe actual paths and record build/flags. Do not promote design proposals into released capability claims.
See vectors, externals, Data Branch, ONNX and CDC.