Lakehouse ecosystem tool tutorials

Start from connections and fixtures, then choose the actual source/destination and execution tool. The tutorials distinguish native APIs/runtimes, explicit SQL/type adapters, blocked official connectors and unqualified routes. Versions or case counts are not production certification. Each page gives usage steps and boundaries for deletes, schema, recovery and fidelity.

Continue with native DataX source scans, native Spark JDBC reads/writes and native Flink binlog, deletes and checkpoint recovery.

Choose a tutorial

Tutorial

Coverage

Prepare connections, fixtures and acceptance checks

Configure the shared connection and exact-value fixtures used by the ecosystem tutorials.

Batch loading with pandas, PETL, Polars and Dask

Load pre-created SQL tables with explicit DataFrame/ETL types and bounded batches.

PyArrow IPC and DuckDB Parquet bridges to MatrixOne

Round-trip exact decimals, binary bytes and microsecond values through files and an explicit SQL sink.

PyIceberg append/scan with an explicit SQL bridge

Use a native Iceberg catalog and Arrow schema while keeping SQL effects and commit boundaries explicit.

Airflow MySqlHook DAG and idempotent retries

Configure the official MySQL provider and a transactional task with fresh readback.

dbt-mysql model builds and data tests

Configure a real dbt project, table model and key tests with explicit incremental boundaries.

DolphinScheduler Shell and native SQL tasks

Configure and distinguish the Shell/Python route and a MYSQL datasource SQL task.

Local testing with the official Fivetran Connector SDK

Read MatrixOne as an SDK source and exercise keyed operations and checkpoint resume locally.

Airbyte connector protocol and destination acceptance

Run official check/discover/read/write and distinguish destination STATE ACK from SQL adapters.

NiFi Record flows and MatrixOne JDBC writes

Configure ExecuteSQLRecord, Avro and PutDatabaseRecord with explicit failure queues and UPSERT probes.

Kafka Connect, Debezium and JDBC CDC sinks

Configure a MySQL CDC source and MatrixOne JDBC sink with tombstones, types and offset recovery.

Exact event loading with Redpanda Connect and kafka-python

Run Kafka input into parameterized SQL with deletes, DLQ and offset replay.

Read fidelity with ConnectorX, Ibis and database drivers

Separate inferred types from explicit CAST/HEX reads and verify metadata and NULL behavior.

dlt loading and Singer state-protocol adapters

Run the official dlt SQLAlchemy destination and define custom Singer target/state boundaries.

Job acceptance for DataX, SeaTunnel, Spark, Flink and Trino

Extend existing runtime tutorials with exact types, partitions, deletes, checkpoints and load acceptance.

Validation and use

Retain original inputs/expected values before running the smallest chain. Increase batches, data volume and failure intensity only after complete fresh readback. Requalify the deployed MatrixOne release and each tool version before production; adapter success does not replace a failed official connector. These source changes target latest and do not automatically update historical branches.

Evidence and historical reports: ecosystem validation #29848 and documentation tracker #428.