从 Kafka 分区读取并续读¶
本地集成测试会启动 broker、写入确定性消息、验证无重复无遗漏的 offset 链和事务性检查点:
make test-kafka-exttab-e2e-local
手工实验时,先在 127.0.0.1:9092 准备含 JSON 对象消息的 orders topic,且每条消息都有 id 和 amount 字段。然后连接同一 MySQL 会话执行:
CREATE DATABASE cookbook_kafka;
CREATE EXTERNAL TABLE cookbook_kafka.orders (id BIGINT, amount DECIMAL(12,2))
ENGINE = KAFKA WITH (
'brokers' = '127.0.0.1:9092', 'topic' = 'orders',
'partition' = '0', 'format' = 'jsonl', 'autocommit' = 'false'
);
SELECT id, amount, __mo_message_id FROM cookbook_kafka.orders
WHERE __mo_read_start_id = -1 AND __mo_read_size = 10 AND __mo_read_timeout = 3;
SELECT LAST_KAFKA_MESSAGE_ID();
记录最后返回的 offset N,在同一会话用 __mo_read_start_id = N 再查,下一批应从 N+1 开始。若要把消息写入目标表,应把目标行和检查点放在同一事务中;详见源码 Kafka E2E。清理:DROP DATABASE cookbook_kafka;,测试 broker 按测试脚本停止。