加载多文件和压缩文件

先在 MatrixOne 服务端创建两个 CSV 文件,每份两行:

mkdir -p /tmp/mo-cookbook-load
printf '1,one\n2,two\n' > /tmp/mo-cookbook-load/part-01.csv
printf '3,three\n4,four\n' > /tmp/mo-cookbook-load/part-02.csv
zstd -q -f /tmp/mo-cookbook-load/part-01.csv -o /tmp/mo-cookbook-load/part-01.csv.zst
zip -j /tmp/mo-cookbook-load/part-02.zip /tmp/mo-cookbook-load/part-02.csv

需要安装 zstd 和 zip 命令。通配符由服务端在 Stage 上展开;LOAD DATA LOCAL 不展开通配符。

CREATE DATABASE cookbook_load;
CREATE STAGE cookbook_load_stage URL='file:///tmp/mo-cookbook-load/';
CREATE TABLE cookbook_load.events(id INT, label VARCHAR(50));
LOAD DATA INFILE {'filepath'='stage://cookbook_load_stage/part-*.csv','format'='csv'}
INTO TABLE cookbook_load.events;
SELECT COUNT(*) AS loaded_rows FROM cookbook_load.events;
LOAD DATA INFILE {'filepath'='stage://cookbook_load_stage/part-01.csv.zst','format'='csv'}
INTO TABLE cookbook_load.events;
LOAD DATA INFILE {'filepath'='stage://cookbook_load_stage/part-02.zip','format'='csv'}
INTO TABLE cookbook_load.events;
SELECT COUNT(*) AS rows_after_compressed_load FROM cookbook_load.events;
DROP DATABASE cookbook_load;
DROP STAGE cookbook_load_stage;

loaded_rows 应为 4;再加载两份压缩文件后,rows_after_compressed_load 应为 8。.zst、.zstd 和 .zip 可由扩展名识别;zip 只读取第一个非隐藏、非目录条目。压缩源和错误路径以源码 回归测试 为准。清理 Stage 中的测试文件。