SDK and scikit-learn: train, evaluate and write back¶
Follow installation and execution, running sdk_ml.py with sdk_common.py. The complete lab has 16 daily synthetic observations, x/z features, one missing x and a generated label.
Chronological split and leakage controls¶
SQL uses cutoff 2026-01-13: 12 earlier rows train and 4 later rows test. Assertions prove the time order. Only x/z are features; label, id and future information are excluded.
The Pipeline is SimpleImputer(median) → StandardScaler → Ridge(alpha=1.0). Fit only training data and transform/predict test data. Imputer statistics must equal training medians. A negative control shows the combined train/test x median differs. Do not fit the complete table before splitting.
Synthetic records are assumed visible at event time. Production also needs event/ingestion times, revisions and actual point-in-time visibility. One chronological split does not prove absence of future information leakage; tuning needs independent validation or time cross-validation.
Computation and metrics¶
scikit-learn trains/predicts in Python, not inside MatrixOne. Tested output is train=12, test=4, MAE approximately 2.713467, MSE 7.445498 and R² 0.273610. Floating-point output need not be character-identical.
These fixed synthetic metrics validate the workflow rather than business suitability. Checks cover finite values, split boundaries, preprocessing and prediction readback. Production needs baselines, residuals, drift, representative samples and a real validation set. These numbers are not predictive-quality or performance promises.
Atomic writeback and reconstruction¶
A JSON manifest stores feature order, cutoff, imputer values, scaler mean/scale, Ridge coefficients/intercept and alpha. Model metadata and four predictions share one Session.
An intentional failure after the model metadata insert proves neither result table has rows. The next transaction commits both, then independent SELECT readback matches Python predictions. Inputs, preprocessing, model and outputs must share a run_id; retries must not silently append duplicates.
The script reads the stored JSON and reconstructs the same Ridge predictions with NumPy, comparing at 1e-12 tolerance. This representation is specific to this Pipeline, not a universal serialization format; no external pickle/joblib model is loaded.
Artifacts and acceptance¶
model.json and metrics.json remain in the printed local temporary directory; lab SQL tables are removed. leakage_control, rollback_verified and writeback_verified must all be true. The exact script passes with SDK 0.1.16 and official MO v4.2.5.
The experiment does not certify production batch inference, distributed/GPU training, a model registry, deployment, CDC, concurrent retries or full historical replay. Persistent model operations should also record dataset version/snapshot ID, extraction SQL, dependency lock, code revision, privileges and retention.