SDK, pandas and SQL exploration¶
Follow installation and execution and run sdk_eda.py with sdk_common.py. The complete example creates 12 records, including one NULL amount, one NULL customer ID and microsecond timestamps.
Bound queries into a DataFrame¶
The helper uses Session.execute(text(sql), params) and builds a DataFrame from SQLAlchemy Result.fetchall/keys. There is no invented SDK to_dataframe call. Client.execute returns a different ResultSet with rows/columns.
Money stays Python Decimal. The helper first constructs an object-dtype DataFrame, preserving nullable large integers before explicitly assigning customer_id the pandas Int64 extension dtype. Values 9007199254740993 and NULL are asserted; converting to float and then Int64 cannot recover lost precision. Microseconds are checked by value. These DATETIME values use an agreed naive UTC convention; per-row offsets are not inferred.
Aggregate in SQL, validate in Python¶
Filtering/GROUP BY execute in the database; pandas checks exact totals. Only plotting converts money to float. NULL amounts are not zero; COUNT(*) and COUNT(amount) distinguish total rows from known amounts.
channel |
n_rows |
n_amount |
revenue |
|---|---|---|---|
mobile |
6 |
6 |
421.5000 |
web |
6 |
5 |
331.2500 |
Total revenue is 752.7500. The script writes sdk_ds_summary and independently SELECTs it back. Decimal, NULL and microseconds have value assertions; a chart is not an exact financial reconciliation.
Pages and chart¶
Keyset SQL WHERE id > :last_id ORDER BY id LIMIT 5 yields 5, 5 and 2 rows, with IDs 1–12. Paging occurs in SQL; ResultSet.fetchmany does not bound the full query’s memory. Production needs a stable key and a cross-page consistency policy. This fixture does not change during paging.
Matplotlib saves daily.png locally; pandas saves daily.csv and checks.json records acceptance. Inspect the printed temporary directory. Chart validation concerns six daily aggregates and their total, not exact pixels. Missing dates and business calendars need separately defined policies.
Expected checks¶
Output includes rows=12, null_amounts=1, revenue=752.7500, keyset_rows=12 and writeback_verified=true. The exact downloaded script executed on official MO v4.2.5 and produced PNG/CSV. SQL tables are cleaned up; local artifacts remain.