MatrixOrigin MatrixOrigin Docs
Product docs
MatrixOne Current product MatrixOne Intelligence
About Get Started Develop Tutorial Deploy Operations Migrate Test Performance Security Reference Troubleshooting FAQs Release Notes Glossary Contribute
/
Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
MatrixOne Docs
MatrixOne Docs
  • Home
  • Overview
    • MatrixOne Feature List
    • MatrixOne Feature Overview
      • Git for Data
      • Multi-Account
      • Scalability
      • Cost-Effective
      • High Availability
      • Timing
      • Streams
      • User-defined functions
      • MySQL Compatibility
      • Feature Overview
      • Native SQL Tasks
      • Stored procedures
    • MatrixOne Architecture
      • Transactional Analytical Engine Architecture
      • Detailed Logservice Architecture
      • Logtail Protocol Architecture
      • Transaction and Lock Mechanisms Architecture
      • Detailed Proxy Architecture
      • WAL Technology Explained
      • Detailed Caching and Hot-Cold Data Separation Architecture
      • Detailed Stream Engine Architecture
      • MatrixOne-Operator design and implementation
      • SQL Tasks and TaskService
      • Index, external engine, branch and inference lifecycles
      • Kernel observability and diagnosis
      • GIS type and function execution boundaries
      • Catalog, startup readiness and upgrade boundaries
      • Object lifecycle, recovery and GC
      • Distributed execution, memory and spill
      • SQL planning and statistics
      • Transaction visibility, errors and retry boundaries
    • Comparisons with Other Databases
      • MatrixOne vs. common OLTP databases
    • What's New
    • Getting Started
      • Deploy on macOS
        • Using binary package
        • Using Docker
      • Deploy on Linux
        • Using binary package
        • Using Docker
      • Basic SQL
      • Installation choices and acceptance
    • Developing Guide
      • Java connect to MatrixOne
        • Connect MatrixOne with Java ORMs
      • Python connect to MatrixOne
      • General Python SDK
      • C# connect to MatrixOne
      • Connecting to MatrixOne with Golang
      • MatrixOne SSL connection
      • Connecting to MatrixOne with TypeScript
      • Connecting to MatrixOne with ODBC
      • Connecting to MatrixOne with Power BI
      • Lakehouse Ecosystem Tutorials
        • Prepare connections, fixtures and acceptance checks
        • Batch loading with pandas, PETL, Polars and Dask
        • PyArrow IPC and DuckDB Parquet bridges to MatrixOne
        • PyIceberg append/scan with an explicit SQL bridge
        • Airflow MySqlHook DAG and idempotent retries
        • dbt-mysql model builds and data tests
        • DolphinScheduler Shell and native SQL tasks
        • Local testing with the official Fivetran Connector SDK
        • Airbyte connector protocol and destination acceptance
        • NiFi Record flows and MatrixOne JDBC writes
        • Kafka Connect, Debezium and JDBC CDC sinks
        • Exact event loading with Redpanda Connect and kafka-python
        • Read fidelity with ConnectorX, Ibis and database drivers
        • dlt loading and Singer state-protocol adapters
        • Job acceptance for DataX, SeaTunnel, Spark, Flink and Trino
        • Native DataX source scans and exact-value writes
        • Native Spark JDBC reads, writes, types and partitions
        • Native Flink MySQL CDC into MatrixOne and recovery acceptance
      • Schema Design
        • Create Database
        • Create Table
        • Replication table
        • Create View
        • Create Temporary Table
        • Create Secondary Index
        • Vector
        • Data Integrity
          • NOT NULL Constraints
          • UNIQUE KEY Constraints
          • PRIMARY KEY Constraints
          • FOREIGN KEY Constraints
          • AUTO INCREMENT Constraints
      • Write Data
        • Bulk Load
          • Load csv format data
          • Load jsonlines format data
          • Load data from S3
          • Load data by using the `source`
        • Update Data
        • DML Target and Conflict Semantics
        • Delete Data
        • Prepared
      • Export Data
        • Export data by MODUMP
      • Read Data
        • Multi-table Join Queries
        • Subquery
        • Views
        • Common Table Expression
        • Window Function
          • Time Window
          • Analytical Windows
      • Data de-duplication
        • BITMAP
      • Account Design
        • Publish-Subscribe
      • Transactions
        • Transaction by MatrixOne Server
          • Explicit Transaction
          • Implicit Transaction
          • Pessimistic Transaction
          • Optimistic Transaction
          • Isolation Level
          • MVCC
          • User Guide
            • Scenario
          • Scenario
      • User-defined function
        • UDF python advanced
      • Vector
        • Vector Search
        • IVF Rank Options
        • Query-level IVF Probe Count
        • Cluster Centers
        • Scalar Vector Query
        • IVF Equi-join Vector Query
      • Data Type Fidelity
      • Ecological Tools
        • BI integration guides and compatibility
          • Shared BI dataset and KPI checks
          • Connect Metabase and build metrics
          • Superset analytic connection checks
          • Grafana MySQL datasource and compatibility checks
          • Connect Redash and save analytic queries
          • Connect DataEase and preview SQL datasets
          • Evidence MySQL source and compatibility checks
          • Rill ingestion and native metrics
          • Apache Zeppelin JDBC notebooks
          • SQLPad saved SQL analyses
          • Lightdash direct-connection boundary
        • Visualizing MatrixOne Reports with Yonghong BI
        • Visual Monitoring of MatrixOne with Superset
        • ETL Tools
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Using DataX to write data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from ClickHouse to MatrixOne
            • Writing Data from Doris to MatrixOne
            • Writing Data from InfluxDB to MatrixOne
            • Writing Data from Elasticsearch to MatrixOne
        • Computing Engine
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Hive to MatrixOne
          • Writing Data from Doris to MatrixOne
          • Using Flink to Write Real-Time Data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from Kafka to MatrixOne
        • Scheduling Tools
      • Ecosystem Tools Overview
        • DataX
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Writing Data from PostgreSQL to MatrixOne
          • Writing Data from SQL Server to MatrixOne
          • Writing Data from MongoDB to MatrixOne
          • Writing Data from TiDB to MatrixOne
          • Writing Data from ClickHouse to MatrixOne
          • Writing Data from Doris to MatrixOne
          • Writing Data from InfluxDB to MatrixOne
          • Writing Data from Elasticsearch to MatrixOne
        • SeaTunnel
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Using DataX to write data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from ClickHouse to MatrixOne
            • Writing Data from Doris to MatrixOne
            • Writing Data from InfluxDB to MatrixOne
            • Writing Data from Elasticsearch to MatrixOne
        • Spark
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Hive to MatrixOne
          • Writing Data from Doris to MatrixOne
          • Using Flink to Write Real-Time Data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from Kafka to MatrixOne
        • Flink
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Writing Data from SQL Server to MatrixOne
          • Writing Data from PostgreSQL to MatrixOne
          • Writing Data from MongoDB to MatrixOne
          • Writing Data from TiDB to MatrixOne
          • Writing Data from Kafka to MatrixOne
        • BI integrations
          • Shared BI dataset and KPI checks
          • Connect Metabase and build metrics
          • Superset analytic connection checks
          • Grafana MySQL datasource and compatibility checks
          • Connect Redash and save analytic queries
          • Connect DataEase and preview SQL datasets
          • Evidence MySQL source and compatibility checks
          • Rill ingestion and native metrics
          • Apache Zeppelin JDBC notebooks
          • SQLPad saved SQL analyses
          • Lightdash direct-connection boundary
        • CDC
          • mo_cdc command reference
            • From MatrixOne to MySQL
            • From MatrixOne to MatrixOne
          • MatrixOne to MySQL
          • MatrixOne to MatrixOne
        • S3 and MinIO
          • Load from S3
          • Import from MinIO
          • Deploy with S3 storage
            • Plan MatrixOne Cluster Topology
              • Experience Environment Deployment Plan
              • Minimum Production Environment Deployment Plan
              • Recommended Production Environment Deployment Plan
              • Production readiness and acceptance
            • Cluster Deployment Guide
              • Deployed Kubernetes and object storage environment
            • Cluster Operations Management
              • Updating
              • Health check and resource monitoring
              • Scaling
              • Managing CN Groups with Proxy
              • Import data from local Minio to MatrixOne
              • Operator Management
                • Kubernetes Operator acceptance operations
            • Deploy Matrixone Cluster
        • Lakehouse ecosystem tutorials
          • Prepare connections, fixtures and acceptance checks
          • Batch loading with pandas, PETL, Polars and Dask
          • PyArrow IPC and DuckDB Parquet bridges to MatrixOne
          • PyIceberg append/scan with an explicit SQL bridge
          • Airflow MySqlHook DAG and idempotent retries
          • dbt-mysql model builds and data tests
          • DolphinScheduler Shell and native SQL tasks
          • Local testing with the official Fivetran Connector SDK
          • Airbyte connector protocol and destination acceptance
          • NiFi Record flows and MatrixOne JDBC writes
          • Kafka Connect, Debezium and JDBC CDC sinks
          • Exact event loading with Redpanda Connect and kafka-python
          • Read fidelity with ConnectorX, Ibis and database drivers
          • dlt loading and Singer state-protocol adapters
          • Job acceptance for DataX, SeaTunnel, Spark, Flink and Trino
          • Native DataX source scans and exact-value writes
          • Native Spark JDBC reads, writes, types and partitions
          • Native Flink MySQL CDC into MatrixOne and recovery acceptance
        • Workflow orchestration
          • DolphinScheduler
        • Application, AI and data development
          • Dify Platform Integration Guide for MatrixOne
          • Prerequisites
          • Steps
          • RAG application foundation
          • Git for Data
        • Database administration, migration and backup tools
          • mo_ctl Distributed Edition
          • mo_ctl Standalone Edition
          • Migration overview
            • MySQL
            • Data migration compatibility cookbook
            • Oracle
              • Heterogeneous type acceptance
            • SQL Server
            • PostgreSQL
            • Reconciliation and cutover
          • Data migration compatibility cookbook
          • MySQL migration guide
          • Oracle migration guide
            • Heterogeneous type acceptance
          • PostgreSQL migration guide
          • SQL Server migration guide
          • mo-dump export utility
          • mo-dump backup and restore
          • mo_br backup and recovery
            • mo_br regular physical backup recovery
            • mo_br snapshot backup recovery
            • mo_br pitr
          • Backup and recovery overview
            • Backup and Recovery Concepts
            • Backup and Restore by using mo-dump
            • mo_br Backup and Recovery
              • mo_br regular physical backup recovery
              • mo_br snapshot backup recovery
              • mo_br pitr
            • MatrixOne active/standby disaster recovery
            • cdc
              • From MatrixOne to MySQL
              • From MatrixOne to MatrixOne
            • Mount Data
            • DECIMAL256 index upgrade
            • Recovery selection and acceptance
            • Operations overview
      • Develop Overview
      • AI Agent Tools
        • Query MatrixOne Documentation with AI Agent
      • Qualify an Ecosystem Integration
      • Failure Recovery
      • Ecosystem tools index
        • ETL Tools Index
        • Compute Engines Index
        • BI Tools Index
      • Data import index
      • Data export index
      • Vector search index
    • Tutorial
    • Deploying
      • Plan MatrixOne Cluster Topology
        • Experience Environment Deployment Plan
        • Minimum Production Environment Deployment Plan
        • Recommended Production Environment Deployment Plan
        • Production readiness and acceptance
      • Cluster Deployment Guide
        • Deployed Kubernetes and object storage environment
      • Cluster Operations Management
        • Updating
        • Health check and resource monitoring
        • Scaling
        • Managing CN Groups with Proxy
        • Import data from local Minio to MatrixOne
        • Operator Management
          • Kubernetes Operator acceptance operations
      • Deploy Matrixone Cluster
    • Maintenance
      • Backup and Recovery Concepts
      • Backup and Restore by using mo-dump
      • mo_br Backup and Recovery
        • mo_br regular physical backup recovery
        • mo_br snapshot backup recovery
        • mo_br pitr
      • MatrixOne active/standby disaster recovery
      • cdc
        • From MatrixOne to MySQL
        • From MatrixOne to MatrixOne
      • Mount Data
      • DECIMAL256 index upgrade
      • Recovery selection and acceptance
      • Operations overview
    • Migrating
      • MySQL
      • Data migration compatibility cookbook
      • Oracle
        • Heterogeneous type acceptance
      • SQL Server
      • PostgreSQL
      • Reconciliation and cutover
    • Testing
      • TPCH Test with MatrixOne
      • TPCC Test with MatrixOne
      • Testing Tool
        • MO-Tester Specification
    • Performance Tuning
      • Understanding the Query Execution Plan
        • Using EXPLAIN to learn the execution plan
        • Explain Statements Using JOIN
        • Explain Statements Using Subqueries
        • Explain Statements Using Aggregation
        • EXPLAIN Statements Using Views
      • Performance tuning best practices
        • Scaling CN for better performance
        • Partition Pruning
        • Usage scenarios of Partition Pruning in KEY Partitioned Tables
        • Usage scenarios of Partition Pruning in HASH Partitioned Tables
        • Performance Tuning Examples for Partition Pruning
        • Constraints
        • Performance Tuning with Partitioned Tables
      • Optimizer Hints
    • Privilege
      • Authentication and Authorization
      • Password Management
      • Access Control
        • Privilege Management Scenario
        • Best Practices
        • SQL File Access
        • Role Grant Consistency
      • SQL File Access
      • User Guide
        • Create accounts, Verify Resource Isolation
        • Use the new account to creates users, roles, grant the privilege
      • Data Transmission Encryption
      • Security Audit
    • Reference
      • System Variables parameters
        • Save query result support
        • Timezone support
        • Lower case table names support
        • Foreign key checking support
        • User-specified case consistency support for query result set column names
        • Illegal login restrictions
        • Password complexity verification
        • Connection whitelist
        • Enable remap hint
      • cte_max_memory_bytes
      • cte_max_recursion_depth
      • enable_explain_scheduling
      • event_scheduler
      • experimental_cagra_index
      • experimental_fulltext_index
      • experimental_fulltext2_index
      • experimental_hnsw_index
      • experimental_ivf_index
      • experimental_ivfpq_index
      • fulltext_bloom_filter_pushdown
      • ft2_relevancy_algorithm
      • group_concat_max_len
      • lock_wait_timeout
      • mo_rollback_txn_on_error
      • protected_databases
      • query_max_workers
      • query_pool_strict
      • remap_rewrites
      • remapdb
      • sort_spill_mem
      • Custom variable
      • SQL Language Structure
        • Comments
      • Data Types
        • Data Type Conversion
        • Date and Time Types
          • YEAR Type
        • Geometry Type
        • JSON Data Type
        • BLOB and TEXT Type
        • DATALINK Type
        • ENUM Type
        • UUID Type
        • VECTOR Type
        • Fixed-Point Types (Exact Value) - DECIMAL
        • Set Type
      • SQL Statements
        • Data Definition Language
          • CREATE INDEX
          • CREATE INDEX...USING IVFFLAT
          • CREATE INDEX...USING HNSW
          • CREATE FULLTEXT INDEX
          • CREATE TABLE
          • CREATE TABLE AS SELECT
          • CREATE TABLE ... LIKE
          • CREATE EXTERNAL TABLE
          • CREATE EXTERNAL TABLE ... ENGINE = DATASTREAM
          • CREATE EXTERNAL TABLE ... ENGINE = SQL|ESQL
          • CREATE EXTERNAL TABLE ... ENGINE = KAFKA
          • External Table Error Mode
          • CREATE FULLTEXT2 INDEX
          • CREATE ICEBERG CATALOG
          • ALTER ICEBERG CATALOG
          • DROP ICEBERG CATALOG
          • CREATE MONGODB CONNECTION
          • ALTER MONGODB CONNECTION
          • DROP MONGODB CONNECTION
          • CREATE CLUSTER TABLE
          • CREATE CLONE
          • CREATE PITR
          • CREATE PUBLICATION
          • CREATE SEQUENCE
          • CREATE STAGE
          • CREATE...FROM...PUBLICATION...
          • CREATE VIEW
          • CREATE FUNCTION...LANGUAGE SQL AS
          • CREATE FUNCTION...LANGUAGE PYTHON AS
          • CREATE SOURCE
          • CREATE DYNAMIC TABLE
          • CREATE SNAPSHOT
          • CREATE BRANCH
          • DELETE BRANCH
          • DIFF BRANCH
          • MERGE BRANCH
          • ALTER TABLE
          • ALTER TABLE ... ALTER REINDEX
          • ALTER PITR
          • ALTER PUBLICATION
          • ALTER SEQUENCE
          • ALTER STAGE
          • ALTER VIEW
          • DROP DATABASE
          • DROP INDEX
          • DROP TABLE
          • DROP PITR
          • DROP PUBLICATION
          • DROP SEQUENCE
          • DROP STAGE
          • DROP SNAPSHOT
          • DROP VIEW
          • DROP FUNCTION
          • TRUNCATE TABLE
          • RENAME TABLE
          • RESTORE PITR
          • RESTORE SNAPSHOT
          • Branch Protect Snapshots
          • Data Branch Pick
          • Sql Task
          • Data Branch Privilege
          • CREATE PROCEDURE / CALL / DROP PROCEDURE
        • Data Manipulation Language
          • INSERT INTO SELECT
          • DELETE
          • UPDATE
          • LOAD DATA INFILE
          • LOAD DATA INLINE
          • UPSERT
            • INSERT ON DUPLICATE KEY UPDATE
            • INSERT IGNORE
            • REPLACE
          • MERGE
          • RETURNING
          • Information Functions
            • LAST_INSERT_ID()
            • ROW_COUNT()
            • Current_Role
          • Case
          • Replace
        • Data Query Language
          • OUTER APPLY
          • JOIN
            • INNER JOIN
            • LEFT JOIN
            • RIGHT JOIN
            • FULL JOIN
            • OUTER JOIN
            • Examples
            • NATURAL JOIN
            • Cross Join
          • SELECT
          • BY RANK WITH OPTION
          • SUBQUERY
            • Derived Tables
            • Comparisons Using Subqueries
            • SUBQUERY with ANY or SOME
            • SUBQUERY with ALL
            • SUBQUERY with EXISTS
            • SUBQUERY with IN
          • With CTE
          • Combining Queries
            • UNION
            • INTERSECT
            • MINUS
        • Data Control Language
          • ALTER ACCOUNT
          • CREATE ROLE
          • CREATE USER
          • ALTER USER
          • DROP ACCOUNT
          • DROP USER
          • DROP ROLE
          • GRANT
          • REVOKE
          • Role Rule
        • Other
          • SHOW DATABASES
          • SHOW CREATE TABLE
          • SHOW CREATE VIEW
          • SHOW CREATE PUBLICATION
          • SHOW TABLES
          • SHOW INDEX
          • SHOW COLLATION
          • SHOW COLUMNS
          • SHOW FUNCTION STATUS
          • SHOW GRANT
          • SHOW PROCESSLIST
          • SHOW PUBLICATIONS
          • SHOW PITRS
          • SHOW ROLES
          • SHOW SEQUENCES
          • SHOW STAGE
          • SHOW SUBSCRIPTIONS
          • SHOW VARIABLES
          • SHOW ICEBERG CATALOGS
          • SHOW ICEBERG NAMESPACES
          • SHOW ICEBERG TABLES
          • SHOW MONGODB CONNECTIONS
          • SHOW PROFILE
          • Show Create Database
          • Show Table Status
          • SET
          • USE
            • ANALYZE TABLE
            • DUMP TABLE
            • LOAD TABLE
            • PERFORM
            • Starlark Procedure Error Reporting
          • KILL
          • Prepared Statements
            • EXECUTE
            • DEALLOCATE
          • Explain
            • EXPLAIN Output Format
            • Explain Analyze
            • Explain Prepared
          • Describe
          • ANALYZE TABLE
          • DUMP TABLE
          • LOAD TABLE
          • PERFORM
          • Starlark Procedure Error Reporting
      • Operators
        • OPERATORS
          • OPERATORS Precedence
          • Arithmetic Operators
            • %,MOD
            • *
            • +
            • -
            • -
            • /
            • DIV
          • Assignment Operators
            • =
          • Bit Functions and Operators
            • &
            • >>
            • <<
            • ^
            • |
            • ~
          • Cast Functions and Operators
            • BINARY
            • CAST
            • CONVERT
            • DECODE
            • ENCODE
            • SERIAL
            • SERIAL_FULL
          • Comparison Functions and Operators
            • >
            • >=
            • <
            • <>,!=
            • <=
            • =
            • BETWEEN ... AND ...
            • IN
            • IS
            • IS NOT
            • IS NOT NULL
            • IS NULL
            • ILIKE
            • ISNULL
            • LIKE
            • NOT BETWEEN ... AND ...
            • NOT IN
            • NOT LIKE
            • COALESCE
            • Function_Interval
            • Function_Greatest
            • Function_Least
            • Function_Strcmp
            • Null Safe Equal
          • Flow Control Functions
            • CASE WHEN
            • IF
            • IFNULL
            • NULLIF
          • Logical Operators
            • AND,&&
            • NOT,!
            • OR
            • XOR
      • Functions and Operators
        • Aggregate Functions
          • AVG
          • APPROX_PERCENTILE
          • BITMAP
          • BIT_AND
          • BIT_OR
          • BIT_XOR
          • COUNT
          • GROUP_CONCAT
          • HLL_ADD_AGG
          • HLL_CARDINALITY
          • HLL_MERGE_AGG
          • MAX
          • MAX_BY
          • MAX_BY_NON_NULL
          • MEDIAN
          • MIN
          • STDDEV_POP
          • SUM
          • VARIANCE
          • VAR_POP
        • Datetime
          • CURDATE()
          • CURRENT_TIMESTAMP()
          • DATE()
          • DATE_ADD()
          • DATE_FORMAT()
          • DATE_SUB()
          • DATE_TRUNC()
          • DATEDIFF()
          • DAY()
          • DAYOFYEAR()
          • EXTRACT()
          • HOUR()
          • FROM_UNIXTIME
          • MINUTE()
          • MONTH()
          • NOW()
          • SECOND()
          • STR_TO_DATE()
          • SYSDATE()
          • TIME()
          • TIMEDIFF()
          • TIMESTAMP()
          • TIMESTAMPDIFF()
          • TO_DATE()
          • TO_DAYS()
          • TO_SECONDS()
          • UNIX_TIMESTAMP
          • UTC_TIMESTAMP()
          • WEEK()
          • WEEKDAY()
          • YEAR()
          • Addtime
          • Curtime
          • Dayname
          • Get Format
          • Maketime
          • Monthname
          • Quarter
          • Subtime
          • Time Format
          • Timestampadd
          • Yearweek
        • Geo Functions
          • H3 Index Functions
          • MBRContains
          • MBRCoveredBy
          • MBRCovers
          • MBRDisjoint
          • MBREquals
          • MBRIntersects
          • MBROverlaps
          • MBRTouches
          • MBRWithin
          • S2 Cell Functions
          • ST_Area()
          • ST_AsGeoJSON
          • ST_AsText()
          • ST_AsWKB()
          • ST_Boundary()
          • ST_Buffer()
          • ST_Centroid()
          • ST_Collect()
          • ST_Contains()
          • ST_ConvexHull()
          • ST_CoveredBy()
          • ST_Covers()
          • ST_Crosses()
          • ST_Difference()
          • ST_Dimension()
          • ST_Disjoint()
          • ST_Distance()
          • ST_Distance_Sphere()
          • ST_EndPoint()
          • ST_Envelope()
          • ST_Equals()
          • ST_ExteriorRing()
          • ST_FrechetDistance()
          • ST_GeoHash
          • ST_GeomCollFromText()
          • ST_GeomCollFromWKB()
          • ST_GeometryN()
          • ST_GeometryType()
          • ST_GeomFromGeoJSON
          • ST_GeomFromText()
          • ST_GeomFromWKB()
          • ST_HausdorffDistance()
          • ST_InteriorRingN()
          • ST_Intersection()
          • ST_Intersects()
          • ST_IsClosed()
          • ST_IsCollection()
          • ST_IsEmpty()
          • ST_IsRing()
          • ST_IsSimple()
          • ST_IsValid()
          • ST_LatFromGeoHash
          • ST_Latitude()
          • ST_Length()
          • ST_LineFromText()
          • ST_LineFromWKB()
          • ST_LineInterpolatePoint()
          • ST_LineInterpolatePoints()
          • ST_LongFromGeoHash
          • ST_Longitude()
          • ST_MakeEnvelope()
          • ST_MLineFromText()
          • ST_MLineFromWKB()
          • ST_MPointFromText()
          • ST_MPointFromWKB()
          • ST_MPolyFromText()
          • ST_MPolyFromWKB()
          • ST_NumGeometries()
          • ST_NumInteriorRings()
          • ST_NumPoints()
          • ST_Overlaps()
          • ST_PointAtDistance()
          • ST_PointFromGeoHash
          • ST_PointFromText()
          • ST_PointFromWKB()
          • ST_PointN()
          • ST_PointOnSurface()
          • ST_PolyFromText()
          • ST_PolyFromWKB()
          • ST_Simplify()
          • ST_SRID()
          • ST_StartPoint()
          • ST_SwapXY()
          • ST_SymDifference()
          • ST_Touches()
          • ST_Union()
          • ST_Validate()
          • ST_Within()
          • ST_X()
          • ST_Y()
        • Mathematical
          • ACOS()
          • ATAN()
          • BIT_COUNT()
          • CEIL()
          • CEILING()
          • COS()
          • COT()
          • CRC32()
          • EXP()
          • FLOOR()
          • LN()
          • LOG()
          • LOG2()
          • LOG10()
          • PI()
          • POWER()
          • ROUND()
          • RAND()
          • SIN()
          • SINH()
          • TAN()
          • Atan2
          • Degrees
          • Radians
          • Sign
          • Truncate
        • String
          • BIT_LENGTH()
          • CHAR_LENGTH()
          • CONCAT()
          • CONCAT_WS()
          • EMPTY()
          • ENDSWITH()
          • FIELD()
          • FIND_IN_SET()
          • FORMAT()
          • FROM_BASE64()
          • HEX()
          • INSTR()
          • LCASE()
          • LEFT()
          • LENGTH()
          • LOCATE()
          • LOWER()
          • LPAD()
          • LTRIM()
          • MD5()
          • NAME_CONST()
          • OCT()
          • REPEAT()
          • REVERSE()
          • RPAD()
          • RTRIM()
          • SHA1()/SHA()
          • SHA2()
          • SPACE()
          • SPLIT_PART()
          • STARTSWITH()
          • STRCMP()
          • SUBSTRING()
          • SUBSTRING_INDEX()
          • TO_BASE64()
          • TRIM()
          • UCASE()
          • UNHEX()
          • UPPER()
          • Regular Expressions
            • NOT REGEXP
            • REGEXP_INSTR()
            • REGEXP_LIKE()
            • REGEXP_REPLACE()
            • REGEXP_SUBSTR()
          • Aes_Decrypt
          • Aes_Encrypt
          • Elt
          • Quote
          • Right
        • Vector
          • Mathematical Calculations
          • CLUSTER_CENTERS()
          • COSINE_SIMILARITY()
          • COSINE_DISTANCE()
          • INNER_PRODUCT()
          • L1_NORM()
          • L1_DISTANCE() (unavailable)
          • L2_NORM()
          • L2_DISTANCE()
          • L2_DISTANCE_SQ()
          • NORMALIZE_L2()
          • SUBVECTOR()
          • VECTOR_DIMS()
        • Table
          • UNNEST()
          • SQL_TVF()
          • ESQL_TVF()
        • Window Functions
          • RANK()
          • ROW_NUMBER()
          • Cume_Dist
          • Percent_Rank
        • JSON Functions
          • JSON_EXTRACT()
          • JSON_EXTRACT_FLOAT64()
          • JSON_EXTRACT_STRING()
          • JSON_QUOTE()
          • JSON_ROW()
          • JSON_SET()
          • JSON_UNQUOTE()
          • TRY_JQ()
          • Json Arrow
          • JSON_ARRAY()
            • JSON_CONTAINS()
            • JSON_CONTAINS_PATH()
            • JSON_KEYS()
            • JSON_LENGTH()
            • JSON_MERGE_PATCH()
            • JSON_MERGE_PRESERVE()
            • JSON_OBJECT()
            • JSON_OVERLAPS()
            • JSON_PRETTY()
            • JSON_REMOVE()
            • JSON_SCHEMA_VALID()
            • JSON_SCHEMA_VALIDATION_REPORT()
            • JSON_TYPE()
            • JSON_VALID()
            • JSON_VALUE()
          • JSON_KEYS()
          • JSON_LENGTH()
          • JSON_OBJECT()
          • JSON_PRETTY()
          • JSON_SCHEMA_VALID()
          • JSON_SCHEMA_VALIDATION_REPORT()
          • JSON_TYPE()
          • JSON_VALID()
          • JSON_VALUE()
        • Other Functions
          • SAVE_FILE()
          • SAMPLE()
          • SERIAL_EXTRACT()
          • SLEEP()
          • STAGE_LIST()
          • ONNX_RUN()
          • UUID()
          • FOREIGN CONNECTION FUNCTIONS
          • LAST_KAFKA_MESSAGE_ID()
        • System OPS Functions
          • CURRENT_ROLE()
          • CURRENT_USER_NAME()
          • CURRENT_USER()
          • PURGE_LOG()
          • GET_LOCK()
          • RELEASE_LOCK()
          • IS_FREE_LOCK()
          • IS_USED_LOCK()
          • RELEASE_ALL_LOCKS()
          • Version
      • System Paramaters
        • Standalone Common Parameters Configuration
        • Distributed Common Parameters Configuration
      • MatrixOne Catalog
      • Privilege Control Types
      • Limitations
        • Partitioning supported features list
      • MatrixOne Directory Structure
      • MatrixOne tools index
      • Task-based tool selection
        • mo_ctl Standalone Tools Guide
        • Standalone mo_ctl scheduled operations
        • mo_ctl Distributed Edition Tool Guide
        • mo-service profiling flags
        • mo-tool advanced diagnostics and offline inspection
        • mo_datax_writer Tool Guide
        • mo_ssb_open Tool Guide
        • mo_tpch_open Tool Guide
        • mo_ts_perf_test Tool Guide
      • Mysql Compatibility Matrix
      • Mysql Unsupported Features
    • Troubleshooting
      • Common statistic data query
      • Database statistics
      • Error Code
    • FAQs
      • Deployment FAQs
      • SQL FAQs
    • Release Notes
    • Glossary
    • Contribution Guide
      • How to Contribute
        • Preparation
          • Kernel developer reading and verification guide
        • Report an Issue
        • Contribute Code
        • Review a Pull Request
        • Contribute Documentation
        • Make a Design
      • Code Style
        • Code Comment Style
        • Commit & Pull Request Style
    • Capability map and product boundaries
    • GIS spatial analysis
  • Get Started
    • Deploy on macOS
      • Using binary package
      • Using Docker
    • Deploy on Linux
      • Using binary package
      • Using Docker
    • Basic SQL
    • Installation choices and acceptance
  • Application development
    • Connections and drivers
      • Java connect to MatrixOne
        • Connect MatrixOne with Java ORMs
      • Python connect to MatrixOne
      • General Python SDK
      • C# connect to MatrixOne
      • Connecting to MatrixOne with Golang
      • MatrixOne SSL connection
      • Connecting to MatrixOne with TypeScript
      • Connecting to MatrixOne with ODBC
      • Connecting to MatrixOne with Power BI
      • Lakehouse Ecosystem Tutorials
        • Prepare connections, fixtures and acceptance checks
        • Batch loading with pandas, PETL, Polars and Dask
        • PyArrow IPC and DuckDB Parquet bridges to MatrixOne
        • PyIceberg append/scan with an explicit SQL bridge
        • Airflow MySqlHook DAG and idempotent retries
        • dbt-mysql model builds and data tests
        • DolphinScheduler Shell and native SQL tasks
        • Local testing with the official Fivetran Connector SDK
        • Airbyte connector protocol and destination acceptance
        • NiFi Record flows and MatrixOne JDBC writes
        • Kafka Connect, Debezium and JDBC CDC sinks
        • Exact event loading with Redpanda Connect and kafka-python
        • Read fidelity with ConnectorX, Ibis and database drivers
        • dlt loading and Singer state-protocol adapters
        • Job acceptance for DataX, SeaTunnel, Spark, Flink and Trino
        • Native DataX source scans and exact-value writes
        • Native Spark JDBC reads, writes, types and partitions
        • Native Flink MySQL CDC into MatrixOne and recovery acceptance
      • Schema Design
        • Create Database
        • Create Table
        • Replication table
        • Create View
        • Create Temporary Table
        • Create Secondary Index
        • Vector
        • Data Integrity
          • NOT NULL Constraints
          • UNIQUE KEY Constraints
          • PRIMARY KEY Constraints
          • FOREIGN KEY Constraints
          • AUTO INCREMENT Constraints
      • Write Data
        • Bulk Load
          • Load csv format data
          • Load jsonlines format data
          • Load data from S3
          • Load data by using the `source`
        • Update Data
        • DML Target and Conflict Semantics
        • Delete Data
        • Prepared
      • Export Data
        • Export data by MODUMP
      • Read Data
        • Multi-table Join Queries
        • Subquery
        • Views
        • Common Table Expression
        • Window Function
          • Time Window
          • Analytical Windows
      • Data de-duplication
        • BITMAP
      • Account Design
        • Publish-Subscribe
      • Transactions
        • Transaction by MatrixOne Server
          • Explicit Transaction
          • Implicit Transaction
          • Pessimistic Transaction
          • Optimistic Transaction
          • Isolation Level
          • MVCC
          • User Guide
            • Scenario
          • Scenario
      • User-defined function
        • UDF python advanced
      • Vector
        • Vector Search
        • IVF Rank Options
        • Query-level IVF Probe Count
        • Cluster Centers
        • Scalar Vector Query
        • IVF Equi-join Vector Query
      • Data Type Fidelity
      • Ecological Tools
        • BI integration guides and compatibility
          • Shared BI dataset and KPI checks
          • Connect Metabase and build metrics
          • Superset analytic connection checks
          • Grafana MySQL datasource and compatibility checks
          • Connect Redash and save analytic queries
          • Connect DataEase and preview SQL datasets
          • Evidence MySQL source and compatibility checks
          • Rill ingestion and native metrics
          • Apache Zeppelin JDBC notebooks
          • SQLPad saved SQL analyses
          • Lightdash direct-connection boundary
        • Visualizing MatrixOne Reports with Yonghong BI
        • Visual Monitoring of MatrixOne with Superset
        • ETL Tools
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Using DataX to write data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from ClickHouse to MatrixOne
            • Writing Data from Doris to MatrixOne
            • Writing Data from InfluxDB to MatrixOne
            • Writing Data from Elasticsearch to MatrixOne
        • Computing Engine
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Hive to MatrixOne
          • Writing Data from Doris to MatrixOne
          • Using Flink to Write Real-Time Data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from Kafka to MatrixOne
        • Scheduling Tools
      • Ecosystem Tools Overview
        • DataX
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Writing Data from PostgreSQL to MatrixOne
          • Writing Data from SQL Server to MatrixOne
          • Writing Data from MongoDB to MatrixOne
          • Writing Data from TiDB to MatrixOne
          • Writing Data from ClickHouse to MatrixOne
          • Writing Data from Doris to MatrixOne
          • Writing Data from InfluxDB to MatrixOne
          • Writing Data from Elasticsearch to MatrixOne
        • SeaTunnel
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Using DataX to write data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from ClickHouse to MatrixOne
            • Writing Data from Doris to MatrixOne
            • Writing Data from InfluxDB to MatrixOne
            • Writing Data from Elasticsearch to MatrixOne
        • Spark
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Hive to MatrixOne
          • Writing Data from Doris to MatrixOne
          • Using Flink to Write Real-Time Data to MatrixOne
            • Writing Data from MySQL to MatrixOne
            • Writing Data from Oracle to MatrixOne
            • Writing Data from SQL Server to MatrixOne
            • Writing Data from PostgreSQL to MatrixOne
            • Writing Data from MongoDB to MatrixOne
            • Writing Data from TiDB to MatrixOne
            • Writing Data from Kafka to MatrixOne
        • Flink
          • Writing Data from MySQL to MatrixOne
          • Writing Data from Oracle to MatrixOne
          • Writing Data from SQL Server to MatrixOne
          • Writing Data from PostgreSQL to MatrixOne
          • Writing Data from MongoDB to MatrixOne
          • Writing Data from TiDB to MatrixOne
          • Writing Data from Kafka to MatrixOne
        • BI integrations
          • Shared BI dataset and KPI checks
          • Connect Metabase and build metrics
          • Superset analytic connection checks
          • Grafana MySQL datasource and compatibility checks
          • Connect Redash and save analytic queries
          • Connect DataEase and preview SQL datasets
          • Evidence MySQL source and compatibility checks
          • Rill ingestion and native metrics
          • Apache Zeppelin JDBC notebooks
          • SQLPad saved SQL analyses
          • Lightdash direct-connection boundary
        • CDC
          • mo_cdc command reference
            • From MatrixOne to MySQL
            • From MatrixOne to MatrixOne
          • MatrixOne to MySQL
          • MatrixOne to MatrixOne
        • S3 and MinIO
          • Load from S3
          • Import from MinIO
          • Deploy with S3 storage
            • Plan MatrixOne Cluster Topology
              • Experience Environment Deployment Plan
              • Minimum Production Environment Deployment Plan
              • Recommended Production Environment Deployment Plan
              • Production readiness and acceptance
            • Cluster Deployment Guide
              • Deployed Kubernetes and object storage environment
            • Cluster Operations Management
              • Updating
              • Health check and resource monitoring
              • Scaling
              • Managing CN Groups with Proxy
              • Import data from local Minio to MatrixOne
              • Operator Management
                • Kubernetes Operator acceptance operations
            • Deploy Matrixone Cluster
        • Lakehouse ecosystem tutorials
          • Prepare connections, fixtures and acceptance checks
          • Batch loading with pandas, PETL, Polars and Dask
          • PyArrow IPC and DuckDB Parquet bridges to MatrixOne
          • PyIceberg append/scan with an explicit SQL bridge
          • Airflow MySqlHook DAG and idempotent retries
          • dbt-mysql model builds and data tests
          • DolphinScheduler Shell and native SQL tasks
          • Local testing with the official Fivetran Connector SDK
          • Airbyte connector protocol and destination acceptance
          • NiFi Record flows and MatrixOne JDBC writes
          • Kafka Connect, Debezium and JDBC CDC sinks
          • Exact event loading with Redpanda Connect and kafka-python
          • Read fidelity with ConnectorX, Ibis and database drivers
          • dlt loading and Singer state-protocol adapters
          • Job acceptance for DataX, SeaTunnel, Spark, Flink and Trino
          • Native DataX source scans and exact-value writes
          • Native Spark JDBC reads, writes, types and partitions
          • Native Flink MySQL CDC into MatrixOne and recovery acceptance
        • Workflow orchestration
          • DolphinScheduler
        • Application, AI and data development
          • Dify Platform Integration Guide for MatrixOne
          • Prerequisites
          • Steps
          • RAG application foundation
          • Git for Data
        • Database administration, migration and backup tools
          • mo_ctl Distributed Edition
          • mo_ctl Standalone Edition
          • Migration overview
            • MySQL
            • Data migration compatibility cookbook
            • Oracle
              • Heterogeneous type acceptance
            • SQL Server
            • PostgreSQL
            • Reconciliation and cutover
          • Data migration compatibility cookbook
          • MySQL migration guide
          • Oracle migration guide
            • Heterogeneous type acceptance
          • PostgreSQL migration guide
          • SQL Server migration guide
          • mo-dump export utility
          • mo-dump backup and restore
          • mo_br backup and recovery
            • mo_br regular physical backup recovery
            • mo_br snapshot backup recovery
            • mo_br pitr
          • Backup and recovery overview
            • Backup and Recovery Concepts
            • Backup and Restore by using mo-dump
            • mo_br Backup and Recovery
              • mo_br regular physical backup recovery
              • mo_br snapshot backup recovery
              • mo_br pitr
            • MatrixOne active/standby disaster recovery
            • cdc
              • From MatrixOne to MySQL
              • From MatrixOne to MatrixOne
            • Mount Data
            • DECIMAL256 index upgrade
            • Recovery selection and acceptance
            • Operations overview
      • Develop Overview
      • AI Agent Tools
        • Query MatrixOne Documentation with AI Agent
      • Qualify an Ecosystem Integration
      • Failure Recovery
      • Ecosystem tools index
        • ETL Tools Index
        • Compute Engines Index
        • BI Tools Index
      • Data import index
      • Data export index
      • Vector search index
  • Tutorial
    • Tools and Integrations
      • AI and Search
        • Vector Search
          • Pinecone-Compatible Vector Search
          • IVF Index Health Monitoring
          • HNSW Vector Index
          • Filtered Vector Top-K
          • Search with a Saved Query Vector
          • Vector Search with an Equality JOIN
          • Vector Search Reference
        • Full-Text Search
          • Natural Language Full-Text Search (Python SDK)
          • Boolean Full-Text Search (Python SDK)
          • JSON Full-Text Search (Python SDK)
          • FULLTEXT2 Phrase and Boolean Search (SQL)
          • JSONLine to JSON Path Search (SQL)
          • Full-Text Index SQL Reference
        • Hybrid Search and Analytics
          • Hybrid Search: Vector, Full-Text, and SQL
          • Search and Analytics SQL Recipes
            • FULLTEXT2 phrase and Boolean search
            • Load JSONLine and query JSON paths
            • Filtered vector Top-K search
            • Search with a stored query vector
            • Search vectors through an equi-join
            • Fill missing time windows for device readings
        • AI Applications and Integrations
          • RAG Application Demo
          • Picture and Text-to-Picture Search Demo
          • Dify Platform Integration Guide for MatrixOne
          • Prerequisites
          • Steps
      • Data Workflows
        • External Data and Files Index
          • External Data Sources and Connectors
            • Read a Bounded Set from MongoDB
            • Read and Resume a Kafka Partition
            • Compare Filter Pushdown for Remote SQL
            • Read a File or JDBC Source through DataStream
          • File Loading, Stage, and Table Export
            • Write to Stage and Read through an External Table
            • Load Multiple and Compressed Files
            • Route Malformed Rows to a Rejects Table
            • Restore a Table through Stage with DUMP/LOAD TABLE
        • Git4Data Demo
      • Iceberg
        • Iceberg orders tutorial
        • Delete and verify Iceberg rows
      • Data Management and SQL
        • Application SQL Recipes
          • Save and reuse a query result
          • Return the final row after DML
          • Reject invalid orders with CHECK
          • Classify SQL errors in a Starlark procedure
        • Data Branching and Sharing
          • Persist and Review Data Branch Differences
          • Publish and Subscribe across Two Accounts
        • Backup and Recovery
          • Point-in-Time Recovery Drill
          • Backup and Restore Guide
            • Backup and Recovery Concepts
            • Backup and Restore by using mo-dump
            • mo_br Backup and Recovery
              • mo_br regular physical backup recovery
              • mo_br snapshot backup recovery
              • mo_br pitr
            • MatrixOne active/standby disaster recovery
            • cdc
              • From MatrixOne to MySQL
              • From MatrixOne to MatrixOne
            • Mount Data
            • DECIMAL256 index upgrade
            • Recovery selection and acceptance
            • Operations overview
      • Spatial and GIS
        • GIS Types and Data Import
        • GIS Geofences and Regional Analysis
        • GIS Nearby-Store Distance Ranking
        • Geofence and Distance SQL Recipe
    • Application and Scenario Examples
      • Python Application Examples
        • Python CRUD Demo
          • Separate public mo-vector package lab
        • SQLAlchemy CRUD Demo
        • Django CRUD Demo
      • Java Application Examples
        • Java JDBC CRUD Demo
        • SpringBoot and JPA CRUD Demo
        • SpringBoot and MyBatis CRUD Demo
      • Other Language Application Examples
        • Golang CRUD Demo
        • Gorm CRUD Demo
        • C# CRUD Demo
        • TypeScript Basic Example
      • HTAP Application demo
      • Analytics and Machine Learning
        • pandas and NumPy data handling
        • Point-in-time features and revisions
        • Multi-table analysis without double counting
        • Calendar and rolling features
        • Spark and Flink analytical pipelines
        • Model weights, preprocessing and prediction readback
        • Window statistics migration
        • Ten business-analysis seeds
        • Measurement and troubleshooting
      • Safe Production Upgrade with Instant Rollback
      • Instant Clone for Multi-Team Development
    • SDK and Development Basics
      • General Python SDK basics
      • Separate vector SDK
      • Python Data Science and Machine Learning
        • Data Science with the Python SDK
          • SDK, pandas and SQL exploration
          • SDK and scikit-learn: train, evaluate and write back
        • DataFrame Operations with the SDK
        • Machine Learning Workflows with the SDK
    • Data Workflows and Use Cases
      • Native SQL Task experiments
      • MySQL migration and reconciliation lab
      • Stored procedures: parameters and rollback
    • Database Labs
      • Database Recovery Labs
        • Updates, Deletes, and Protected History
        • Restart Recovery of Committed Data
      • Query and Transaction Labs
        • Sorting, JOIN, and Spill
        • From JOIN SQL to an Execution Plan
        • Statement Errors, Rollback, and Two-Session Isolation
        • Commit and Visibility across Two CNs
  • Deploy
    • Plan MatrixOne Cluster Topology
      • Experience Environment Deployment Plan
      • Minimum Production Environment Deployment Plan
      • Recommended Production Environment Deployment Plan
      • Production readiness and acceptance
    • Cluster Deployment Guide
      • Deployed Kubernetes and object storage environment
    • Cluster Operations Management
      • Updating
      • Health check and resource monitoring
      • Scaling
      • Managing CN Groups with Proxy
      • Import data from local Minio to MatrixOne
      • Operator Management
        • Kubernetes Operator acceptance operations
    • Deploy Matrixone Cluster
  • Operations
    • Backup and Recovery Concepts
    • Backup and Restore by using mo-dump
    • mo_br Backup and Recovery
      • mo_br regular physical backup recovery
      • mo_br snapshot backup recovery
      • mo_br pitr
    • MatrixOne active/standby disaster recovery
    • cdc
      • From MatrixOne to MySQL
      • From MatrixOne to MatrixOne
    • Mount Data
    • DECIMAL256 index upgrade
    • Recovery selection and acceptance
    • Operations overview
  • Migrate
    • MySQL
    • Data migration compatibility cookbook
    • Oracle
      • Heterogeneous type acceptance
    • SQL Server
    • PostgreSQL
    • Reconciliation and cutover
  • Test
    • TPCH Test with MatrixOne
    • TPCC Test with MatrixOne
    • Testing Tool
      • MO-Tester Specification
  • Performance
    • Understanding the Query Execution Plan
      • Using EXPLAIN to learn the execution plan
      • Explain Statements Using JOIN
      • Explain Statements Using Subqueries
      • Explain Statements Using Aggregation
      • EXPLAIN Statements Using Views
    • Performance tuning best practices
      • Scaling CN for better performance
      • Partition Pruning
      • Usage scenarios of Partition Pruning in KEY Partitioned Tables
      • Usage scenarios of Partition Pruning in HASH Partitioned Tables
      • Performance Tuning Examples for Partition Pruning
      • Constraints
      • Performance Tuning with Partitioned Tables
    • Optimizer Hints
  • Security
    • Authentication and Authorization
    • Password Management
    • Access Control
      • Privilege Management Scenario
      • Best Practices
      • SQL File Access
      • Role Grant Consistency
    • SQL File Access
    • User Guide
      • Create accounts, Verify Resource Isolation
      • Use the new account to creates users, roles, grant the privilege
    • Data Transmission Encryption
    • Security Audit
  • Reference
    • System Variables parameters
      • Save query result support
      • Timezone support
      • Lower case table names support
      • Foreign key checking support
      • User-specified case consistency support for query result set column names
      • Illegal login restrictions
      • Password complexity verification
      • Connection whitelist
      • Enable remap hint
    • cte_max_memory_bytes
    • cte_max_recursion_depth
    • enable_explain_scheduling
    • event_scheduler
    • experimental_cagra_index
    • experimental_fulltext_index
    • experimental_fulltext2_index
    • experimental_hnsw_index
    • experimental_ivf_index
    • experimental_ivfpq_index
    • fulltext_bloom_filter_pushdown
    • ft2_relevancy_algorithm
    • group_concat_max_len
    • lock_wait_timeout
    • mo_rollback_txn_on_error
    • protected_databases
    • query_max_workers
    • query_pool_strict
    • remap_rewrites
    • remapdb
    • sort_spill_mem
    • Custom variable
    • SQL Language Structure
      • Comments
    • Data Types
      • Data Type Conversion
      • Date and Time Types
        • YEAR Type
      • Geometry Type
      • JSON Data Type
      • BLOB and TEXT Type
      • DATALINK Type
      • ENUM Type
      • UUID Type
      • VECTOR Type
      • Fixed-Point Types (Exact Value) - DECIMAL
      • Set Type
    • SQL Statements
      • Data Definition Language
        • CREATE INDEX
        • CREATE INDEX...USING IVFFLAT
        • CREATE INDEX...USING HNSW
        • CREATE FULLTEXT INDEX
        • CREATE TABLE
        • CREATE TABLE AS SELECT
        • CREATE TABLE ... LIKE
        • CREATE EXTERNAL TABLE
        • CREATE EXTERNAL TABLE ... ENGINE = DATASTREAM
        • CREATE EXTERNAL TABLE ... ENGINE = SQL|ESQL
        • CREATE EXTERNAL TABLE ... ENGINE = KAFKA
        • External Table Error Mode
        • CREATE FULLTEXT2 INDEX
        • CREATE ICEBERG CATALOG
        • ALTER ICEBERG CATALOG
        • DROP ICEBERG CATALOG
        • CREATE MONGODB CONNECTION
        • ALTER MONGODB CONNECTION
        • DROP MONGODB CONNECTION
        • CREATE CLUSTER TABLE
        • CREATE CLONE
        • CREATE PITR
        • CREATE PUBLICATION
        • CREATE SEQUENCE
        • CREATE STAGE
        • CREATE...FROM...PUBLICATION...
        • CREATE VIEW
        • CREATE FUNCTION...LANGUAGE SQL AS
        • CREATE FUNCTION...LANGUAGE PYTHON AS
        • CREATE SOURCE
        • CREATE DYNAMIC TABLE
        • CREATE SNAPSHOT
        • CREATE BRANCH
        • DELETE BRANCH
        • DIFF BRANCH
        • MERGE BRANCH
        • ALTER TABLE
        • ALTER TABLE ... ALTER REINDEX
        • ALTER PITR
        • ALTER PUBLICATION
        • ALTER SEQUENCE
        • ALTER STAGE
        • ALTER VIEW
        • DROP DATABASE
        • DROP INDEX
        • DROP TABLE
        • DROP PITR
        • DROP PUBLICATION
        • DROP SEQUENCE
        • DROP STAGE
        • DROP SNAPSHOT
        • DROP VIEW
        • DROP FUNCTION
        • TRUNCATE TABLE
        • RENAME TABLE
        • RESTORE PITR
        • RESTORE SNAPSHOT
        • Branch Protect Snapshots
        • Data Branch Pick
        • Sql Task
        • Data Branch Privilege
        • CREATE PROCEDURE / CALL / DROP PROCEDURE
      • Data Manipulation Language
        • INSERT INTO SELECT
        • DELETE
        • UPDATE
        • LOAD DATA INFILE
        • LOAD DATA INLINE
        • UPSERT
          • INSERT ON DUPLICATE KEY UPDATE
          • INSERT IGNORE
          • REPLACE
        • MERGE
        • RETURNING
        • Information Functions
          • LAST_INSERT_ID()
          • ROW_COUNT()
          • Current_Role
        • Case
        • Replace
      • Data Query Language
        • OUTER APPLY
        • JOIN
          • INNER JOIN
          • LEFT JOIN
          • RIGHT JOIN
          • FULL JOIN
          • OUTER JOIN
          • Examples
          • NATURAL JOIN
          • Cross Join
        • SELECT
        • BY RANK WITH OPTION
        • SUBQUERY
          • Derived Tables
          • Comparisons Using Subqueries
          • SUBQUERY with ANY or SOME
          • SUBQUERY with ALL
          • SUBQUERY with EXISTS
          • SUBQUERY with IN
        • With CTE
        • Combining Queries
          • UNION
          • INTERSECT
          • MINUS
      • Data Control Language
        • ALTER ACCOUNT
        • CREATE ROLE
        • CREATE USER
        • ALTER USER
        • DROP ACCOUNT
        • DROP USER
        • DROP ROLE
        • GRANT
        • REVOKE
        • Role Rule
      • Other
        • SHOW DATABASES
        • SHOW CREATE TABLE
        • SHOW CREATE VIEW
        • SHOW CREATE PUBLICATION
        • SHOW TABLES
        • SHOW INDEX
        • SHOW COLLATION
        • SHOW COLUMNS
        • SHOW FUNCTION STATUS
        • SHOW GRANT
        • SHOW PROCESSLIST
        • SHOW PUBLICATIONS
        • SHOW PITRS
        • SHOW ROLES
        • SHOW SEQUENCES
        • SHOW STAGE
        • SHOW SUBSCRIPTIONS
        • SHOW VARIABLES
        • SHOW ICEBERG CATALOGS
        • SHOW ICEBERG NAMESPACES
        • SHOW ICEBERG TABLES
        • SHOW MONGODB CONNECTIONS
        • SHOW PROFILE
        • Show Create Database
        • Show Table Status
        • SET
        • USE
          • ANALYZE TABLE
          • DUMP TABLE
          • LOAD TABLE
          • PERFORM
          • Starlark Procedure Error Reporting
        • KILL
        • Prepared Statements
          • EXECUTE
          • DEALLOCATE
        • Explain
          • EXPLAIN Output Format
          • Explain Analyze
          • Explain Prepared
        • Describe
        • ANALYZE TABLE
        • DUMP TABLE
        • LOAD TABLE
        • PERFORM
        • Starlark Procedure Error Reporting
    • Operators
      • OPERATORS
        • OPERATORS Precedence
        • Arithmetic Operators
          • %,MOD
          • *
          • +
          • -
          • -
          • /
          • DIV
        • Assignment Operators
          • =
        • Bit Functions and Operators
          • &
          • >>
          • <<
          • ^
          • |
          • ~
        • Cast Functions and Operators
          • BINARY
          • CAST
          • CONVERT
          • DECODE
          • ENCODE
          • SERIAL
          • SERIAL_FULL
        • Comparison Functions and Operators
          • >
          • >=
          • <
          • <>,!=
          • <=
          • =
          • BETWEEN ... AND ...
          • IN
          • IS
          • IS NOT
          • IS NOT NULL
          • IS NULL
          • ILIKE
          • ISNULL
          • LIKE
          • NOT BETWEEN ... AND ...
          • NOT IN
          • NOT LIKE
          • COALESCE
          • Function_Interval
          • Function_Greatest
          • Function_Least
          • Function_Strcmp
          • Null Safe Equal
        • Flow Control Functions
          • CASE WHEN
          • IF
          • IFNULL
          • NULLIF
        • Logical Operators
          • AND,&&
          • NOT,!
          • OR
          • XOR
    • Functions and Operators
      • Aggregate Functions
        • AVG
        • APPROX_PERCENTILE
        • BITMAP
        • BIT_AND
        • BIT_OR
        • BIT_XOR
        • COUNT
        • GROUP_CONCAT
        • HLL_ADD_AGG
        • HLL_CARDINALITY
        • HLL_MERGE_AGG
        • MAX
        • MAX_BY
        • MAX_BY_NON_NULL
        • MEDIAN
        • MIN
        • STDDEV_POP
        • SUM
        • VARIANCE
        • VAR_POP
      • Datetime
        • CURDATE()
        • CURRENT_TIMESTAMP()
        • DATE()
        • DATE_ADD()
        • DATE_FORMAT()
        • DATE_SUB()
        • DATE_TRUNC()
        • DATEDIFF()
        • DAY()
        • DAYOFYEAR()
        • EXTRACT()
        • HOUR()
        • FROM_UNIXTIME
        • MINUTE()
        • MONTH()
        • NOW()
        • SECOND()
        • STR_TO_DATE()
        • SYSDATE()
        • TIME()
        • TIMEDIFF()
        • TIMESTAMP()
        • TIMESTAMPDIFF()
        • TO_DATE()
        • TO_DAYS()
        • TO_SECONDS()
        • UNIX_TIMESTAMP
        • UTC_TIMESTAMP()
        • WEEK()
        • WEEKDAY()
        • YEAR()
        • Addtime
        • Curtime
        • Dayname
        • Get Format
        • Maketime
        • Monthname
        • Quarter
        • Subtime
        • Time Format
        • Timestampadd
        • Yearweek
      • Geo Functions
        • H3 Index Functions
        • MBRContains
        • MBRCoveredBy
        • MBRCovers
        • MBRDisjoint
        • MBREquals
        • MBRIntersects
        • MBROverlaps
        • MBRTouches
        • MBRWithin
        • S2 Cell Functions
        • ST_Area()
        • ST_AsGeoJSON
        • ST_AsText()
        • ST_AsWKB()
        • ST_Boundary()
        • ST_Buffer()
        • ST_Centroid()
        • ST_Collect()
        • ST_Contains()
        • ST_ConvexHull()
        • ST_CoveredBy()
        • ST_Covers()
        • ST_Crosses()
        • ST_Difference()
        • ST_Dimension()
        • ST_Disjoint()
        • ST_Distance()
        • ST_Distance_Sphere()
        • ST_EndPoint()
        • ST_Envelope()
        • ST_Equals()
        • ST_ExteriorRing()
        • ST_FrechetDistance()
        • ST_GeoHash
        • ST_GeomCollFromText()
        • ST_GeomCollFromWKB()
        • ST_GeometryN()
        • ST_GeometryType()
        • ST_GeomFromGeoJSON
        • ST_GeomFromText()
        • ST_GeomFromWKB()
        • ST_HausdorffDistance()
        • ST_InteriorRingN()
        • ST_Intersection()
        • ST_Intersects()
        • ST_IsClosed()
        • ST_IsCollection()
        • ST_IsEmpty()
        • ST_IsRing()
        • ST_IsSimple()
        • ST_IsValid()
        • ST_LatFromGeoHash
        • ST_Latitude()
        • ST_Length()
        • ST_LineFromText()
        • ST_LineFromWKB()
        • ST_LineInterpolatePoint()
        • ST_LineInterpolatePoints()
        • ST_LongFromGeoHash
        • ST_Longitude()
        • ST_MakeEnvelope()
        • ST_MLineFromText()
        • ST_MLineFromWKB()
        • ST_MPointFromText()
        • ST_MPointFromWKB()
        • ST_MPolyFromText()
        • ST_MPolyFromWKB()
        • ST_NumGeometries()
        • ST_NumInteriorRings()
        • ST_NumPoints()
        • ST_Overlaps()
        • ST_PointAtDistance()
        • ST_PointFromGeoHash
        • ST_PointFromText()
        • ST_PointFromWKB()
        • ST_PointN()
        • ST_PointOnSurface()
        • ST_PolyFromText()
        • ST_PolyFromWKB()
        • ST_Simplify()
        • ST_SRID()
        • ST_StartPoint()
        • ST_SwapXY()
        • ST_SymDifference()
        • ST_Touches()
        • ST_Union()
        • ST_Validate()
        • ST_Within()
        • ST_X()
        • ST_Y()
      • Mathematical
        • ACOS()
        • ATAN()
        • BIT_COUNT()
        • CEIL()
        • CEILING()
        • COS()
        • COT()
        • CRC32()
        • EXP()
        • FLOOR()
        • LN()
        • LOG()
        • LOG2()
        • LOG10()
        • PI()
        • POWER()
        • ROUND()
        • RAND()
        • SIN()
        • SINH()
        • TAN()
        • Atan2
        • Degrees
        • Radians
        • Sign
        • Truncate
      • String
        • BIT_LENGTH()
        • CHAR_LENGTH()
        • CONCAT()
        • CONCAT_WS()
        • EMPTY()
        • ENDSWITH()
        • FIELD()
        • FIND_IN_SET()
        • FORMAT()
        • FROM_BASE64()
        • HEX()
        • INSTR()
        • LCASE()
        • LEFT()
        • LENGTH()
        • LOCATE()
        • LOWER()
        • LPAD()
        • LTRIM()
        • MD5()
        • NAME_CONST()
        • OCT()
        • REPEAT()
        • REVERSE()
        • RPAD()
        • RTRIM()
        • SHA1()/SHA()
        • SHA2()
        • SPACE()
        • SPLIT_PART()
        • STARTSWITH()
        • STRCMP()
        • SUBSTRING()
        • SUBSTRING_INDEX()
        • TO_BASE64()
        • TRIM()
        • UCASE()
        • UNHEX()
        • UPPER()
        • Regular Expressions
          • NOT REGEXP
          • REGEXP_INSTR()
          • REGEXP_LIKE()
          • REGEXP_REPLACE()
          • REGEXP_SUBSTR()
        • Aes_Decrypt
        • Aes_Encrypt
        • Elt
        • Quote
        • Right
      • Vector
        • Mathematical Calculations
        • CLUSTER_CENTERS()
        • COSINE_SIMILARITY()
        • COSINE_DISTANCE()
        • INNER_PRODUCT()
        • L1_NORM()
        • L1_DISTANCE() (unavailable)
        • L2_NORM()
        • L2_DISTANCE()
        • L2_DISTANCE_SQ()
        • NORMALIZE_L2()
        • SUBVECTOR()
        • VECTOR_DIMS()
      • Table
        • UNNEST()
        • SQL_TVF()
        • ESQL_TVF()
      • Window Functions
        • RANK()
        • ROW_NUMBER()
        • Cume_Dist
        • Percent_Rank
      • JSON Functions
        • JSON_EXTRACT()
        • JSON_EXTRACT_FLOAT64()
        • JSON_EXTRACT_STRING()
        • JSON_QUOTE()
        • JSON_ROW()
        • JSON_SET()
        • JSON_UNQUOTE()
        • TRY_JQ()
        • Json Arrow
        • JSON_ARRAY()
          • JSON_CONTAINS()
          • JSON_CONTAINS_PATH()
          • JSON_KEYS()
          • JSON_LENGTH()
          • JSON_MERGE_PATCH()
          • JSON_MERGE_PRESERVE()
          • JSON_OBJECT()
          • JSON_OVERLAPS()
          • JSON_PRETTY()
          • JSON_REMOVE()
          • JSON_SCHEMA_VALID()
          • JSON_SCHEMA_VALIDATION_REPORT()
          • JSON_TYPE()
          • JSON_VALID()
          • JSON_VALUE()
        • JSON_KEYS()
        • JSON_LENGTH()
        • JSON_OBJECT()
        • JSON_PRETTY()
        • JSON_SCHEMA_VALID()
        • JSON_SCHEMA_VALIDATION_REPORT()
        • JSON_TYPE()
        • JSON_VALID()
        • JSON_VALUE()
      • Other Functions
        • SAVE_FILE()
        • SAMPLE()
        • SERIAL_EXTRACT()
        • SLEEP()
        • STAGE_LIST()
        • ONNX_RUN()
        • UUID()
        • FOREIGN CONNECTION FUNCTIONS
        • LAST_KAFKA_MESSAGE_ID()
      • System OPS Functions
        • CURRENT_ROLE()
        • CURRENT_USER_NAME()
        • CURRENT_USER()
        • PURGE_LOG()
        • GET_LOCK()
        • RELEASE_LOCK()
        • IS_FREE_LOCK()
        • IS_USED_LOCK()
        • RELEASE_ALL_LOCKS()
        • Version
    • System Paramaters
      • Standalone Common Parameters Configuration
      • Distributed Common Parameters Configuration
    • MatrixOne Catalog
    • Privilege Control Types
    • Limitations
      • Partitioning supported features list
    • MatrixOne Directory Structure
    • MatrixOne tools index
    • Task-based tool selection
      • mo_ctl Standalone Tools Guide
      • Standalone mo_ctl scheduled operations
      • mo_ctl Distributed Edition Tool Guide
      • mo-service profiling flags
      • mo-tool advanced diagnostics and offline inspection
      • mo_datax_writer Tool Guide
      • mo_ssb_open Tool Guide
      • mo_tpch_open Tool Guide
      • mo_ts_perf_test Tool Guide
    • Mysql Compatibility Matrix
    • Mysql Unsupported Features
  • Troubleshooting
    • Common statistic data query
    • Database statistics
    • Error Code
  • FAQs
    • Deployment FAQs
    • SQL FAQs
  • Release Notes
    • MatrixOne v26.4.2.8 Release Notes
    • MatrixOne v26.4.2.7 Release Notes
    • MatrixOne v26.4.2.6 Release Notes
    • MatrixOne v26.4.2.5 Release Notes
    • MatrixOne v26.4.2.4 Release Notes
    • MatrixOne v26.4.2.3 Release Notes
    • MatrixOne v26.4.2.2 Release Notes
    • MatrixOne v26.4.2.1 Release Notes
    • MatrixOne v26.4.2.0 Release Notes
    • MatrixOne v26.4.1.4 Release Notes
    • MatrixOne v26.4.1.3 Release Notes
    • MatrixOne v26.4.1.2 Release Notes
    • MatrixOne v26.4.1.1 Release Notes
    • MatrixOne v26.4.1.0 Release Notes
    • MatrixOne v26.4.0.0-rc4 Release Notes
    • MatrixOne v26.4.0.0-rc3 Release Notes
    • MatrixOne v26.4.0.0-rc2 Release Notes
    • MatrixOne v26.4.0.0-rc1 Release Notes
    • MatrixOne v26.3.0.10 Release Notes
    • MatrixOne v26.3.0.11 Release Notes
    • MatrixOne v26.3.0.12 Release Notes
    • MatrixOne v26.3.0.13 Release Notes
    • MatrixOne v26.3.0.14 Release Notes
    • MatrixOne v26.3.0.15 Release Notes
    • MatrixOne v26.3.0.5 Release Notes
    • MatrixOne v26.3.0.6 Release Notes
    • MatrixOne v26.3.0.7 Release Notes
    • MatrixOne v26.3.0.8 Release Notes
    • MatrixOne v26.3.0.9 Release Notes
    • MatrixOne v25.2.0.2 Release Notes
    • MatrixOne v25.2.0.3 Release Notes
    • MatrixOne v25.2.1.0 Release Note
    • MatrixOne v25.2.1.1 Release Notes
    • MatrixOne v25.2.2.0 Release Notes
    • MatrixOne v25.2.2.1 Release Notes
    • MatrixOne v25.2.2.2 Release Notes
    • MatrixOne v25.3.0.0 Release Note
    • MatrixOne v25.3.0.1 Release Notes
    • MatrixOne v25.3.0.2 Release Notes
    • MatrixOne v25.3.0.3 Release Notes
    • MatrixOne v25.3.0.4 Release Notes
    • Key Improvements
    • MatrixOne v24.1.1.1 Release Notes
    • MatrixOne v24.1.1.2 Release Notes
    • MatrixOne v24.1.1.3 Release Notes
    • MatrixOne v24.1.2.0 Release Notes
    • MatrixOne v24.1.2.1 Release Notes
    • MatrixOne v24.1.2.2 Release Notes
    • MatrixOne v24.1.2.3 Release Notes
    • MatrixOne v24.1.2.4 Release Notes
    • MatrixOne v24.2.0.0 Release Notes
    • MatrixOne v24.2.0.1 Release Notes
    • MatrixOne v23.0.7.0 Release Notes
    • MatrixOne v0.8.0 Release Notes
    • MatrixOne v23.1.0.0 Release Notes
    • MatrixOne v23.1.0.0-RC1 Release Notes
    • MatrixOne v23.1.0.0-RC2 Release Notes
    • MatrixOne v23.1.0.1 Release Notes
    • MatrixOne v23.1.0.2 Release Notes
    • MatrixOne v23.1.1.0 Release Notes
    • MatrixOne v22.0.2.0 Release Notes
    • MatrixOne v22.0.3.0 Release Notes
    • Docker
    • Features
    • Known issues
    • Contributors
    • MatrixOne v22.0.4.0 Release Notes
    • Docker
    • Features
    • Known issues
    • Contributors
    • MatrixOne v22.0.5.0 Release Notes
    • MatrixOne v22.0.5.1 Release Notes
    • MatrixOne v220.6.0 Release Notes
    • MatrixOne v21.0.1.0 Release Notes
  • Glossary
  • Contribute
    • How to Contribute
      • Preparation
        • Kernel developer reading and verification guide
      • Report an Issue
      • Contribute Code
      • Review a Pull Request
      • Contribute Documentation
      • Make a Design
    • Code Style
      • Code Comment Style
      • Commit & Pull Request Style
Back to top

Write Kafka data to MatrixOne using Flink¶

This chapter describes how to write Kafka data to MatrixOne using Flink.

Pre-preparation¶

This practice requires the installation and deployment of the following software environments:

  • Complete standalone MatrixOne deployment.

  • Download and install lntelliJ IDEA (2022.2.1 or later version).

  • Select the JDK 8+ version version to download and install depending on your system environment.

  • Download and install Kafka.

  • Download and install Flink with a minimum supported version of 1.11.

  • Download and install the MySQL Client.

Operational steps¶

Step one: Start the Kafka service¶

Kafka cluster coordination and metadata management can be achieved through KRaft or ZooKeeper. Here, instead of relying on standalone ZooKeeper software, we’ll use Kafka’s own KRaft for metadata management. Follow these steps to configure the configuration file, which is located in config/kraft/server.properties in the root of the Kafka software.

The configuration file reads as follows:

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# This configuration file is intended for use in KRaft mode, where
# Apache ZooKeeper is not present. See config/kraft/README.md for details.
#

############################# Server Basics #############################

# The role of this server. Setting this puts us in KRaft mode
process.roles=broker,controller

# The node id associated with this instance's roles
node.id=1

# The connect string for the controller quorum
controller.quorum.voters=1@xx.xx.xx.xx:9093

############################# Socket Server Settings #############################

# The address the socket server listens on.
# Combined nodes (i.e. those with `process.roles=broker,controller`) must list the controller listener here at a minimum.
# If the broker listener is not defined, the default listener will use a host name that is equal to the value of java.net.InetAddress.getCanonicalHostName(),
# with PLAINTEXT listener name, and port 9092.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092,CONTROLLER://:9093 listeners=PLAINTEXT://xx.xx.xx.xx:9092,CONTROLLER://xx.xx.xx.xx:9093

# Name of listener used for communication between brokers.
inter.broker.listener.name=PLAINTEXT

# Listener name, hostname and port the broker will advertise to clients.
# If not set, it uses the value for "listeners".
#advertised.listeners=PLAINTEXT://localhost:9092

# A comma-separated list of the names of the listeners used by the controller.
# If no explicit mapping set in `listener.security.protocol.map`, default will be using PLAINTEXT protocol
# This is required if running in KRaft mode.
controller.listener.names=CONTROLLER

# Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL

# The number of threads that the server uses for receiving requests from the network and sending responses to the network
num.network.threads=3

# The number of threads that the server uses for processing requests, which may include disk I/O
num.io.threads=8

# The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=102400

# The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=102400

# The maximum size of a request that the socket server will accept (protection against OOM)
socket.request.max.bytes=104857600


############################# Log Basics #############################

# A comma separated list of directories under which to store log files
log.dirs=/home/software/kafka_2.13-3.5.0/kraft-combined-logs

# The default number of log partitions per topic. More partitions allow greater
# parallelism for consumption, but this will also result in more files across
# the brokers.
num.partitions=1

# The number of threads per data directory to be used for log recovery at startup and flushing at shutdown.
# This value is recommended to be increased for installations with data dirs located in RAID array.
num.recovery.threads.per.data.dir=1

############################# Internal Topic Settings #############################
# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state"
# For anything other than development testing, a value greater than 1 is recommended to ensure availability such as 3.
offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1

############################# Log Flush Policy #############################

# Messages are immediately written to the filesystem but by default we only fsync() to sync
# the OS cache lazily. The following configurations control the flush of data to disk.
# There are a few important trade-offs here:
#    1. Durability: Unflushed data may be lost if you are not using replication.
#    2. Latency: Very large flush intervals may lead to latency spikes when the flush does occur as there will be a lot of data to flush.
#    3. Throughput: The flush is generally the most expensive operation, and a small flush interval may lead to excessive seeks.
# The settings below allow one to configure the flush policy to flush data after a period of time or
# every N messages (or both). This can be done globally and overridden on a per-topic basis.

# The number of messages to accept before forcing a flush of data to disk
#log.flush.interval.messages=10000

# The maximum amount of time a message can sit in a log before we force a flush
#log.flush.interval.ms=1000

############################# Log Retention Policy #############################

# The following configurations control the disposal of log segments. The policy can
# be set to delete segments after a period of time, or after a given size has accumulated.
# A segment will be deleted whenever *either* of these criteria are met. Deletion always happens
# from the end of the log.

# The minimum age of a log file to be eligible for deletion due to age
log.retention.hours=72

# A size-based retention policy for logs. Segments are pruned from the log unless the remaining
# segments drop below log.retention.bytes. Functions independently of log.retention.hours.
#log.retention.bytes=1073741824

# The maximum size of a log segment file. When this size is reached a new log segment will be created.
log.segment.bytes=1073741824

# The interval at which log segments are checked to see if they can be deleted according
# to the retention policies
log.retention.check.interval.ms=300000 

When the file configuration is complete, start the Kafka service by executing the following command:

#Generate cluster ID 
$ KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)" #Set log directory format 
$ bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c config/kraft/server.properties #Start Kafka service 
$ bin/kafka-server-start.sh config/kraft/server.properties

Step two: Create a Kafka theme¶

In order for Flink to read data from and write to MatrixOne, we need to first create a Kafka theme called “matrixone.” Specify the listening address of the Kafka service as xx.xx.xx.xx:9092 using the --bootstrap-server parameter in the following command:

$ bin/kafka-topics.sh --create --topic matrixone --bootstrap-server xx.xx.xx.xx:9092

Step Three: Read MatrixOne Data¶

After connecting to the MatrixOne database, you need to do the following to create the required databases and data tables:

  1. Create databases and data tables in MatrixOne and import data:

    CREATE TABLE `users` (
    `id` INT DEFAULT NULL,
    `name` VARCHAR(255) DEFAULT NULL,
    `age` INT DEFAULT NULL
    )
    
  2. Write code in the IDEA integrated development environment:

    In IDEA, create two classes: User.java and Kafka2Mo.java. These classes are used to read data from Kafka using Flink and write the data to the MatrixOne database.

package com.matrixone.flink.demo.entity;

public class User {

    private int id;
    private String name;
    private int age;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }
} 
package com.matrixone.flink.demo;

import com.alibaba.fastjson2.JSON;
import com.matrixone.flink.demo.entity.User;
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.serialization.AbstractDeserializationSchema;
import org.apache.flink.connector.jdbc.JdbcExecutionOptions;
import org.apache.flink.connector.jdbc.JdbcSink;
import org.apache.flink.connector.jdbc.JdbcStatementBuilder;
import org.apache.flink.connector.jdbc.internal.options.JdbcConnectorOptions;
import org.apache.flink.connector.kafka.source.KafkaSource;
import org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializer;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.kafka.clients.consumer.OffsetResetStrategy;

import java.nio.charset.StandardCharsets;

/**
 * @author MatrixOne
 * @desc
 */
public class Kafka2Mo {

    private static String srcServer = "xx.xx.xx.xx:9092";
    private static String srcTopic = "matrixone";
    private static String consumerGroup = "matrixone_group";

    private static String destHost = "xx.xx.xx.xx";
    private static Integer destPort = 6001;
    private static String destUserName = "root";
    private static String destPassword = "111";
    private static String destDataBase = "test";
    private static String destTable = "person";

    public static void main(String[] args) throws Exception {

        // Initialize the environment
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        // Set parallelism
        env.setParallelism(1);

        // Set kafka source information
        KafkaSource<User> source = KafkaSource.<User>builder()
                //Kafka service
                .setBootstrapServers(srcServer)
                // message subject
                .setTopics(srcTopic)
                // consumption group
                .setGroupId(consumerGroup)
                // offset Consume from the beginning when no offset is submitted
                .setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.LATEST))
                // custom parse message content
                .setValueOnlyDeserializer(new AbstractDeserializationSchema<User>() {
                    @Override
                    public User deserialize(byte[] message) {
                        return JSON.parseObject(new String(message, StandardCharsets.UTF_8), User.class);
                    }
                })
                .build();
        DataStreamSource<User> kafkaSource = env.fromSource(source, WatermarkStrategy.noWatermarks(), "kafka_maxtixone");
        //kafkaSource.print();

        // Set matrixone sink information
        kafkaSource.addSink(JdbcSink.sink()
                "insert into users (id,name,age) values(?,?,?)",
                (JdbcStatementBuilder<User>) (preparedStatement, user) -> {
                    preparedStatement.setInt(1, user.getId());
                    preparedStatement.setString(2, user.getName());
                    preparedStatement.setInt(3, user.getAge());
                },
                JdbcExecutionOptions.builder()
                        //Default value 5000
                        .withBatchSize(1000)
                        //Default value is 0
                        .withBatchIntervalMs(200)
                        // Maximum number of attempts
                        .withMaxRetries(5)
                        .build(),
                JdbcConnectorOptions.builder()
                        .setDBUrl("jdbc:mysql://"+destHost+":"+destPort+"/"+destDataBase)
                        .setUsername(destUserName)
                        .setPassword(destPassword)
                        .setDriverName("com.mysql.cj.jdbc.Driver")
                        .setTableName(destTable)
                        .build()
        ));
        env.execute();
    }
} 

Once the code is written, you can run the Flink task, which is to select the Kafka2Mo.java file in IDEA and execute Kafka2Mo.Main().

Step Four: Generating Data¶

Using the command-line producer tools provided by Kafka, you can add data to Kafka’s “matrixone” theme. In the following command, use the --topic parameter to specify the topic to add to, and the --bootstrap-server parameter to specify the listening address of the Kafka service.

bin/kafka-console-producer.sh --topic matrixone --bootstrap-server xx.xx.xx.xx:9092

After executing the above command, you will wait on the console to enter the message content. Simply enter the message value (value) directly, one message per line (separated by a newline character), as follows:

{"id": 10, "name": "xiaowang", "age": 22}
{"id": 20, "name": "xiaozhang", "age": 24}
{"id": 30, "name": "xiaogao", "age": 18}
{"id": 40, "name": "xiaowu", "age": 20}
{"id": 50, "name": "xiaoli", "age": 42}

Step Five: View Implementation Results¶

Execute the following SQL query results in MatrixOne:

mysql> select * from test.users;
+------+-----------+------+
| id   | name      | age  |
+------+-----------+------+
|   10 | xiaowang  |   22 |
|   20 | xiaozhang |   24 |
|   30 | xiaogao   |   18 |
|   40 | xiaowu    |   20 |
|   50 | xiaoli    |   42 |
+------+-----------+------+
5 rows in set (0.01 sec)
Next
Connecting MatrixOne with DolphinScheduler
Previous
Write TiDB data to MatrixOne using Flink
Copyright © 2026, MatrixOrigin
Made with Sphinx and @pradyunsg's Furo
On this page
  • Write Kafka data to MatrixOne using Flink
    • Pre-preparation
    • Operational steps
      • Step one: Start the Kafka service
      • Step two: Create a Kafka theme
      • Step Three: Read MatrixOne Data
      • Step Four: Generating Data
      • Step Five: View Implementation Results