Skip to content

CREATE STAGE

Grammar Description

CREATE STAGE is used to provide an efficient and secure way to interact with external storage (such as Amazon S3, file systems). By creating an external Stage, MatrixOne can read files from external storage and quickly load them into MatrixOne database tables. For example, load a CSV file on Amazon S3 into a table.

Currently the following external storage is supported:

  • Amazon S3
  • MinIO
  • File System

Grammar Structure

> CREATE STAGE [ IF NOT EXISTS ] { stage_name }
   { StageParams }
   [ COMMENT = '<string_literal>'

StageParams (for MinIO/Amazon S3):
URL = "s3://<bucket>[/<path>/]" CREDENTIALS = {"AWS_KEY_ID"='<string>', "AWS_SECRET_KEY"='<string>', "AWS_ROLE"='<string>', "AWS_TOKEN"='<string>', "AWS_REGION"='<string>', "COMPRESSION"='<string>', 'PROVIDER'='<string>', 'ENDPOINT'='<string>'}

StageParams (for File System):
URL= 'file://[/path/]'

StageParams (for sub-stage):
URL= "stage://<stagename>[/path/]"

Grammar Explanation

  • IF NOT EXISTS: Optional parameter to check whether the Stage of the same name already exists when creating a Stage, and avoid repeated creation.

  • stage_name: The name of the Stage to be created.

  • `StageParams (for MinIO/Amazon S3): Used to specify the configuration parameters for Stages whose objects are stored as MinIO or S3.

    • URL: Specify the file path or directory in S3 storage

    • CREDENTIALS: This is a JSON object that contains the credential information required to connect to the object storage service.

      • access_key_id: The access key ID used for authentication.
      • secret_access_key: The key associated with the access key ID.
      • aws_role: Non-required, if an IAM role is used, it is used to specify the role name. Roles can be configured on AWS to assign different permissions.
      • aws_token: Non-required, security token for temporary access to AWS services.
      • aws_region: Specifies the AWS Region where Amazon S3 storage resides.
      • compression: Non-required, specify the compression type of the file.
      • provider: Specify cloud storage provider.
      • endpint: Specifies a service that connects to custom or third-party S3 API compatible.

  • StageParams (for File System): Used to specify the configuration parameters of the Stage stored in the file system.

    • URL: Specifies the file path or directory in the file store.

  • StageParams (for sub-stage): Configuration parameters for sub-stage.

    • URL: Specifies the file path or directory in the file store.

  • COMMENT: Comment.

Example

#File System
mysql> create stage stage_fs url = 'file:///Users/admin/test';

#Substage
mysql> create stage sub_stage url = 'stage://fs_stage/test1/';

#s3
mysql>create stage stage01 url = 's3://bucket1/test' credentials = {"aws_key_id"='AKIAYOFAMAB7FM7Axxxx',"aws_secret_key"='UjuSDmekK6uXK6CrUs9YhZzY27VOk9W3qMwYxxxx',"AWS_REGION"='us-west-2','PROVIDER'='Amazon', 'ENDPOINT'='s3.us-west-2.amazonaws.com'};