Query the connector table structure

Read the field structure and mapping reference information of a structured source object. Query the object first, and then pass the selected object ID into this interface.

POST https://moi.matrixorigin.cn/newmoi/connectors/structured/v1/source/schema

Preparation before calling

First query connector table and select an object. Prepare a personal access token and target workspace ID that has access to the target workspace; verify that both enabled and capabilities.describe_schema are true for this entry in the browsable connector interface.

The example below uses:

  • $AI_STUDIO_API_KEY: The actual personal access token, passed through the X-API-Key Header.

  • $WORKSPACE_ID: The workspace ID to be queried, passed through the X-Workspace-ID Header.

  • $CONNECTOR_ID: View the connector ID returned by the browsable connector interface.

  • $SOURCE_TYPE: View the source type returned by the browsable connector interface.

  • $DATABASE: Query the database of the selected object in the object interface.

  • $SCHEMA: Query the schema of the selected object in the object interface; only used by SQL Server, Oracle and PostgreSQL.

  • $TABLE: Query the table of the selected object in the object interface; MongoDB does not use this variable.

  • $COLLECTION: Query the collection of the selected object in the object interface; only used by MongoDB.

Request example

Relational database

curl -X POST "https://moi.matrixorigin.cn/newmoi/connectors/structured/v1/source/schema" \
  -H "X-API-Key: $AI_STUDIO_API_KEY" \
  -H "X-Workspace-ID: $WORKSPACE_ID" \
  -H 'Content-Type: application/json' \
  -d "{
    \"connector_id\": \"$CONNECTOR_ID\",
    \"source\": {
      \"connector_id\": \"$CONNECTOR_ID\",
      \"source_type\": \"$SOURCE_TYPE\",
      \"database\": \"$DATABASE\",
      \"schema\": \"$SCHEMA\",
      \"table\": \"$TABLE\"
    }
  }"

SQL Server, Oracle, and PostgreSQL pass in schema; Hive and MySQL omit this field from the request body.

MongoDB

MongoDB uses collection and does not pass table. When you need to limit the amount of structural inference samples, pass mongo_sample_limit at the top level of the request body; 100 in the following example is a sample value, not a default value.

curl -X POST "https://moi.matrixorigin.cn/newmoi/connectors/structured/v1/source/schema" \
  -H "X-API-Key: $AI_STUDIO_API_KEY" \
  -H "X-Workspace-ID: $WORKSPACE_ID" \
  -H 'Content-Type: application/json' \
  -d "{
    \"connector_id\": \"$CONNECTOR_ID\",
    \"source\": {
      \"connector_id\": \"$CONNECTOR_ID\",
      \"source_type\": \"SOURCE_TYPE_MONGODB\",
      \"database\": \"$DATABASE\",
      \"collection\": \"$COLLECTION\"
    },
    \"mongo_sample_limit\": 100
  }"

Request body

Field

Type

Is it required

Description

connector_id

string

Yes

The connector ID.

source

object

Yes

Source object reference.

source.connector_id

string

Yes

Connector ID, must be the same as the top-level connector_id.

source.source_type

string

Yes

View the structured source type returned by the browsable connector interface.

source.database

string

Yes

The database name returned by the query connector database interface.

source.schema

string

No

Query the Schema name returned by the database classification interface; omit when the source does not support Schema hierarchy.

source.table

string

If not MongoDB

The table name returned by the query connector table interface.

source.collection

string

Required for MongoDB

The collection name returned by the query connector table interface.

mongo_sample_limit

integer

No

MongoDB-specific upper limit on the number of sampled documents. Only passed in when source.source_type is SOURCE_TYPE_MONGODB.

[] after a type denotes an array. [] in a field path denotes each item in an array.

Successful response

Returns 200 on success. Read the field structure from data.schema_snapshot; the remaining fields are used for subsequent mapping and consistency checking. The JSON structure returned by different sources may be different, and the caller should retain unknown fields.

{
  "code": "OK",
  "msg": "OK",
  "data": {
    "schema_snapshot": {
      "fields": [
        {
          "name": "id",
          "type": "bigint"
        }
      ]
    },
    "mapping_draft": [],
    "target_type_catalog": [],
    "field_compatibilities": [],
    "source_schema_hash": "schema-hash",
    "source_object_hash": "object-hash",
    "connector_config_hash": "connector-hash",
    "mapping_hash": "mapping-hash",
    "flatten_hash": "flatten-hash",
    "endpoint_results": []
  }
}

The response fields are as follows.

Field

Type

Description

code

string

OK on success.

msg

string

OK on success.

data.schema_snapshot

JSON

A snapshot of the field structure of the source object.

data.mapping_draft

JSON

The derived mapping draft; returns an empty array if there are no mapping items.

data.target_type_catalog

JSON

Catalog of available target types; returns an empty array if there are no catalog entries.

data.field_compatibilities

JSON

Field compatibility information; returns an empty array if there are no compatibility items.

data.source_schema_hash

string

Source schema snapshot summary.

data.source_object_hash

string

Source object summary.

data.connector_config_hash

string

A summary of the connector configuration, without the original key.

data.mapping_hash

string

Summary of mapping configuration; may be omitted if not applicable.

data.flatten_hash

string

Summary of flattening configuration; may be omitted if not applicable.

data.endpoint_results

JSON[]

Currently returns an empty array.

In field paths, [] means each item in an array. For example, items[].name is the name field of each item in items.

Error response

{
  "code": "ErrParamInvalid",
  "msg": "invalid parameter",
  "data": null
}

Common HTTP errors

HTTP status code

error code

Common causes

Recommended actions

400

ErrParamInvalid

connector_id or source is missing, or the object reference does not match the connector.

Reassemble source using the object fields returned by the query object interface.

501

STRUCTURED_CONNECTOR_SERVICE_UNAVAILABLE

The current service does not provide structured data source discovery.

This group of interfaces is temporarily unavailable.

200

STRUCTURED_LOAD_FEATURE_DISABLED

The structured data source feature is not enabled in the current workspace.

Check if this feature is available for this workspace.

200

STRUCTURED_LOAD_MONGODB_DISABLED

MongoDB structured discovery is not enabled for the current workspace.

Do not continue calling the MongoDB connector’s discovery interface.

500

ErrServer

The service failed to read the source field structure.

Keep the desensitization error message and try again.

Follow-up operations

Save the returned fields and object identifiers, and go to Create Task of the import task to configure structured import.

Last updated on