# Import data from REST APIs

REST API data ingestion loads JSON records from external HTTP/HTTPS endpoints into Catalog tables. Save the base URL and authentication settings in a REST API connector first, then configure the endpoint path, pagination, incremental cursor, and target table mapping in an import task.

## 1. Create a REST API connector

1. Go to **Data Connection > Connectors** and click **New connector**.
2. Enter a connector name, select **REST API** as the data source type, and keep the import purpose enabled.
3. Enter the **API Base URL** (for example, `https://api.example.com/v1/`—must be a complete HTTP/HTTPS URL without query strings or fragments).
4. Choose an **Authentication** mode and fill in the required credentials:

| Authentication mode | Required fields |
| --- | --- |
| **None** | No credentials required |
| **Bearer Token** | **Bearer Token** |
| **API Key** | **Key Location** (Header or Query), **Key Name**, and **Key Value** |
| **HTTP Basic** | **Username** and **Password** |
| **OAuth 2.0 · Client Credentials** | **Token URL**, **Client ID**, **Client Secret**, **Client Authentication**, optional **Scope**, and optional **Token Request Parameters** |

5. Expand **Advanced options** as needed to add **Shared Headers**, **Shared Query Parameters**, and **Request Timeout (seconds)** (an integer between 1 and 60).
6. To test credentials before saving, enable **Configure verification endpoint (optional)**, specify a lightweight read-only **Verification endpoint path** (starting with a single `/`), **Method**, optional parameters, and optional **Business status field path** + **Expected value (JSON)**, then click **Test connection**.
7. Save the connector once verified.

When editing an existing connector, leave secret fields blank to retain saved credentials.

## 2. Create a REST API import task

1. Go to **Data Connection > Import Tasks**, choose **Structured**, and select **REST API** as the source.
2. Select your REST API connector and enter the **Endpoint path** (starting with a single `/`, appended to the connector's base URL).
3. Expand advanced options to match the endpoint contract:
   - **Record path in the response** (optional): provide the JSON path (for example, `$.data.items`) when records are nested inside an object;
   - **Pagination**: choose **None**, **Page number**, **Offset**, **Cursor**, or **Next page URL**, and configure the query parameter, starting page/offset, or response path (such as `$.paging.next`);
   - **Response schema** (optional): enable **Use the schema returned by the API** and enter the schema path (such as `$.schema.fields`) and format (**Column descriptors** or **JSON Schema**); otherwise field types are inferred from preview records.
4. Click **Preview data** (supports `GET`, `OPTIONS`, and `POST` after confirming the endpoint is read-only). Previewing reads the first page and displays up to **20** records without creating an import task.
5. Choose a target table (new or existing) and review column mappings, target types (`BIGINT`, `DECIMAL`, `TEXT`, `VARCHAR`, `DATE`, `TIMESTAMP`, `BOOL`, `JSON`, `BLOB`, `VARBINARY`), and binary encodings (`Base64` / `Hex`).
6. Choose one-time or scheduled execution:
   - For scheduled incremental loads, configure the **Incremental query parameter**, **Cursor field in records**, and **Initial cursor**, confirm inclusive (`>=`) semantics, and map at least one primary key plus a non-null `BIGINT` or `TIMESTAMP(6)` version field;
   - A full replacement import replaces all rows in an existing target table after succeeding.
7. Click **Create and start import**. Track execution in [Import tasks](load-tasks.md) and confirm written rows in Catalog once completed.

## Verification stages and completion check

- **Test connection** (in Connectors): checks endpoint reachability and credentials against the verification path only.
- **Preview data** (in Import Tasks): verifies pagination, record path, and type mappings against the first page without writing to the target table.
- **Completion check**: After the job completes in [Import tasks](load-tasks.md), open the target Catalog table to verify written rows and confirm scheduled runs advance the watermark cursor.
