SingleStore
Preparation
To allow Shaped to connect to your SingleStore database, you need to create a read-only user and share those credentials through the Create Dataset endpoint. You can create a read-only user in your the SingleStore service by following the SingleStore User Management documentation.
Shaped connects to SingleStore using the MySQL protocol, so you can follow the same steps as you would for a MySQL database, with the schema_type set to MYSQL.
Dataset Configuration
Required fields
| Field | Example | Description |
|---|---|---|
| schema_type | MYSQL | Specifies the connector type, in this case "MYSQL" due to wire-compatibility. |
| table | movies | The name of the table to sync. |
| user | your_user | Access account username. |
| password | pAssw0rd1! | Access account Password. |
| host | my-db.aws-virginia-6.svc.singlestore.com | Database hostname. |
| port | 3333 | Database port (the default for SingleStore is 3333). |
| database | movielens | The name of the database that contains table to sync. |
| replication_key | updated_at | The name of the column that contains a datetime key or ascending id for ordering data during incremental syncs. |
Optional fields
| Field | Example | Description |
|---|---|---|
| columns | ["productId", "color", "brand", "stockLevel"] | The name of the columns you wish to sync from SingleStore into Shaped. If not specified, all columns will be synced. |
| unique_keys | ["productId"] | Specify a list of columns that uniquely identify a row in the table, if duplicate rows are inserted with these keys, the latest row will be used. |
| batch_size | 100000 | The number of rows to fetch from the database in each batch, changing this can improve throughput for large tables. The default is 10000. |
Dataset Creation Example
Below is an example of a SingleStore dataset connector configuration:
name: your_singlestore_dataset
schema_type: MYSQL
table: movies
user: your_user
password: pAssw0rd1!
host: my-db.aws-virginia-6.svc.singlestore.com
port: 3306
database: movielens
replication_key: updated_at
The following payload will create a SingleStore dataset and begin syncing data from Shaped using the Shaped CLI.
shaped create-dataset --file dataset.yaml