Skip to main content

Add a data table

Shaped stores data in tables, which are imported via a data connector. Shaped supports import from over 20 different sources.

Prerequisites​

Upload a local table​

The easiest way to get started is to upload a sample table, like movielens. You can upload your data via the Shaped UI or via the command line interface.

shaped create-table-from-uri --name movielens_items --path movielens_items.jsonl --type jsonl
info

Don't have a table?​

We've prepared a sample table to start with. Download it from S3:

curl -L -O "https://shaped-onboarding-demo-datasets.s3.us-east-2.amazonaws.com/movielens/movielens_items.jsonl"

Use a data connector​

You can also connect Shaped to a production data source, such as a MySQL database or a Kafka cluster.

shaped create-table --file connector_config.yaml

Your connector config should contain the login details and schema for your data source. The following example shows connecting Shaped to a Clickhouse database:

name: customer_click_events
description: Customer click events table for analytics and recommendations
schema_type: CLICKHOUSE
host: clickhouse.example.com
port: 9000
user: clickhouse_user
password: your_secure_password
database: analytics
table: customer_clicks
replication_key: created_at
columns:
  - user_id
  - item_id
  - click_timestamp
  - page_url
  - session_id
  - device_type
  - created_at
unique_keys:
  - user_id
  - item_id
  - click_timestamp

Learn more about the data connectors we support: Data connector reference