Quickstart

In this tutorial, you'll learn how to upload your first data table and train your first engine. You can do it via the CLI or in the console.

CLI

Install the CLI

The command-line interface is the fastest way to update your config in Shaped:

pip install shaped

Having trouble installing the CLI? Common errors

Initialize the client with your API key

Add your API key to authenticate into your account. If you don't have an API key yet, see Get your API key.

shaped init --api-key <YOUR_API_KEY>

Download sample data and engine config

To start quickly, use our sample data based on movielens to build a semantic search model.

Download the sample data and engine config from S3:

curl -L \
  -O "https://shaped-onboarding-demo-datasets.s3.us-east-2.amazonaws.com/movielens/movielens_items.jsonl" \
  -O "https://shaped-onboarding-demo-datasets.s3.us-east-2.amazonaws.com/movielens/movielens_semantic_search.yaml"

Upload your table

Upload the table that you downloaded locally:

shaped create-table-from-uri --name movielens_items --path movielens_items.jsonl --type jsonl

Create your engine

Every relevance engine is configured with YAML. Upload the config in movielens_semantic_search.yaml using the CLI to create a semantic search engine:

shaped create-engine --file movielens_semantic_search.yaml

Full engine config

version: v2
name: movielens_semantic_search
data:
  item_table:
    name: movielens_items
    type: table
index:
  lexical_search: # enables BM25 lexical search on item fields
    item_fields:
      - movie_title
      - description
      - cast
      - genres
      - writers
      - directors
      - interests
  embeddings:
    - name: content_embedding # enables vector search
      encoder:
        type: hugging_face
        model_name: sentence-transformers/all-MiniLM-L6-v2
        batch_size: 256
        item_fields:
          - movie_title
          - description
          - cast
          - genres
          - writers
          - directors
          - interests

Console

We'll use the movielens table enriched with data from IMDb and build a semantic search model.

Step 1: Upload the data as a new table

1. Download the sample table data here - movielens_items.jsonl
2. Open your Shaped dashboard
3. Click Tables in the leftpane
4. Click Add new table in the top right
5. Choose "Local file upload"
6. Choose the JSONL file you downloaded

Step 2: Configure a new engine

1. Download the sample engine config here - movielens_semantic_search
2. In the leftpane, go to Engines
3. Click Upload a new engine in the top right
4. Choose the YAML file you downloaded in step 1

Step 3: Query your engine

1. In the leftpane, go to the Query section
2. Copy the query below into the query editor on the left
3. Press Run

SELECT *
FROM text_search(
  query='movies with Christopher Nolan',  
  mode='vector',  
  text_embedding_ref='content_embedding', 
  limit=50)
LIMIT 200

Once your engine is up and running, you’re ready to integrate it into your application. Read the Queries guide to learn how to get search results using your engine.