Skip to main content
warning

This is an article from the Shaped 1.0 documentation. The APIs have changed and information may be outdated. Go to Shaped 2.0 docs

AI Enrichment Transforms

AI Enrichment Transforms enhance your existing Shaped Datasets using large language models (LLMs). They take columns from a source dataset, apply an LLM with a prompt, and output new columns with enriched information. Use them specifically to enrich data that will power semantic search.

Prerequisites​

  • You have already created a Shaped Dataset to use as the source
  • You have an API key for the Shaped API

How it works​

  1. Select columns from an existing Shaped Dataset.
  2. Pass those columns to an LLM with task-specific instructions.
  3. The transform writes a new enriched dataset with additional features (e.g., semantic attributes, improved text). For example, with fashion products, you can enrich titles/descriptions into detailed attributes like style, material, fit, and use-case.

These enriched datasets can be used like Shaped Datasets for building and training Shaped models.

API​

All requests require these headers:

headers = {
    "x-api-key": SHAPED_API_KEY,
    "Content-Type": "application/json"
}

Create a transform​

curl -X POST 'https://api.shaped.ai/v1/transforms' \
  -H 'x-api-key: '"$SHAPED_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "movielens_items_ai_enrichment_transform",
    "description": "AI enrichment transforms on Movielens items data.",
    "transform_type": "AI_ENRICHMENT",
    "source_dataset": "movielens_items",
    "source_columns": ["movie_title","release_date"],
    "source_columns_in_output": ["movie_id", "movie_title"],
    "enriched_output_columns": ["generated_genre", "generated_era", "generated_director"],
    "prompt": "Given the movie_title and release_date as input data, you will have to output generated_genre, generated_era and generated_director in json format."
  }'

Body parameters​

  • name (string, required): Unique name for the transform. Example: movielens_items_ai_enrichment_transform.
  • description (string, optional): Human-readable description. Example: AI enrichment transforms on Movielens items data.
  • transform_type (string, required): Must be AI_ENRICHMENT.
  • source_dataset (string, required): Name of the existing Shaped Dataset to enrich. Example: movielens_items.
  • source_columns (array of strings, required): Source dataset columns passed to the model. Example: ["movie_title", "release_date"].
  • source_columns_in_output (array of strings, optional): Source columns to copy into the enriched output dataset. Example: ["movie_id", "movie_title"].
  • enriched_output_columns (array of strings, required): New columns to be generated by the model. Example: ["generated_genre", "generated_era", "generated_director"].
  • prompt (string, required): Task instructions, including how to map inputs to outputs and the expected output format.

List transforms​

curl -X GET 'https://api.shaped.ai/v1/transforms' \
  -H 'x-api-key: '"$SHAPED_API_KEY" \
  -H 'Content-Type: application/json'

Get transform details​

curl -X GET 'https://api.shaped.ai/v1/transforms/movielens_items_ai_enrichment_transform/' \
  -H 'x-api-key: '"$SHAPED_API_KEY" \
  -H 'Content-Type: application/json'

Delete a transform​

curl -X DELETE 'https://api.shaped.ai/v1/transforms/movielens_items_ai_enrichment_transform' \
  -H 'x-api-key: '"$SHAPED_API_KEY" \
  -H 'Content-Type: application/json'

Usage​

  • We use AI Enrichment Transforms only to enrich the data.
  • The enriched data improves search system performance by adding semantic features.
  • The enriched dataset can be used like Shaped Datasets to build and train Shaped models.