Skip to main content

Overview

Shaped Transforms allow you to enhance and transform your existing Datasets to improve the quality and relevance of your search and recommendation models. Instead of manually preprocessing your data before ingesting it into Shaped, you can leverage Shaped Transforms to automatically enrich and modify your datasets within the platform.

What are Shaped Transforms?​

Transforms are data processing operations that take your existing datasets and apply various transformations to create enhanced versions of your data. This can include enriching text fields with additional context, captioning or deriving new text features from images, deriving new features from existing ones, or applying complex SQL operations to reshape your data.

Shaped Transforms operate on your already-ingested datasets and create new, transformed datasets that can be used for model training. This allows you to:

  • Enrich your data with additional context and features without modifying your source systems
  • Experiment with different transformations to improve model performance
  • Iterate quickly on data preprocessing without rebuilding your entire data pipeline
  • Leverage AI to automatically generate rich features from existing data

Types of Transforms​

Shaped currently supports two types of transforms:

LLM Enrichment Transforms​

LLM Enrichment Transforms use large language models to automatically enhance your data with AI-generated insights. This is particularly powerful for:

  • Generating embeddings from text descriptions
  • Extracting structured information from unstructured text
  • Creating category tags or classifications
  • Enriching product or content descriptions
  • Deriving semantic features from text fields

LLM transforms can significantly improve recommendation quality by extracting deeper meaning and context from your existing text data.

SQL Transforms (Coming Soon)​

SQL Transforms allow you to use standard SQL queries to transform your datasets. This is useful for:

  • Filtering and aggregating data
  • Joining multiple datasets
  • Creating derived features from existing columns
  • Reshaping data structures
  • Applying complex business logic

SQL Transforms give you the full power of SQL to manipulate your data while keeping it within the Shaped ecosystem.

Getting Started​

To use Shaped Transforms, you'll first need to have datasets already ingested into Shaped. Once your data is in place, you can create transforms using the Shaped API. Each transform type has its own configuration and parameters, which are detailed in the following sections.

For detailed information on each transform type, see: