Managing Your Models

In the previous guides we've discussed different ways to create models using the Model API. To help you monitor and manage these models, Shaped also provides the following endpoints:

• List Models
• View Model
• Delete Model

In this guide we'll go through each of these endpoints, explain how they can be used and what information you can find in the response.

List Models

List models is used to view all the models you've created and a brief explanation of their status. It can be used with the following CLI command:

shaped list-models

Example Response:

{
  "models": [
    {
      "created_at": "2023-02-22T19:17:51 UTC",
      "model_name": "video_recommendation_model",
      "model_uri": "https://api.prod.shaped.ai/v1/models/video_recommendation_model",
      "status": "ACTIVE",
      "trained_at": "2023-03-18T00:32:47 UTC"
    },
    {
      "created_at": "2023-01-02T16:57:13 UTC",
      "model_name": "product_recommendation_model",
      "model_uri": "https://api.prod.shaped.ai/v1/models/product_recommendation_model",
      "status": "TRAINING",
      "trained_at": "2023-03-18T00:29:53 UTC"
    },
  ]
}

As you can see, the response returns a list of all the models you've created. Each model response object has the following information:

1. created_at: when the model was initially created.
2. model_name: the unique model_name identifier.
3. model_uri: your model's unique endpoint URI, you use this to make model specific requests e.g. ranking.
4. status: your model's training and deployment status, this can have the following values:
   • SCHEDULING — your model is awaiting resources to be provisioned and used to fetch, train, and deploy your model.
   • FETCHING — your historic data is being fetched.
   • TRAINING — your model is being trained on your historic dataset.
   • DEPLOYING — your model is being deployed to your custom real-time endpoint.
   • ACTIVE — your model is active and ready for rank requests.
   • ERROR — there was an error at some point when creating your model.
5. trained_at: the last time your model was trained on your data. This can be useful to understand at what points we schedule a training run. Depending on your data characteristics Shaped can train up to every few hours.

View Model

View model is used to retrieve a detailed view about a specific model. As well as the data from list models, it also returns the fetch query used to create the model and the derived schema.

shaped view-model --model-name video_recommendation_model

Example Response:

{
  "created_at": "2023-02-22T19:17:51 UTC",
  "fetch": {
    "events": "SELECT user_id, item_id, created_at, (CASE WHEN event = 'click' THEN 1 ELSE 0 END) as label FROM bigquery_connector.click_events",
    "items": "SELECT item_id, created_at, description, hashtags FROM bigquery_connector.videos"
  },
  "model_name": "video_recs",
  "model_uri": "https://api.prod.shaped.ai/v1/models/video_recommendation_model",
  "schema": {
    "interaction": {
      "created_at": "created_at",
      "features": [],
      "label": {
        "name": "label",
        "type": "Binary"
      }
    },
    "item": {
      "created_at": "created_at",
      "features": [
        {
          "name": "description",
          "type": "Text"
        },
        {
          "name": "hashtags",
          "type": "Set[Category]"
        }
      ],
      "id": "item_id"
    },
    "user": {
      "id": "user_id",
      "features": []
    }
  },
  "status": "ACTIVE",
  "trained_at": "2023-03-18T00:32:47 UTC"
}

You'll notice that all the fields in the list models response are within the output here, there's also two new more verbose fields:

1. fetch: this contains the fetch query used to create the model. It's useful to understand exactly what data this model is using.
2. schema: this is the derived schema that we create from your model's data. It gives you a transparent understanding of how your data's columns are being understood by Shaped's underlying algorithms. If you spot somethihing wrong, it may mean there's some data quality issues with the data being fed through.

Delete Model

The final endpoint you'll want to know is Delete Model. This endpoint allows you to delete any of your unneeded or failing models.

shaped delete-model --model-name video_recommendation_model