Overview

Shaped supports a wide range of ranking metrics that are used to evaluate models in an offline setting (e.g. through cross-validation) and in an online setting (e.g. while monitoring online performance, or running online tests such as A/B or MAB testing).

After you create a Shaped model you can view all of these metrics in the Shaped dashboard. There is some information about them there, however, this section of the docs explains each one in more detail.

🎣 Recall

Fraction of relevant items successfully retrieved within the recommendations.

🎯 Precision

Fraction of recommended items that are truly relevant to the user.

✅ HitRate

Fraction of recommendation lists containing at least one relevant item (@K).

📈 AUC

Area Under the ROC Curve, measuring the model's ability to rank relevant items higher than irrelevant ones.

🏆 MRR

Mean Reciprocal Rank: Average reciprocal rank of the first relevant item in the list.

📊 mAP

Mean Average Precision: Considers precision at different recall levels across users/queries.

🏅 NDCG

Normalized Discounted Cumulative Gain: Measures ranking quality considering graded relevance and position.

👤👥 Personalization

Measures the dissimilarity between recommendation lists generated for different users.

🗺️ Coverage

Proportion of the total item catalog that gets recommended by the system (item coverage).

⭐ Average Popularity

Average popularity score of recommended items, used to assess popularity bias.

📄 Content Relevance

How well the content of recommended items matches user interests/query, based on item features.

💻 Online Metrics

Metrics measured from live user interactions in production (e.g., CTR, CVR, Session Length).