Working with Shaped
The Shaped team includes world-class machine-learning, data and infrastructure engineers that have built real-time machine-learning systems at companies like Meta, Apple, Google, and H&M. We have published AI researchers and data scientists that work to ensure your RecSys models use the latest AI research, and are proud of the research to production pipelines we've built to make this happen.
When working with Shaped, you can think of us as your personalized machine-learning or ranking team. We'll assign you a point-of-contact personalization expert that'll work with you every step of the way, from model creation to production deployment and uplift maintenance. As part of the white-glove service we offer, we can connect Slacks, organize regular meetings and work together to ensure you're consistently hitting your metric uplift goals.
On-boarding with Shaped typically has the following steps:
1. Goal Alignment
Before getting started with an integration, we first take the time to align on your hero use-case and uplift goals. Although Shaped works for a variety of different use-cases, we start with the use-case that you think has the most to gain from personalization. For example, this may be your main feed or home-page discovery experiences.
2. Schema Jam
We then brainstorm with you the different user, item and event data sources that might be helpful signal to for personalization. Usually this involves working out the exact Shaped connectors you'll want to use with Shaped, and if there's one we don't have, we'll build it for you. Part of this process also involves suggesting best-practices for storing and logging these data types to improve your models.
The most common data logging issues we see is that companies don't log impressions! These are the events that are fired when an item is shown to a user but isn't clicked. Although not a requirement, they're important for recommendation systems so that you have a low confidence negative signal of what the user doesn't like.
3. Model Creation
We then provision your API key and create your initial model. Model creation involves connecting to your user, item and event data sources and writing the transforms to get the data into the right shape. We backfill any of the data that your sources provide, and the great thing about connecting directly to the source is that the model will continually get up-dated with new data from the start. Getting the model connected, setup and trained takes about 1-2 days.
4. Offline Evaluation
Once you have your bespoke Shaped model setup, you can start evaluating it in an offline setting. By offline we mean, without having to integrate Shaped's rank endpoints into production. You can use Shaped's CLI to hit our inference endpoints to evaluate results manually and we provide several tools to help you visualize your results clearly. This typically takes 1 week.
5. A/B Test
We'll then work with you to run an online A/B test that evaluates performance uplift on a subset of production users. We can help recommend tools and evaluation criteria to make sure you're monitoring things correctly. This typically takes 1-3 weeks.
6. Production roll-out
Once we've gained the confidence that Shaped will hit your metric uplift goals, the final step is a complete roll-out to your user-base. If you let us know when you're doing the full roll-out we can ensure we're available to help with any issues that come up during this process.
7. Model Iteration
The journey isn't over, though, Shaped can help you with many different personalization use-cases, and now it's time to make more models that target these other use-cases. Once you're happy with your first model, we can repeat steps 3. to 6. for all your other personalization use-cases. Typically the iteration speed gets faster each time.