Jewel ML’s Content-based Recommendation System powered by Deep Learning
Pradnyesh Vineet Joshi
Machine Learning Engineer
Recommendation systems play a vital role in increasing user engagement and maximizing revenue for e-commerce websites. Similar product recommendation is one of the most common tasks in recommendation systems. In this blog post, we will discuss the high level architecture of Jewel ML’s content-based similar item recommendation system powered by Deep Learning.
Before we dig into architecture, let’s look at the ways Jewel's Machine Learning approach will help your business:
Increase session length: Similar product recommendations increase session length by showing users items most relevant to them.
Autonomous: Model Training is completely autonomous and updates recommendations frequently to deal with fast-changing catalog. There is no need to retrain the model or schedule data refresh from the client side.
Increase revenue: Enhanced Deep Learning models increase revenue by facilitating product discovery (validated by systematic A/B testing).
The following diagram sums up the model building and serving process for Jewel ML’s content-based similar item recommendation system:
Model building phase consists of:
Processing features: To learn similarity between products, Jewel looks at different product attributes such as title, description, brand, price and so on. The first step is to clean the text and non-text features.
Extract embeddings: In the next step, we extract embeddings from text using a pre-trained FastText model.
Train Deep Neural Network: Text embeddings and non-text features are then fed as inputs to a deep ranking model that gives as output fine-grained embeddings. The model learns fine-grained product similarity using an efficient triplet sampling algorithm.
Monitor model performance and model refinement: Model performance is monitored throughout the training process using metrics such as top-k score and average similarity precision. Through experimentation with different neural architectures and features, we finalize a model for the client that gives the best performance on a validation set.
Now that the model is finalized, let’s see how we serve it. We maintain a Mongo collection of recommended items for each item in the catalog. Each API call retrieves the recommended items from this collection, which results in very low latency serving times. The recommendations are updated frequently to keep up with changes in catalog. Updating recommendations entails preprocessing new catalog features, getting fine-grained embeddings, finding the top-k most similar items for each item in the catalog using those embeddings, and finally updating the Mongo collection. The process is fully automated using GPU-powered instances over Google Cloud Platform (GCP). Stay tuned for more details on how we leverage GCP to automate model updates!