Serving Large User Sequence Models in Large Scale Applications

What is it about?

We propose an industrial framework to reduce the cost of large user sequence models, termed as UBS (User Behavioral Service). First, we design a novel infrastructure to asynchronously call the user model and store the output (a.k.a user embeddings) offline with periodical refresh. Second, by persisting the user embeddings offline, these can be utilized by various recommendation models "for free". We conduct extensive offline and online experiments to validate the effectiveness and efficiency of our UBS framework within one of the largest SFV recommendation systems currently in use. It is also worth mentioning that our framework has been deployed in production with significant impact.

Featured Image

Photo by Sara Kurfeß on Unsplash

Photo by Sara Kurfeß on Unsplash

Why is it important?

Sequential models are invaluable for powering personalized recommendation systems. In the context of short-form video (SFV) feeds, where user behavior history is typically longer, systems must be able to understand users’ long-term interests. However, deploying large sequence models to extensive web-scale applications faces challenges due to high serving cost. To address this, we propose an industrial framework designed for efficiently serving large user sequence models