Big Data in Recommendation Systems: An In-Depth Survey of Methods and Experiments

What is it about?

This survey paper provides a comprehensive analysis of big data algorithms in recommendation systems, addressing the lack of depth and precision in existing literature. The paper incorporates both empirical and experimental evaluations to differentiate between the techniques. The empirical evaluation ranks the techniques based on four criteria. The experimental assessments rank the algorithms that belong to the same category, sub-category, technique, and sub-technique. Also, the paper illuminates the future prospects of big data techniques in recommendation systems, underscoring potential advancements and opportunities for further research in this fields.

Featured Image

Photo by Emily Morter on Unsplash

Photo by Emily Morter on Unsplash

Why is it important?

Here is an outline of the main findings from our experimental evaluation: • Content-based filtering showed good scalability and effectiveness in sparse data scenarios but tended to create a "filter bubble." • Item-based collaborative filtering excelled in precision but struggled with cold-start problems and data scalability. • Matrix factorization algorithms outperformed others in high data sparsity situations, though they were sensitive to hyperparameter settings and faced scalability challenges. • User-based collaborative filtering and graph-based models demonstrated high precision, especially in sparse interaction contexts, but had limitations in scalability and dataset density. • Rule-based models were fast and precise in specific scenarios but less effective with large and diverse datasets. • Context-aware models, attention and memory network models, and neural graph-based models showed significant improvements in precision and handling sparse datasets. • Autoencoders and RNNs were effective in learning complex patterns and sequential user behaviors, though they faced challenges in computational complexity and scalability. • Convolutional operations-based models balanced precision with scalability but required higher computational resources. • Self-attention models excelled in precision and scalability but had limitations in handling diverse user interests. • Algorithmic and mathematical modeling algorithms efficiently processed large, sparse datasets, maintaining quick response times and minimal computational demands. Each algorithmic approach exhibited unique advantages in specific contexts, underscoring the need for careful selection and optimization based on the specific requirements of the recommendation system being developed.

Perspectives