A review of multimedia recommender systems

Abstract :

The purpose of this study is to comprehensively review multimedia recommender systems with a focus on content-based approaches. The diversity and abundance of data on the web has prompted experts to research and develop recommender systems for automatically predicting the user's favorite content. In the upcoming research, various types of recommender systems were introduced, which are collaborative, content-based, and hybrid. Then, a review of 47 articles centered on content-based recommender systems in the three areas of movie, music, and television program recommendations was undertaken. In addition, the approaches used in recommender systems used in large and famous companies such as Netflix, YouTube, Facebook, Amazon, and TikTok were examined in detail. Due to the importance of the body used in recommender systems, MMTF-14K, MovieLens and Spotify Audio Features datasets were also introduced. According to the conducted research, it is clear that many practical recommender systems in large companies have benefited from the content-based approach for recommendations.

 

References:

[1] Deldjoo, Y., et al. Multimedia recommender systems. in Proceedings of the 12th ACM Conference on Recommender Systems. 2018.

[2] Deldjoo, Y., et al. MMTF-14K: a multifaceted movie trailer feature dataset for recommendation and retrieval. in Proceedings of the 9th ACM Multimedia Systems Conference. 2018.

[3] Chong, D., Deep dive into netflix’s recommender system. 2020, Medium.

[4] Amato, F., et al., SOS: A multimedia recommender System for Online Social networks. Future generation computer systems, 2019. 93: p. 914-923.

[5] Liang, T., et al., A hybrid recommendation model based on estimation of distribution algorithms. Journal of Computational Information Systems, 2014. 10(2): p. 781-788.

[6] Chen, Y.-L., Y.-H. Yeh, and M.-R. Ma, A movie recommendation method based on users' positive and negative profiles. Information Processing & Management, 2021. 58(3): p. 102531.

[7] Pradeep, N., et al., Content based movie recommendation system. International Journal of Research in Industrial Engineering, 2020. 9(4): p. 337-348.

[8] Singh, R.H., et al., Movie recommendation system using cosine similarity and KNN. International Journal of Engineering and Advanced Technology, 2020. 9(5): p. 556-559.

[9] Almeida, M.S. and A. Britto. MOEA-RS: A Content-Based Recommendation System Supported by a Multi-objective Evolutionary Algorithm. in International Conference on Artificial Intelligence and Soft Computing. 2020. Springer.

[10] Meel, P., et al. Movie Recommendation Using Content-Based and Collaborative Filtering. in International Conference on Innovative Computing and Communications. 2021. Springer.

[11] Singla, R., et al. FLEX: A Content Based Movie Recommender. in 2020 International Conference for Emerging Technology (INCET). 2020. IEEE.

[12] Sottocornola, G., et al. Towards a deep learning model for hybrid recommendation. in Proceedings of the International Conference on Web Intelligence. 2017.

[13] Yin, H., et al., Dynamic user modeling in social media systems. ACM Transactions on Information Systems (TOIS), 2015. 33(3): p. 1-44.

[14] Albanese, M., et al., A multimedia recommender system. ACM Transactions on Internet Technology (TOIT), 2013. 13(1): p. 1-32.

[15] Rajasekar, R., Radhakrishnan, N., Sridar, K., Viji, C., Mohanraj, M., Kalpana, C., & Rajkumar, N. (2025). Intelligent movie recommendation system. Salud, Ciencia y Tecnología-Serie de Conferencias, (4), 1438.

[16]Shashaani, S. (2024, October). Explainability in music recommender system. In Proceedings of the 18th ACM Conference on Recommender Systems (pp. 1395-1401).

[17] Van Den Oord, A., S. Dieleman, and B. Schrauwen. Deep content-based music recommendation. in Neural Information Processing Systems Conference (NIPS 2013). 2013. Neural Information Processing Systems Foundation (NIPS).

[18] Wang, X. and Y. Wang. Improving content-based and hybrid music recommendation using deep learning. in Proceedings of the 22nd ACM international conference on Multimedia. 2014.

[19] Liu, N.-H., Comparison of content-based music recommendation using different distance estimation methods. Applied intelligence, 2013. 38(2): p. 160-174.

[20] Bogdanov, D., et al., Semantic audio content-based music recommendation and visualization based on user preference examples. Information Processing & Management, 2013. 49(1): p. 13-33.

[21] Wang, X., D. Rosenblum, and Y. Wang. Context-aware mobile music recommendation for daily activities. in Proceedings of the 20th ACM international conference on Multimedia. 2012.

[22] Barragáns-Martínez, A.B., et al., Exploiting social tagging in a web 2.0 recommender system. IEEE Internet Computing, 2010. 14(6): p. 23-30.

[23] Chiliguano, P. and G. Fazekas. Hybrid music recommender using content-based and social information. in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2016. IEEE.

[24] Chou, S.-Y., et al. Addressing cold start for next-song recommendation. in Proceedings of the 10th ACM Conference on Recommender Systems. 2016.

[25] R. Turrin, A.C., R. Pagano, M. Quadrana, and P. Cremonesi, Large scale music recommendation, in LSRS. 2015.

[26] Vasile, F., E. Smirnova, and A. Conneau. Meta-prod2vec: Product embeddings using side-information for recommendation. in Proceedings of the 10th ACM Conference on Recommender Systems. 2016.

[27] Wang, X., et al., Exploration in interactive personalized music recommendation: a reinforcement learning approach. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2014. 11(1): p. 1-22.

[28] Rao, A., et al. Supervised Feature Learning for Music Recommendation. in International Symposium on Signal Processing and Intelligent Recognition Systems. 2020. Springer.

[29] Hosseinzadeh Aghdam, M., et al. Adapting recommendations to contextual changes using hierarchical hidden markov models. in Proceedings of the 9th ACM Conference on Recommender Systems. 2015.

[30] Jannach, D., L. Lerche, and I. Kamehkhosh. Beyond" hitting the hits" Generating coherent music playlist continuations with the right tracks. in Proceedings of the 9th ACM Conference on Recommender Systems. 2015.

[31] Vall, A., et al. A hybrid approach to music playlist continuation based on playlist-song membership. in Proceedings of the 33rd Annual ACM Symposium on Applied Computing. 2018.

[32] Vall, A., et al. Music playlist continuation by learning from hand-curated examples and song features: Alleviating the cold-start problem for rare and out-of-set songs. in Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems. 2017.

[33] Gharahighehi, A. and C. Vens, Personalizing diversity versus accuracy in session-based recommender systems. SN Computer Science, 2021. 2(1): p. 1-12.

[34] Engelbert, B.B., M.B.; Kruthoff-Bruwer, R.; Morisse, K. A user supporting personal video recorder by implementing a generic Bayesian classifier based recommendation system. in Proceedings of the 2011 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops). 2011.

[35] Quan, J.-C.C., S.-B, A Hybrid Recommender System Based on AHP That Awares Contexts with Bayesian Networks for Smart TV, in International Conference on Hybrid Artificial Intelligence Systems (HAIS 2014). 2014.

[36] Oh, J.S., Y.; Kim, J.; Humayoun, M.; Park, Y.H.; Yu, H. Time-dependent user profiling for TV recommendation. in Proceedings of the 2nd International Conference on Cloud and Green Computing and 2nd International Conference on Social Computing and Its Applications, CGC/SCA 2012. 2012.

[37] Turrin, R.C., A.; Cremonesi, P.; Pagano, R. Time-based TV programs prediction. in Proceedings of the 1st Workshop on Recommender Systems for Television and Online Video (RecSysTV) 2014. 2014.

[38] da Silva, F.S., L.G.P. Alves, and G. Bressan, Personal TVware: an infrastructure to support the context-aware recommendation for personalized digital TV. International Journal of Computer Theory and Engineering, 2012. 4(2): p. 131.

[39] Zibriczky, D., et al. Personalized recommendation of linear content on interactive TV platforms: beating the cold start and noisy implicit user feedback. in UMAP workshops. 2012.

[40] Song, S., H. Moustafa, and H. Afifi, Advanced IPTV services personalization through context-aware content recommendation. IEEE Transactions on Multimedia, 2012. 14(6): p. 1528-1537.

[41] Chaudhry, M., et al. Heterogeneous information network based TV program recommendation. in Proceedings of the 16th International Symposium on Advanced Intelligent Systems. 2015.

[42] Barraza-Urbina, A., et al., Using social media data for online television recommendation services at RTÉ Ireland. 2015.

[43] Hromic, H., et al., Event panning in a stream of big data. Research Day 2013 Schedule, 2012: p. 44.

[44] Yuan, J., et al., Context-aware LDA: Balancing relevance and diversity in TV content recommenders. 2015.

[45] Symeonidis, P., et al. Recommending the video to watch next: an offline and online evaluation at YOUTV. de. in Fourteenth ACM conference on recommender systems. 2020.

[46] Basilico, J. Recent Trends in Personalization: A Netflix Perspective. 2019. ICML.

[47] Covington, P., J. Adams, and E. Sargin. Deep neural networks for youtube recommendations. in Proceedings of the 10th ACM conference on recommender systems. 2016.

[48] Implementing the YouTube Recommendations Paper in TensorFlow — Part 1. Available from: https://theiconic.tech/implementing-the-youtube-recommendations-paper-in-tensorflow-part-1-d1e1299d5622.

[49] Krawiec, T., The Amazon Recommendations Secret to Selling More Online. Rejoiner http://rejoiner. com/resources/amazon-recommendations-secret-selling-online/website visited, 2018: p. 1-18.

[50] Mangalindan, J. Amazon’s recommendation secret. 2012; Available from: https://fortune.com/2012/07/30/amazons-recommendation-secret/.

[51] Linden, G., B. Smith, and J. York, Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, 2003. 7(1): p. 76-80.

[52] Baatarjav, E.-A., S. Phithakkitnukoon, and R. Dantu. Group recommendation system for facebook. in OTM Confederated International Conferences" On the Move to Meaningful Internet Systems". 2008. Springer.

[53] Facebook recommender system. Available from: https://github.com/facebookresearch/dlrm.

[54] Metaxiotis, K., et al., Decision support through knowledge management: the role of the artificial intelligence. Information Management & Computer Security, 2003.

[55] Wondercube® TV Broadcasting GSD/HD digital broadcast. Available from: http://www.jvc.slak.si/pdf/tv%20broadcasting.pdf.

[56] Recommendation Systems: Applications, Examples & Benefits. Available from: https://research.aimultiple.com/recommendation-system/#amazoncom.

[57] Optimizely. Available from: https://aimultiple.com/.

[58] CRM. Available from: https://www.cnbc.com/quotes/CRM.

[59] Clarifai. Available from: https://www.clarifai.com/.

[60] Jinni. Available from: http://www.jinni.com/.

[61] RottenTomatoes. Available from: www.rottentomatoes.com.

[62] MovieLens 100K Dataset. Available from: https://grouplens.org/datasets/movielens/100k/.

[63] Criticker. Available from: http://www.criticker.com/.

[64] Bestsimilar. Available from: bestsimilar.com.

[65] PEACH. 2021  [cited 2021 1 June 2021]; Available from: https://peach.ebu.io/.

[66] Sidana, S., Recommendation systems for online advertising. 2018, Université Grenoble Alpes.

[67] Netflix Prize Data Set. Available from: https://academictorrents.com/details/9b13183dc4d60676b773c9e2cd6de5e5542cee9a.

[68] Eliashberg, J., et al., Demand-driven scheduling of movies in a multiplex. International Journal of Research in Marketing, 2009. 26(2): p. 75-88.

[69] Zarandi, M.H.F., et al., A state of the art review of intelligent scheduling. Artificial Intelligence Review, 2020. 53(1): p. 501-593.

[70] Brown, D.E., J.A. Marin, and W.T. Scherer, A survey of intelligent scheduling systems, in Intelligent Scheduling Systems. 1995, Springer. p. 1-40.

[71] Horen, J.H., Scheduling of network television programs. Management Science, 1980. 26(4): p. 354-370.

[72] Welbank, M., A review of knowledge acquisition techniques for expert systems. 1983: Martlesham Consultancy Services Martlesham Heath, Ipswich.

[73] Piroozfard, H., K.Y. Wong, and A. Hassan, A hybrid genetic algorithm with a knowledge-based operator for solving the job shop scheduling problems. Journal of Optimization, 2016. 2016.

[74] Mitchell, T.M., Machine learning. 1997.

[75] Kocsis, T., et al., Case-Based Reasoning system for mathematical modelling options and resolution methods for production scheduling problems: Case representation, acquisition and retrieval. Computers & Industrial Engineering, 2014. 77: p. 46-64.

[76] Panaggio, M.J., et al., Prediction and Optimal Scheduling of Advertisements in Linear Television. arXiv preprint arXiv:1608.07305, 2016.

[77] Ghassemi Tari, F. and R. Alaei, Scheduling TV commercials using genetic algorithms. International Journal of Production Research, 2013. 51(16): p. 4921-4929.

[78] AlShami, H., Optimising television programming and scheduling. 2017: Lancaster University (United Kingdom).

[79] A case study of designing TV schedules. Available from: http://www.kr.inf.uc3m.es/wp-content/uploads/2019/12/AI4SE-Meer.pdf.

[80] Le, Q. and T. Mikolov. Distributed representations of sentences and documents. in International conference on machine learning. 2014. PMLR.

[81] He, X., et al. Neural collaborative filtering. in Proceedings of the 26th international conference on world wide web. 2017.

[82] Cheng, H.-T., et al. Wide & deep learning for recommender systems. in Proceedings of the 1st workshop on deep learning for recommender systems. 2016.

[83] Roshdy, Youssef, and Mennat Allah Hassan. "An Efficient Content-Based Video Recommendation." Journal of Computing and Communication 1.1 (2022): 48-64.

[84] Markapudi, Baburao, et al. "Content-based video recommendation system (CBVRS): a novel approach to predict videos using multilayer feed forward neural network and Monte Carlo sampling method." Multimedia Tools and Applications 82.5 (2023): 6965-6991.

[85] Ali, Shaik Faizan Roshan, et al. "Recommender System using Audio and Lyrics." 2023 4th International Conference on Electronics and Sustainable Communication Systems (ICESC). IEEE, 2023.

[86] Burch, Charats, Robert Sprowl, and Mehmet Ergezer. "A multi-user virtual world with music recommendations and mood-based virtual effects." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 13. 2023.

[87] Dudekula, Khasim Vali, et al. "Convolutional Neural Network-Based Personalized Program Recommendation System for Smart Television Users." Sustainability 15.3 (2023): 2206.

[88] Boeker, Maximilian and Aleksandra Urman. “An Empirical Investigation of Personalization Factors on TikTok.” Proceedings of the ACM Web Conference 2022 (2022): n. pag.

[89] Liu, Zhuoran, et al. "Monolith: real time recommendation system with collisionless embedding table." arXiv preprint arXiv:2209.07663 (2022).

[90] Ansari, Farooq. The Spotify Audio Features Hit Predictor Dataset (1960-2019). 1, 4TU.Centre for Research Data, 6 Apr. 2020, doi:10.4121/UUID:D77E74B0-66BC-47AC-8B25-5796D3084478.

[91] Moura, Luan; Fontelles, Emanuel; Sampaio, Vinicius; França, Mardônio (2020), “Music Dataset: Lyrics and Metadata from 1950 to 2019”, Mendeley Data, V2, doi: 10.17632/3t9vbwxgr5.2

[92] Deldjoo, Y., Schedl, M., Hidasi, B., Wei, Y., He, X. (2022). Multimedia Recommender Systems: Algorithms and Challenges. In: Ricci, F., Rokach, L., Shapira, B. (eds) Recommender Systems Handbook. Springer, New York, NY. https://doi.org/10.1007/978-1-0716-2197-4_25

[93] Deldjoo, Yashar and Schedl, Markus and Cremonesi, Paolo and Pasi, Gabriella, “Recommender Systems Leveraging Multimedia Content”, ACM Computing Surveys, 2020, 53, 5,

[94] A Survey of Multimedia Recommender Systems: Challenges and Opportunities Mouzhi Ge (1Faculty of Informatics, Masaryk University, Brno 60200, Czech Republic) and Fabio Persia (2Faculty of Computer Science, Free University of Bozen-Bolzano, Bozen-Bolzano, 39100, Italy) International Journal of Semantic Computing 2017 11:03, 411-428

[95] Jiang, Y., Xia, L., Wei, W., Luo, D., Lin, K., & Huang, C. (2024, October). Diffmm: Multi-modal diffusion model for recommendation. In Proceedings of the 32nd ACM International Conference on Multimedia (pp. 7591-7599).

[96] Malitesta, D., Cornacchia, G., Pomo, C., Merra, F. A., Di Noia, T., & Di Sciascio, E. (2025). Formalizing multimedia recommendation through multimodal deep learning. ACM Transactions on Recommender Systems, 3(3), 1-33.