Deep Reinforcement Learning based Recommender System in Tensorflow

The implemetation of Deep Reinforcement Learning based Recommender System from the paper Deep Reinforcement Learning based Recommendation with Explicit User-Item Interactions Modeling by Liu et al. Build recommender system with DDPG algorithm. Add state representation module to produce trainable state for RL algorithm from data. This is not the official implementation of the paper.

Dataset

MovieLens 1M Datset

unzip ./ml-1m.zip

Procedure

• Trying to improve performance of RL based recommender system. The report contains the result of Using the actor network with embedding layer, reducing overestimated Q value, using several pretrained embedding and applying PER.

• Making new embedding files. Previous one contains the information for entire timelines which can mislead model.

• Updating Train and Evaluation part. Currently, I didn't follow the exact same training and evaluation procedure in the paper.

Result

Please check here - Experiment Report (Korean)

• for evalutation data
  • precision@5 : 0.479, ndcg@5 : 0.471
  • precision@10 : 0.444, ndcg@10 : 0.429

Usage

Training

• The saved model of actor and critic are generated after the training is done.

python train.py

Evalutation

• Make sure there exist the saved models in the right directory
• Run jupyter notebook and check "evaluation.ipynb"

requirements

tensorflow==2.5.0
scikit-learn==0.23.2
matplotlib==3.3.3

reference

https://github.com/LeejwUniverse/RL_Rainbow_Pytorch

https://github.com/kyunghoon-jung/MacaronRL

https://github.com/pasus/Reinforcement-Learning-Book