Reinforcement_Learning

*Resources used

A Deep Reinforcement Learning Chatbot

• https://arxiv.org/abs/1709.02349

Deep Reinforcement Learning for recommender systems:

• https://arxiv.org/pdf/1801.00209.pdf
• https://arxiv.org/pdf/1810.12027.pdf (good literature review section)
• http://www.personal.psu.edu/~gjz5038/paper/www2018_reinforceRec/www2018_reinforceRec.pdf

Stanford CS 234: Reinforcement Learning

• https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&index=2&t=0s

Berkeley CS 285:

• http://rail.eecs.berkeley.edu/deeprlcourse/
• Associated github for the HW assignments: https://github.com/berkeleydeeprlcourse/homework_fall2020

Chapter 13 is on Policy Gradient Methods

David Silver's Deep Mind lectures are a good supplementary resource:

• https://www.youtube.com/playlist?list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ

Textbooks:

• Algorithms of Reinforcement Learning https://sites.ualberta.ca/~szepesva/rlbook.html

• Sutton and Barton's "Reinforcement Learning: An Introduction". Make sure you get the second edition (as of 2020). There are many pdfs online such as this one: https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf

Open AI:

https://spinningup.openai.com/en/latest/spinningup/rl_intro.html

This entire site is worth reading

Q learning:

Autonomous reinforcement learning from raw visual data, Lange & Riedmiller (2010) Q learning on top of latent space leared with autoencoder, uses fitted Q-iteration

"Human level control through deep reinforcement learning", Mnih et al (2013)

"Continous control with Deep Reinforcement Learning", Lillicrap et. al. (2015)

Classic papers

• Watkins. (1989). Learning from delayed rewards: introduces Q-learning
• Riedmiller. (2005). Neural fitted Q-iteration: batch-mode Q-learning with neural networks

Deep reinforcement learning Q-learning papers

• Lange, Riedmiller. (2010). Deep auto-encoder neural networks in reinforcement learning: early image-based Q-learning method using autoencoders to construct embeddings

  • http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.172.1873&rep=rep1&type=pdf

• Mnih et al. (2013). Human-level control through deep reinforcement learning: Qlearning with convolutional networks for playing Atari.

  • https://www.nature.com/articles/nature14236

• Van Hasselt, Guez, Silver. (2015). Deep reinforcement learning with double Q-learning: a very effective trick to improve performance of deep Q-learning.

  • https://arxiv.org/pdf/1509.06461.pdf

• Lillicrap et al. (2016). Continuous control with deep reinforcement learning: continuous Q-learning with actor network for approximate maximization.

  • https://arxiv.org/pdf/1509.02971.pdf

• Gu, Lillicrap, Stuskever, L. (2016). Continuous deep Q-learning with model-based acceleration: continuous Q-learning with action-quadratic value functions.

  • https://arxiv.org/pdf/1603.00748.pdf

• Wang, Schaul, Hessel, van Hasselt, Lanctot, de Freitas (2016). Dueling network architectures for deep reinforcement learning: separates value and advantage estimation in Q-function.

  • http://proceedings.mlr.press/v48/wangf16.pdf

Robots!

• "Robotic manipulation with Deep Reinforcement Learning ant...", Gu, Holly, et. al. (2017)

• "QT Opt: scalable Deep Reinforcement Learning of Vision-based Robotic Manipulation Skills". Kalashnikov, Irpan, Pastor

Recurrent models of visual attention:

• https://papers.nips.cc/paper/2014/file/09c6c3783b4a70054da74f2538ed47c6-Paper.pdf

Monte Carlo Tree Search:

• Browne, Powley, Whitehouse, Lucas, Cowling, Rohlfshagen, Tavener, Perez, Samothrakis, Colton. (2012). A Survey of Monte Carlo Tree Search Methods

Fun exercises

Blackjack! Also talked about in the David Silver lectures and chapter 5 of Sutton and Barto https://www.davidsilver.uk/wp-content/uploads/2020/03/Easy21-Johannes.pdf