强化学习(RL)的最新发展结合深度学习(DL)，在训练代理以类似人的方式解决复杂问题方面取得了****的进步。Google使用算法在著名的Atari街机游戏中获胜将该领域推至高峰，研究人员也在源源不断地产生新的想法。
本书是关于最新DL工具及其局限性的全面指南。在应用于真实环境之前，你得评估包括交叉熵和策略梯度在内的多种方法。试试Atari的虚拟游戏和像connect4这样的家庭最爱。本书介绍了RL的基础知识，为你提供了编写智能学习代理所需的原理，以承担一系列艰巨的实际任务。让你了解如何在“网格世界”环境中实现Q-learning，教你的代理购买和交易股票，发现自然语言模型如何推动了聊天机器人的火爆。
马克西姆·拉潘，深度学习研究者，作为一名软件开发人员和系统架构师，具有超过15年的专业经验，涵盖了从Linux内核驱动程序开发到可在数千台服务器上工作的分布式应用项目的设计与性能优化。他在大数据、机器学习以及大型并行分布式HPC系统方面拥有丰富的工作经验，并擅长使用简单的文字和生动的示例来解释复杂事物。他目前专注的领域是深度学习的实际应用，例如深度自然语言处理和深度强化学习。Maxim目前在以色列一家初创公司工作，担任高级NLP开发人员。
Preface
Chapter 1: What is Reinforcement Learning?
Learning - supervised, unsupervised, and reinforcement
RL formalisms and relations
Reward
The agent
The environment
Actions
Observations
Markov decision processes
Markov process
Markov reward process
Markov decision process
Summary
Chapter 2: OpenAI Gym
The anatomy of the agent
Hardware and software requirements
OpenAI Gym API
Action space
Observation space
The environment
Creation of the environment
The CartPole session
The random CartPole agent
The extra Gym functionality - wrappers and monitors
Wrappers
Monitor
Summary
Chapter 3: Deep Learning with PyTorch
Tensors
Creation of tensors
Scalar tensors
Tensor operations
GPU tensors
Gradients
Tensors and gradients
NN building blocks
Custom layers
Final glue - loss functions and optimizers
Loss functions
Optimizers
Monitoring with TensorBoard
TensorBoard 101
Plotting stuff
Example -GAN on Atari images
Summary
Chapter 4: The Cross-Entropy Method
Taxonomy of RL methods
Practical cross-entropy
Cross-entropy on CartPole
Cross-entropy on FrozenLake
Theoretical background of the cross-entropy method
Summary
Chapter 5: Tabular Learning and the Bellman Equation
Value, state, and optimality
The Bellman equation of optimality
Value of action
The value iteration method
Value iteration in practice
Q-learning for FrozenLake
Summary
Chapter 6: Deep Q-Networks
Chapter 7: DQN Extensions
Chapter 8: Stocks Trading Using RL
Chapter 9: Policy Gradients - An Alternative
Chapter 10: The Actor-Critic Method
Chapter 11: Asynchronous Advantaqe Actor-Critic
Chapter 12: Chatbots Training with RL
Chapter 13: Web Navigation
Chapter 14: Continuous Action Space
Chapter 15: Trust Regions - TRPO, PPO, and ACKTR
Chapter 16: Black-Box Optimization in RL
Chapter 17: Beyond Model-Free - Imagination
Chapter 18: AlphaGo Zero
Other Books You May Enjoy
Index
