Posts

• Aug 12, 2018 review  autoencoder 

  From Autoencoder to Beta-VAE

  Autocoders are a family of neural network models aiming to learn compressed latent variables of high-dimensional data. Starting from the basic autocoder model, this post reviews several variations, including denoising, sparse, and contractive autoencoders, and then Variational Autoencoder (VAE) and its modification $\beta$-VAE.

• Jun 24, 2018 review 

  Attention? Attention!

  Attention has been a fairly popular concept and a useful tool in the deep learning community in recent years. In this post, we are gonna look into how attention was invented, and various attention mechanisms and models, such as transformer and SNAIL.

• May 5, 2018 tutorial  tensorflow  reinforcement-learning 

  Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI Gym

  Let’s see how to implement a number of classic deep reinforcement learning models in code. The full implementation is available in lilianweng/deep-reinforcement-learning-gym

• Apr 8, 2018 reinforcement-learning  review 

  Policy Gradient Algorithms

  Abstract: In this post, we are going to look deep into policy gradient, why it works, and many new policy gradient algorithms proposed in recent years: vanilla policy gradient, actor-critic, off-policy actor-critic, A3C, A2C, DPG, DDPG, D4PG, MADDPG, TRPO, PPO, ACER, ACTKR, and Soft AC.

• Feb 19, 2018 reinforcement-learning  review  long-read 

  A (Long) Peek into Reinforcement Learning

  In this post, we are gonna briefly go over the field of Reinforcement Learning (RL), from fundamental concepts to classic algorithms. Hopefully, this review is helpful enough so that newbies would not get lost in specialized terms and jargons while starting. [WARNING] This is a long read.

• Jan 23, 2018 reinforcement-learning 

  The Multi-Armed Bandit Problem and Its Solutions

  The multi-armed bandit problem is a class example to demonstrate the exploration versus exploitation dilemma. This post introduces the bandit problem and how to solve it using different exploration strategies. The algorithms are implemented for Bernoulli bandit in lilianweng/multi-armed-bandit.

• Dec 31, 2017 object-recognition 

  Object Recognition for Dummies Part 3: R-CNN and Fast/Faster/Mask R-CNN and YOLO

  In Part 3, we would examine five object recognition models: R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN and YOLO. These models are highly related and the new versions show great speed improvement compared to the older ones.

• Dec 15, 2017 object-recognition 

  Object Recognition for Dummies Part 2: CNN, DPM and Overfeat

  Part 2 introduces several classic convolutional neural work architecture designs for image classification (AlexNet, VGG, ResNet), as well as DPM (Deformable Parts Model) and Overfeat models for object recognition.

• Oct 29, 2017 object-recognition 

  Object Recognition for Dummies Part 1: Gradient Vector, HOG, and SS

  In this series of posts on “Object Recognition for Dummies”, we will go through several basic concepts, algorithms, and popular deep learning models for image processing and objection detection. Hopefully, it would be a good read for people with no experience in this field but want to learn more. The Part 1 introduces the concept of Gradient Vectors, the HOG (Histogram of Oriented Gradients) algorithm, and Selective Search for image segmentation.

• Oct 15, 2017 review 

  Learning Word Embedding

  Word embedding is a dense representation of words in the form of numeric vectors. It can be learned using a variety of language models. The word embedding representation is able to reveal many hidden relationships between words. For example, vector(“cat”) - vector(“kitten”) is similar to vector(“dog”) - vector(“puppy”). This post introduces several models for learning word embedding and how their loss functions are designed for the purpose.

• Sep 28, 2017 information-theory  review 

  Anatomize Deep Learning with Information Theory

  This post is a summary of Prof Naftali Tishby’s recent talk on “Information Theory in Deep Learning”. It presented how to apply the information theory to study the growth and transformation of deep neural networks during training.

• Aug 20, 2017 gan  long-read 

  From GAN to WGAN

  This post explains the maths behind a generative adversarial network (GAN) model and why it is hard to be trained. Wasserstein GAN is intended to improve GANs’ training by adopting a smooth metric for measuring the distance between two probability distributions.

• Aug 1, 2017 review 

  How to Explain the Prediction of a Machine Learning Model?

  This post reviews some research in model interpretability, covering two aspects: (i) interpretable models with model-specific interpretation methods and (ii) approaches of explaining black-box models. I included an open discussion on explainable artificial intelligence at the end.

• Jul 22, 2017 tutorial  rnn  tensorflow 

  Predict Stock Prices Using RNN: Part 2

  This post is a continued tutorial for how to build a recurrent neural network using Tensorflow to predict stock market prices. Part 2 attempts to predict prices of multiple stocks using embeddings. The full working code is available in lilianweng/stock-rnn.

• Jul 8, 2017 tutorial  rnn  tensorflow 

  Predict Stock Prices Using RNN: Part 1

  This post is a tutorial for how to build a recurrent neural network using Tensorflow to predict stock market prices. Part 1 focuses on the prediction of S&P 500 index. The full working code is available in lilianweng/stock-rnn.

• Jun 21, 2017 review 

  An Overview of Deep Learning for Curious People

  Starting earlier this year, I grew a strong curiosity of deep learning and spent some time reading about this field. To document what I’ve learned and to provide some interesting pointers to people with similar interests, I wrote this overview of deep learning models and their applications.