Lil'Log
🦄 Contact
🙋 FAQ
⌛ Archive
  • Mar 21, 2021 nlp  language-model  safety 

    Reducing Toxicity in Language Models

    Toxicity prevents us from safely deploying powerful pretrained language models for real-world applications. To reduce toxicity in language models, in this post, we will delve into three aspects of the problem: training dataset collection, toxic content detection and model detoxification.

  • Jan 2, 2021 nlp  language-model  reinforcement-learning  long-read 

    Controllable Neural Text Generation

    The modern language model with SOTA results on many NLP tasks is trained on large scale free text on the Internet. It is challenging to steer such a model to generate content with desired attributes. Although still not perfect, there are several approaches for controllable text generation, such as guided or learned decoding strategy, smart prompt design, or fine-tuning the model with various methods.

  • Oct 29, 2020 nlp  language-model  attention  transformer 

    How to Build an Open-Domain Question Answering System?

    A model that is capable of answering any question with regard to factual knowledge can enable many useful applications. This post delves into how we can build an Open-Domain Question Answering (ODQA) system, assuming we have access to a powerful pretrained language model. Both closed-book and open-book approachs are discussed.

  • Aug 6, 2020 reinforcement-learning  evolution  auto-ML 

    Neural Architecture Search

    Neural Architecture Search (NAS) automates network architecture engineering. It aims to learn a network topology that can achieve best performance on a certain task. By dissecting the methods for NAS into three components: search space, search algorithm and child model evolution strategy, this post reviews many interesting ideas for better, faster and more cost-efficient automatic neural architecture search.

  • Jun 7, 2020 reinforcement-learning  exploration  long-read 

    Exploration Strategies in Deep Reinforcement Learning

    Exploitation versus exploration is a critical topic in reinforcement learning. This post introduces several common approaches for better exploration in Deep RL.

  • Apr 7, 2020 attention  transformer  reinforcement-learning 

    The Transformer Family

    Inspired by recent progress on various enhanced versions of Transformer models, this post presents how the vanilla Transformer can be improved for longer-term attention span, less memory and computation consumption, RL task solving, etc.

  • Jan 29, 2020 reinforcement-learning  generative-model  meta-learning 

    Curriculum for Reinforcement Learning

    A curriculum is an efficient tool for humans to progressively learn from simple concepts to hard problems. It breaks down complex knowledge by providing a sequence of learning steps of increasing difficulty. In this post, we will examine how the idea of curriculum can help reinforcement learning models learn to solve complicated tasks.

  • Nov 10, 2019 representation-learning  long-read  generative-model  object-recognition  reinforcement-learning 

    Self-Supervised Representation Learning

    Self-supervised learning opens up a huge opportunity for better utilizing unlabelled data, while learning in a supervised learning manner. This post covers many interesting ideas of self-supervised learning tasks on images, videos, and control problems.

  • Sep 5, 2019 evolution  reinforcement-learning 

    Evolution Strategies

    Gradient descent is not the only option when learning optimal model parameters. Evolution Strategies (ES) works out well in the cases where we don’t know the precise analytic form of an objective function or cannot compute the gradients directly. This post dives into several classic ES methods, as well as how ES can be used in deep reinforcement learning.

  • Jun 23, 2019 meta-learning  reinforcement-learning 

    Meta Reinforcement Learning

    Meta-RL is meta-learning on reinforcement learning tasks. After trained over a distribution of tasks, the agent is able to solve a new task by developing a new RL algorithm with its internal activity dynamics. This post starts with the origin of meta-RL and then dives into three key components of meta-RL.

  • May 5, 2019 robotics  meta-learning  reinforcement-learning 

    Domain Randomization for Sim2Real Transfer

    If a model or policy is mainly trained in a simulator but expected to work on a real robot, it would surely face the sim2real gap. Domain Randomization (DR) is a simple but powerful idea of closing this gap by randomizing properties of the training environment.

  • Mar 14, 2019 information-theory  foundation 

    Are Deep Neural Networks Dramatically Overfitted?

    If you are, like me, confused by why deep neural networks can generalize to out-of-sample data points without drastic overfitting, keep on reading.

  • Jan 31, 2019 nlp  long-read  transformer  attention  language-model 

    Generalized Language Models

    As a follow up of word embedding post, we will discuss the models on learning contextualized word vectors, as well as the new trend in large unsupervised pre-trained language models which have achieved amazing SOTA results on a variety of language tasks.

  • Dec 27, 2018 object-detection  object-recognition 

    Object Detection Part 4: Fast Detection Models

    Part 4 of the “Object Detection for Dummies” series focuses on one-stage models for fast detection, including SSD, RetinaNet, and models in the YOLO family. These models skip the explicit region proposal stage but apply the detection directly on dense sampled areas.

  • Nov 30, 2018 meta-learning  long-read 

    Meta-Learning: Learning to Learn Fast

    Meta-learning, also known as “learning to learn”, intends to design models that can learn new skills or adapt to new environments rapidly with a few training examples. There are three common approaches: 1) learn an efficient distance metric (metric-based); 2) use (recurrent) network with external or internal memory (model-based); 3) optimize the model parameters explicitly for fast learning (optimization-based).

  • Oct 13, 2018 generative-model 

    Flow-based Deep Generative Models

    In this post, we are looking into the third type of generative models: flow-based generative models. Different from GAN and VAE, they explicitly learn the probability density function of the input data.

  • Aug 12, 2018 autoencoder  generative-model 

    From Autoencoder to Beta-VAE

    Autocoders are a family of neural network models aiming to learn compressed latent variables of high-dimensional data. Starting from the basic autocoder model, this post reviews several variations, including denoising, sparse, and contractive autoencoders, and then Variational Autoencoder (VAE) and its modification beta-VAE.

  • Jun 24, 2018 attention  transformer  rnn 

    Attention? Attention!

    Attention has been a fairly popular concept and a useful tool in the deep learning community in recent years. In this post, we are gonna look into how attention was invented, and various attention mechanisms and models, such as transformer and SNAIL.

  • May 5, 2018 tutorial  tensorflow  reinforcement-learning 

    Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI Gym

    Let’s see how to implement a number of classic deep reinforcement learning models in code.

  • Apr 8, 2018 reinforcement-learning  long-read 

    Policy Gradient Algorithms

    Abstract: In this post, we are going to look deep into policy gradient, why it works, and many new policy gradient algorithms proposed in recent years: vanilla policy gradient, actor-critic, off-policy actor-critic, A3C, A2C, DPG, DDPG, D4PG, MADDPG, TRPO, PPO, ACER, ACTKR, SAC, TD3 & SVPG.

  • Feb 19, 2018 reinforcement-learning  long-read 

    A (Long) Peek into Reinforcement Learning

    In this post, we are gonna briefly go over the field of Reinforcement Learning (RL), from fundamental concepts to classic algorithms. Hopefully, this review is helpful enough so that newbies would not get lost in specialized terms and jargons while starting. [WARNING] This is a long read.

  • Jan 23, 2018 reinforcement-learning  exploration 

    The Multi-Armed Bandit Problem and Its Solutions

    The multi-armed bandit problem is a class example to demonstrate the exploration versus exploitation dilemma. This post introduces the bandit problem and how to solve it using different exploration strategies.

  • Dec 31, 2017 object-detection  object-recognition 

    Object Detection for Dummies Part 3: R-CNN Family

    In Part 3, we would examine four object detection models: R-CNN, Fast R-CNN, Faster R-CNN, and Mask R-CNN. These models are highly related and the new versions show great speed improvement compared to the older ones.

  • Dec 15, 2017 object-detection  object-recognition 

    Object Detection for Dummies Part 2: CNN, DPM and Overfeat

    Part 2 introduces several classic convolutional neural work architecture designs for image classification (AlexNet, VGG, ResNet), as well as DPM (Deformable Parts Model) and Overfeat models for object recognition.

  • Oct 29, 2017 object-detection  object-recognition 

    Object Detection for Dummies Part 1: Gradient Vector, HOG, and SS

    In this series of posts on “Object Detection for Dummies”, we will go through several basic concepts, algorithms, and popular deep learning models for image processing and objection detection. Hopefully, it would be a good read for people with no experience in this field but want to learn more. The Part 1 introduces the concept of Gradient Vectors, the HOG (Histogram of Oriented Gradients) algorithm, and Selective Search for image segmentation.

  • Oct 15, 2017 nlp  language-model 

    Learning Word Embedding

    Word embedding is a dense representation of words in the form of numeric vectors. It can be learned using a variety of language models. The word embedding representation is able to reveal many hidden relationships between words. For example, vector(“cat”) - vector(“kitten”) is similar to vector(“dog”) - vector(“puppy”). This post introduces several models for learning word embedding and how their loss functions are designed for the purpose.

  • Sep 28, 2017 information-theory  foundation 

    Anatomize Deep Learning with Information Theory

    This post is a summary of Prof Naftali Tishby’s recent talk on “Information Theory in Deep Learning”. It presented how to apply the information theory to study the growth and transformation of deep neural networks during training.

  • Aug 20, 2017 gan  long-read  generative-model 

    From GAN to WGAN

    This post explains the maths behind a generative adversarial network (GAN) model and why it is hard to be trained. Wasserstein GAN is intended to improve GANs’ training by adopting a smooth metric for measuring the distance between two probability distributions.

  • Aug 1, 2017 foundation 

    How to Explain the Prediction of a Machine Learning Model?

    This post reviews some research in model interpretability, covering two aspects: (i) interpretable models with model-specific interpretation methods and (ii) approaches of explaining black-box models. I included an open discussion on explainable artificial intelligence at the end.

  • Jul 22, 2017 tutorial  rnn  tensorflow 

    Predict Stock Prices Using RNN: Part 2

    This post is a continued tutorial for how to build a recurrent neural network using Tensorflow to predict stock market prices. Part 2 attempts to predict prices of multiple stocks using embeddings. The full working code is available in lilianweng/stock-rnn.

  • Jul 8, 2017 tutorial  rnn  tensorflow 

    Predict Stock Prices Using RNN: Part 1

    This post is a tutorial for how to build a recurrent neural network using Tensorflow to predict stock market prices. Part 1 focuses on the prediction of S&P 500 index. The full working code is available in lilianweng/stock-rnn.

  • Jun 21, 2017 foundation  tutorial 

    An Overview of Deep Learning for Curious People

    Starting earlier this year, I grew a strong curiosity of deep learning and spent some time reading about this field. To document what I’ve learned and to provide some interesting pointers to people with similar interests, I wrote this overview of deep learning models and their applications.

2021 © by Lilian Weng. All Rights Reserved. Built by Jekyll. | View on Github | Tags | Contact | FAQ