tutorials

A few months ago I attempted to understand the natural gradient, and wrote a post to help organize what I knew. Unfortunately there was too little detail and all I really understood was a "black box" version of the natural gradient: what

A few weeks ago I made a post on variational autoencoders, and how they can be applied to image generation. In this post, we'll be taking a look at DRAW: a model based off of the VAE that generates images using a sequence of

A term that sometimes shows up in machine learning is the "natural gradient". While there hasn't been much of a focus on using it in practice, a variety of algorithms can be shown as a variation of the natural gradient. In this

In my previous post about generative adversarial networks, I went over a simple method to training a network that could generate realistic-looking images. However, there were a couple of downsides to using a plain GAN. First, the images are generated off some arbitrary noise.

I've been experimenting with OpenAI gym recently, and one of the simplest environments is CartPole. The problem consists of balancing a pole connected with one joint on top of a moving cart. The only actions are to add a force of -1 or +1

There's been a lot of advances in image classification, mostly thanks to the convolutional neural network. It turns out, these same networks can be turned around and applied to image generation as well. If we've got a bunch of images, how can we generate

What really is a model? In short, it's an agent's understanding of the surrounding environment. In Markov processes, a model can be represented as probabilities of what will happen if the agent takes action A from state S. Implementing a model into your reinforcement

So far, our policy has simply been to act greedily on some value function. What if we tried to learn the policy itself? We can represent this policy as the probability to take a certain action, given the state. π = P (a|s) This

It's been shown many times that convolutional neural nets are very good at recognizing patterns in order to classify images. But what patterns are they actually looking for? I attempted to recreate the techniques described in Visualizing and Understanding Convolutional Networks to project features

The problem with the methods covered earlier is that it requires a model. Oftentimes, the agent does not know how the environment works and must figure it out by themselves. Model-free prediction is predicting the value function of a certain policy without a concrete

Let's say we've got a Markov Decision Process, and a policy π. How good is this policy? Well, one way to figure it out is by simply iterating the policy over a value function for each state. In this example, we have a small

When solving reinforcement learning problems, there has to be a way to actually represent states in the environment. A Markov State is a bunch of data that not only contains information about the current state of the environment, but all useful information from the

In the past, there have been two main kinds of machine learning. In supervised learning, the computer is given both data and labels, and is simply tasked to find patterns that can map the input data to each corresponding label. In unsupervised learning, there

The paper A Neural Algorithm of Artistic Style detailed on how to extract two sets of features from a given image: the content, and and the style. In convolutional neural networks, each layer stores information in an abstraction based on the previous layer. For