Jul 28, 2017
In a previous post, I described teaching a humanoid to walk by imitating a fellow humanoid. Here I give my humanoid the capability to learn on its own. The humanoid’s only resources are the Proximal Policy Optimization algorithm, two randomly initialized neural networks, and a teacher that rewards forward progress. I also train nine other simulated robots to thrive in their environments: a swimming snake, a 4-legged “ant,” a reaching robot arm, a hopping robot, and several others. I have uploaded the results to the OpenAI Gym evaluation scoreboards, and the algorithm achieves several top scores.
Jun 29, 2017
In this post, I describe the results of implementing and training a variation of the VGG-16 convolutional neural network (convnet). The convnet is trained and evaluated on the Tiny ImageNet dataset. Tiny ImageNet spans 200 image classes with 500 training examples per class. The post also explores alternatives to the cross-entropy loss function. And, finally, I show pictures with their predictions vs. true labels, saliency maps, and visualizations the convolution filters.
May 18, 2017
Reinforcement Learning (RL) is what first sparked my interest in AI. It can take a human more than a year to learn to walk and even longer to master running. The challenge here is to teach a simulated human to run in a much shorter amount of time. In this post, I share some fun results and videos from teaching a MuJoCo-modeled humanoid to run in the OpenAI Gym.
Apr 4, 2017
I recently compared the performance of various RNN cells in a word prediction application. While doing this, I noticed that the Gated Recurrent Unit (GRU) ran slower per epoch than the LSTM cell. Based on the computation graphs for both cells, I expected the GRU to be a bit faster (also confirmed in literature). I launched an investigation into runtimes, including building TensorFlow from source, hand-building a GRU, laptop vs. Amazon EC2 GPU, and feeddict vs. QueueRunner.
Mar 26, 2017
I am nearly finished with my coursework in machine learning and artificial intelligence. As I wrap up, I thought I would recommend a curriculum for others that are just beginning a similar journey - incorporating the benefit of hindsight. My guiding philosophy is to build strong fundamental understanding. This leads to intuition, and ultimately the ability to creatively solve new problems in multiple areas.
Mar 25, 2017
Using QueueRunners is the proper way to keep a hungry TensorFlow model fed. You don’t want your high-performance compute resources waiting for data. QueueRunners are launched as independent threads. They look after themselves: refilling when they get low on data and closing when their input source has no more examples. I’ve built 2 notebooks to demonstrate the use of QueueRunners.
Mar 9, 2017
In my previous post I used a model similar to Skip-Gram to learn word vectors. For this project I built a RNN language model so I could experiment with RNN cell types and training methods. Because I recently watched a great TensorBoard demo from the TensorFlow Dev Summit, I’ve also added extensive TensorBoard visualization to this project.
Feb 4, 2017
A few months ago I jumped into a Kaggle competition at the last minute. My laptop didn’t have enough memory to handle the data set. And, I wanted more CPUs to throw at the problem. So now I had a great excuse to get myself set up to use Amazon cloud computing.
Feb 2, 2017
I recently completed a project using TensorFlow to learn word vectors from 3 Sherlock Holmes books. The code can be easily modified to “read” other books from Project Gutenberg. This project also contains some handy routines for exploring the learned word vectors.