Feature Visualization

How neural networks build up their understanding of images.

Why Momentum Really Works

We often think of optimization with momentum as a ball rolling down a hill. This isn’t wrong, but there is much more to the story.

Experiments in Handwriting with a Neural Network

Several interactive visualizations of a generative model of handwriting. Some are fun, some are serious.

Deconvolution and Checkerboard Artifacts

When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts.

How to Use t-SNE Effectively

Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading.

Attention and Augmented Recurrent Neural Networks

A visual overview of neural attention, and the powerful extensions of neural networks being built on top of it.

Research Debt

Science is a human activity. When we fail to distill and explain research, we accumulate a kind of debt...

is dedicated to clear explanations of machine learning