Latest Articles
Why Momentum Really Works
We often think of optimization with momentum as a ball rolling down a hill. This isn’t wrong, but there is much more to the story.
Experiments in Handwriting with a Neural Network
Several interactive visualizations of a generative model of handwriting. Some are fun, some are serious.
Deconvolution and Checkerboard Artifacts
When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts.
How to Use t-SNE Effectively
Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading.
Attention and Augmented Recurrent Neural Networks
A visual overview of neural attention, and the powerful extensions of neural networks being built on top of it.
Commentary
Research Debt
Science is a human activity. When we fail to distill and explain research, we accumulate a kind of debt...