We often think of optimization with momentum as a ball rolling down a hill. This isn’t wrong, but there is much more to the story.
Several interactive visualizations of a generative model of handwriting. Some are fun, some are serious.
When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts.
Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading.
A visual overview of neural attention, and the powerful extensions of neural networks being built on top of it.
Science is a human activity. When we fail to distill and explain research, we accumulate a kind of debt...