Multimodal Neurons in Artificial Neural Networks

Gabriel Goh, Nick Cammarata †, Chelsea Voss †, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah

We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain.

Understanding RL Vision

Jacob Hilton, Nick Cammarata, Shan Carter, Gabriel Goh, and Chris Olah

With diverse environments, we can analyze, diagnose and edit deep reinforcement learning models using attribution.

Communicating with Interactive Articles

Fred Hohman, Matthew Conlen, Jeffrey Heer, and Duen Horng (Polo) Chau

Examining the design of interactive articles by synthesizing theory from disciplines such as education, journalism, and visualization.

Thread: Differentiable Self-organizing Systems

Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, Michael Levin, and Sam Greydanus

A collection of articles and comments with the goal of understanding how to design robust and general purpose self-organizing systems.

Exploring Bayesian Optimization

Apoorv Agnihotri and Nipun Batra

How to tune hyperparameters for your machine learning model using Bayesian optimization.

Visualizing Neural Networks with the Grand Tour

Mingwei Li, Zhenge Zhao, and Carlos Scheidegger

By focusing on linear dimensionality reduction, we show how to visualize many dynamic phenomena in neural networks.

Thread: Circuits

Nick Cammarata, Shan Carter, Gabriel Goh, Chris Olah, Michael Petrov, and Ludwig Schubert

What can we learn if we invest heavily in reverse engineering a single neural network?

Visualizing the Impact of Feature Attribution Baselines

Pascal Sturmfels, Scott Lundberg, and Su-In Lee

Exploring the baseline input hyperparameter, and how it impacts interpretations of neural network behavior.

Computing Receptive Fields of Convolutional Neural Networks

André Araujo, Wade Norris, and Jack Sim

Detailed derivations and open-source code to analyze the receptive fields of convnets.

The Paths Perspective on Value Learning

Sam Greydanus and Chris Olah

A closer look at how Temporal Difference Learning merges paths of experience for greater statistical efficiency

A Discussion of ‘Adversarial Examples Are Not Bugs, They Are Features’

Logan Engstrom, Justin Gilmer, Gabriel Goh, Dan Hendrycks, Andrew Ilyas, Aleksander Madry, Reiichiro Nakano, Preetum Nakkiran, Shibani Santurkar, Brandon Tran, Dimitris Tsipras, and Eric Wallace

Six comments from the community and responses from the original authors

Open Questions about Generative Adversarial Networks

Augustus Odena

What we’d like to find out about GANs that we don’t know yet.

A Visual Exploration of Gaussian Processes

Jochen Görtler, Rebecca Kehlbeck, and Oliver Deussen

How to turn a collection of small building blocks into a versatile tool for solving regression problems.

Visualizing memorization in RNNs

Andreas Madsen

Inspecting gradient magnitudes in context can be a powerful tool to see when recurrent units use short-term or long-term contextual understanding.

Activation Atlas

Shan Carter, Zan Armstrong, Ludwig Schubert, Ian Johnson, and Chris Olah

By using feature inversion to visualize millions of activations from an image classification network, we create an explorable activation atlas of features the network has learned and what concepts it typically represents.

AI Safety Needs Social Scientists

Geoffrey Irving and Amanda Askell

If we want to train AI to do what humans want, we need to study humans.

Distill Update 2018

Distill Editors

An Update from the Editorial Team

Differentiable Image Parameterizations

Alexander Mordvintsev, Nicola Pezzotti, Ludwig Schubert, and Chris Olah

A powerful, under-explored tool for neural network visualizations and art.

Feature-wise transformations

Vincent Dumoulin, Ethan Perez, Nathan Schucher, Florian Strub, Harm de Vries, Aaron Courville, and Yoshua Bengio

A simple and surprisingly effective family of conditioning mechanisms.

The Building Blocks of Interpretability

Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schubert, Katherine Ye, and Alexander Mordvintsev

Interpretability techniques are normally studied in isolation. We explore the powerful interfaces that arise when you combine them — and the rich structure of this combinatorial space.

Using Artificial Intelligence to Augment Human Intelligence

Shan Carter and Michael Nielsen

By creating user interfaces which let us work with the representations inside machine learning models, we can give people new tools for reasoning.

Sequence Modeling with CTC

Awni Hannun

A visual guide to Connectionist Temporal Classification, an algorithm used to train deep neural networks in speech recognition, handwriting recognition and other sequence problems.

Feature Visualization

Chris Olah, Alexander Mordvintsev, and Ludwig Schubert

How neural networks build up their understanding of images

Why Momentum Really Works

Gabriel Goh

We often think of optimization with momentum as a ball rolling down a hill. This isn’t wrong, but there is much more to the story.

Research Debt

Chris Olah and Shan Carter

Science is a human activity. When we fail to distill and explain research, we accumulate a kind of debt...

Experiments in Handwriting with a Neural Network

Shan Carter, David Ha, Ian Johnson, and Chris Olah

Several interactive visualizations of a generative model of handwriting. Some are fun, some are serious.

Deconvolution and Checkerboard Artifacts

Augustus Odena, Vincent Dumoulin, and Chris Olah

When we look very closely at images generated by neural networks, we often see a strange checkerboard pattern of artifacts.

How to Use t-SNE Effectively

Martin Wattenberg, Fernanda Viégas, and Ian Johnson

Although extremely useful for visualizing high-dimensional data, t-SNE plots can sometimes be mysterious or misleading.

Attention and Augmented Recurrent Neural Networks

Chris Olah and Shan Carter

A visual overview of neural attention, and the powerful extensions of neural networks being built on top of it.

is dedicated to clear explanations of machine learning