Feb
From MLPs to WaveNet: Why Squashing Information Kills Learning

I was watching Karpathy’s tutorial nn-zero-to-hero.

When moving from a simple Multi-Layer Perceptron (MLP) language model to a WaveNet (Convolutional)...