Watch a gradient vanish or explode as it travels back.
The gradient
Backpropagation through time multiplies the gradient by one factor per step, so it shrinks or grows geometrically. Raise w: the bound |w|k explodes, but tanh saturates, each factor w(1 − h²) collapses and the real gradient still vanishes on these illustrative inputs. Switch to the LSTM, whose cell state multiplies by the forget gate f, and set f near 1 to carry the gradient across all 30 steps.