Long Short-Term Memory
Hochreiter & Schmidhuber, 1997, Long Short-Term Memory (Neural Computation)
Subscriber deck — free during early access
1 cards in “4.6 Computational complexity” · whole deck
4.6 Computational complexity
- What is LSTM’s computational complexity, and what does “local in space and time” mean?\(O(W)\) per time step (\(O(1)\) per weight), like BPTT and far better than RTRL. Local in space: update cost per weight doesn’t depend on network size. Local in time: storage doesn’t depend on sequence length, unlike full BPTT, which must store activations for the whole sequence.
- Hochreiter & Schmidhuber, 1997, Long Short-Term Memory (Neural Computation), §4.6
Source
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. In Neural Computation, pp. 1735–1780. doi:10.1162/neco.1997.9.8.1735