Attention Mechanisms: Stop Compressing, Start Looking Back

This article delves into the limitations of LSTMs in maintaining context, even with their improved memory capabilities over vanilla RNNs. The author uses a personal experience of learning English to illustrate the three specific problems LSTMs still don't solve, setting the stage for discussing attention mechanisms.