Deep Learning RNN Cheat Sheet

Neural Networks has various variants like CNN (Convolutional Neural Networks), RNN (Recurrent Neural Networks), AutoEncoders etc. RNN are designed to work with sequence prediction problems (One to Many, Many to Many, Many to One). RNN is recurrent as it performs the same task for every element of a sequence, with the output being dependent on the previous computations. As a result, we used it in Natural Language Processing problems a lot. RNN is used for following:

• Text data
• Speech data
• Classification prediction problems
• Regression prediction problems
• Generative models

This cheatsheet includes following in brief:

• Architecture of a traditional RNN
• Types of RNN
• Commonly used activation functions
• GRU/LSTM
• Vanishing/exploding gradient
• Learning word representation: Representation techniques, Embedding matrix
• Comparing words: Cosine similarity, t-SNE
• Language model: n-gram model, Perplexity
• Attention: Attention model, Attention weight
• Machine translation: Beam search, Beam width, Length normalization, Error analysis