Implementing the Transformer Encoder from Scratch in TensorFlow and Keras
Having seen how to implement the scaled dot-product attention and integrate it within the multi-head attention of the Transformer model, let’s progress one step further toward implementing a complete Transformer model by applying its encoder. Our end goal remains to apply the complete model to Natural Language Processing (NLP).
In this tutorial, you will discover how to implement the Transformer encoder from scratch in TensorFlow and Keras.
After completing this tutorial, you will know:
- The layers that form part of the Transformer encoder.
- How to implement the Transformer encoder from scratch.
Kick-start your project with my book Building Transformer Models with Attention. It provides self-study tutorials with working code to guide you