# Tensorflow 2 code for Attention Mechanisms chapter of Dive into Deep Learning (D2L) book

This code has been merged with D2L book. See PR: 1756, 1768

This post contains Tensorflow 2 code for Attention Mechanisms chapter of Dive into Deep Learning (D2L) book. The chapter has 7 sections and code for each section can be found at the following links. We have given only code implementations. For theory, readers should refer the book.

10.1. Attention Cues

10.2. Attention Pooling: Nadaraya-Watson Kernel Regression

10.3. Attention Scoring Functions

10.4. Bahdanau Attention

10.6. Self-Attention and Positional Encoding

10.7. Transformer

9.7. Sequence to Sequence Learning

Note: We claim no originality for the code. Credit goes to the authors of this excellent book. However, all errors and omissions are my own and readers are encouraged to bring it to my notice. Finally, no TF code was available (to the best of my knowledge) for Attention Mechanisms chapter when this repo was first made public.