Tutorials > Attention and Transformers from Scratch

Attention and Transformers from Scratch

Date: 29 August 2026 | Posted by: voska89

Free Download Attention and Transformers from Scratch
Published 8/2026
MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz, 2 Ch
Language: English | Duration: 3h 23m | Size: 1.52 GB
From RNN Seq2Seq to Bahdanau and Luong attention to a full Transformer — built, trained and explained in PyTorch

What you'll learn

Explain the fixed-context-vector problem of RNN Seq2Seq models and measure it yourself
Derive additive (Bahdanau) and multiplicative (Luong) attention and implement both in PyTorch
Read 'Attention Is All You Need' and map each component to code
Implement positional encoding, masking, multi-head attention, Add & Norm and feed-forward blocks
Assemble, train and use a complete Transformer for translation and inspect its cross-attention

Requirements

A computer with Python 3.10+ (a free Google Colab account is enough for most sessions)
Comfort reading Python code; you do not need to be an expert
Basic PyTorch and an idea of what an RNN does

Description

This course contains the use of artificial intelligence.
The voice-over in this course is synthesized with a text-to-speech model from scripts written and reviewed by the instructor, and the on-screen material (notebooks, code, slides) is the instructor's own work.
Attention is the idea behind every modern language model, and it was invented to solve one concrete problem: a sequence-to-sequence model squeezes a whole sentence into one fixed vector, and long sentences fall apart. This course starts from that problem and builds all the way up to a working Transformer.
We begin with what language models can do and the roadmap of an NLP engineer. Then we implement an RNN Seq2Seq model in PyTorch and measure its limits: the fixed context vector, what a wider hidden state does and does not fix. We derive the attention mechanism in depth — the core idea, the alignment weights, Bahdanau's additive attention — and implement Bahdanau and Luong attention with alignment visualisations. Next we read "Attention Is All You Need" section by section: self-attention, multi-head attention, positional encoding, feed-forward blocks, encoder and decoder. Finally we assemble a Transformer from parts — positional encoding, masking, multi-head attention, Add & Norm, encoder and decoder layers — train it, translate with it, and inspect cross-attention.
Theory sessions use slides; implementation sessions show the notebook on screen while the narration explains every cell. Small models that train in minutes, full understanding of what each line does.

Who this course is for

Developers who use Transformers and want to understand the mechanism
Students starting NLP or preparing for deep-learning interviews
Engineers who want to build a Transformer from parts rather than import one

Homepage

https://www.udemy.com/course/attention-and-transformers-from-scratch/

DOWNLOAD LINK

No Password - Links are Interchangeable

See full article