• views: 149
• date: 27 May 2025
• posted by: voska89

Master the inner workings of how large language models like GPT really work with hands-on coding sessions led by bestselling author Sebastian Raschka. These companion videos to Build a Large Language Model from Scratch walk you through real-world implementation, with each session ending in a "test yourself" challenge to solidify your skills and deepen your understanding.
Table of contents
Chapter 1. Python Environment Setup
Chapter 2. Tokenizing text
Chapter 2. Converting tokens into token IDs
Chapter 2. Adding special context tokens
Chapter 2. Byte pair encoding
Chapter 2. Data sampling with a sliding window
Chapter 2. Creating token embeddings
Chapter 2. Encoding word positions
Chapter 3. A simple self-attention mechanism without trainable weights | Part 1
Chapter 3. A simple self-attention mechanism without trainable weights | Part 2
Chapter 3. Computing the attention weights step by step
Chapter 3. Implementing a compact self-attention Python class
Chapter 3. Applying a causal attention mask
Chapter 3. Masking additional attention weights with dropout
Chapter 3. Implementing a compact causal self-attention class
Chapter 3. Stacking multiple single-head attention layers
Chapter 3. Implementing multi-head attention with weight splits
Chapter 4. Coding an LLM architecture
Chapter 4. Normalizing activations with layer normalization
Chapter 4. Implementing a feed forward network with GELU activations
Chapter 4. Adding shortcut connections
Chapter 4. Connecting attention and linear layers in a transformer block
Chapter 4. Coding the GPT model
Chapter 4. Generating text
Chapter 5. Using GPT to generate text
Chapter 5. Calculating the text generation loss: cross entropy and perplexity
Chapter 5. Calculating the training and validation set losses
Chapter 5. Training an LLM
Chapter 5. Decoding strategies to control randomness
Chapter 5. Temperature scaling
Chapter 5. Top-k sampling
Chapter 5. Modifying the text generation function
Chapter 5. Loading and saving model weights in PyTorch
Chapter 5. Loading pretrained weights from OpenAI
Chapter 6. Preparing the dataset
Chapter 6. Creating data loaders
Chapter 6. Initializing a model with pretrained weights
Chapter 6. Adding a classification head
Chapter 6. Calculating the classification loss and accuracy
Chapter 6. Fine-tuning the model on supervised data
Chapter 6. Using the LLM as a spam classifier
Chapter 7. Preparing a dataset for supervised instruction fine-tuning
Chapter 7. Organizing data into training batches
Chapter 7. Creating data loaders for an instruction dataset
Chapter 7. Loading a pretrained LLM
Chapter 7. Fine-tuning the LLM on instruction data
Chapter 7. Extracting and saving responses
Chapter 7. Evaluating the fine-tuned LLM





   
   
   

Forex Trading

Crypto Trading. 20 USD Bonus after KYC

We need your support!
Make a donation to help us stay online
        
Bitcoin (BTC)
bc1q08g9d22cxkawsjlf8etuek2pc9n2a3hs4cdrld
	
Bitcoin Cash (BCH)
qqvwexzhvgauxq2apgc4j0ewvcak6hh6lsnzmvtkem

Ethereum (ETH)
0xb55513D2c91A6e3c497621644ec99e206CDaf239

Litecoin (LTC)
ltc1qt6g2trfv9tjs4qj68sqc4uf0ukvc9jpnsyt59u

USDT (ERC20)
0xb55513D2c91A6e3c497621644ec99e206CDaf239

USDT (TRC20)
TYdPNrz7v1P9riWBWZ317oBgJueheGjATm

0 1 2 3 4 5

| Print version |

	Information
		Users of GUESTS are not allowed to comment this publication.