• views: 178
• date: 25 July 2023
• posted by: voska89

How transformers grew to dominate neural network architectures
What you'll learn
How attention works and how to make it more efficient
How to build a transformer block from scratch, with multi-headed attention
How transformers grew from their origins in NLP to take over multiple modalities
How LLMs like ChatGPT work
Requirements
A basic understanding of neural networks
Description
Transformers are the dominant neural architecture in modern AI.  This course covers everything there is to know about transformers, including what inspired their birth, how they have evolved and become more efficient, and how they grew from their origins in natural language processing into the most ubiquitous neural architecture.  The course primarily covers theory and general concepts.  There are a few coding exercises and equations scattered throughout, but if you are looking for a deep technical dive, you should probably look elsewhere.  The most technical parts of this course cover building multi-headed attention and transformer blocks from scratch, using Pytorch.   Some of the topics covered in this course include:Efficient transformersLongformerReformerVision transformer (ViT)wave2vec2stable diffusionthe perceiver familydecision transformerChatGPT and how to combine generative pre-trained (GPT) models with reinforcement learning with human feedback (RLHF)Tokenizers and positional encoding strategiesFine tuning a language model, like BERT, using Pytorch and HuggingFaceHow to think about attention from a sequence perspective and a graph perspective, and how these perspectives can yield insights into improving transformersThe course spends more time on text than other data types.  This is because of the recent hype around LLMs.  The course aims to demystify LLMs like ChatGPT, and help you learn how to explain them to non-technical audiences.
Who this course is for
Anyone in the field of data science who wants to learn about transformer based models
Homepage

https://www.udemy.com/course/the-evolution-of-transformers/

   
   
   

Forex Trading

Crypto Trading. 20 USD Bonus after KYC

We need your support!
Make a donation to help us stay online
        
Bitcoin (BTC)
bc1q08g9d22cxkawsjlf8etuek2pc9n2a3hs4cdrld
	
Bitcoin Cash (BCH)
qqvwexzhvgauxq2apgc4j0ewvcak6hh6lsnzmvtkem

Ethereum (ETH)
0xb55513D2c91A6e3c497621644ec99e206CDaf239

Litecoin (LTC)
ltc1qt6g2trfv9tjs4qj68sqc4uf0ukvc9jpnsyt59u

USDT (ERC20)
0xb55513D2c91A6e3c497621644ec99e206CDaf239

USDT (TRC20)
TYdPNrz7v1P9riWBWZ317oBgJueheGjATm

0 1 2 3 4 5

| Print version |

	Information
		Users of GUESTS are not allowed to comment this publication.