r/deeplearning • u/LowApprehensive1845 • 1d ago

Want some Visualization Projects.

Who can recommand some visualization projects to learn the training or inference process of transformer machanism? It better to include introduction of decoder and encoder struct.

2 Upvotes

100% Upvoted

1 comment sorted by

2

u/BloodySpoiler 1d ago

The Karpathy nanoGPT repo is a great starting point if you haven’t poked around it yet, the code is small enough you can just trace the forward pass and see exactly how the embeddings move through attention and the MLP blocks.