Built GPT-2, Llama 3, and DeepSeek from scratch in PyTorch - open source code + book [p]

A senior engineer spent the past year implementing five LLM architectures from scratch in PyTorch, including GPT-2, Llama 3, and DeepSeek. The project resulted in open-source code and a detailed book documenting the process, explaining advanced concepts like KV cache, MoE, and FP8 quantization.