A clean PyTorch implementation of the original Transformer model + A German -> English translation example
-
Updated
Jan 24, 2022 - Python
#
transformer-from-scratch
Here are 22 public repositories matching this topic...
Repository for a transformer I coded from scratch and trained on the tiny-shakespeare dataset.
deep-learning text-generation pytorch neural-networks transformer-encoder transformer-architecture transformer-from-scratch
-
Updated
Jul 3, 2023 - Python
Transformer-based Beat Saber track generator
-
Updated
Feb 17, 2026 - Python
elaborate transformer implementation + detailed explanation
-
Updated
Dec 9, 2025 - Python
A Comprehensive Implementation of Transformers Architecture from Scratch
-
Updated
Aug 23, 2023 - Python
A structured, hands-on journey to mastering Transformers — from attention and mathematical foundations to building, training, and understanding modern Transformer architectures with PyTorch.
nlp deep-learning transformers pytorch attention gpt bert self-attention multi-head-attention transformer-architecture llam transformer-from-scratch llm generative-ai large-language-model
-
Updated
Sep 28, 2026 - Jupyter Notebook
Implement the "Attention Is All You Need" paper from scratch using PyTorch, focusing on building a sequence-to-sequence transformer architecture for translating text from English to Italian
translation pytorch sequence-to-sequence encoder-decoder attention-is-all-you-need transformer-from-scratch english-italian
-
Updated
May 21, 2025 - Jupyter Notebook
A complete implementation of the "Attention Is All You Need" Transformer model from scratch using PyTorch. This project focuses on building and training a Transformer for neural machine translation (English-to-Italian) on the OpusBooks dataset.
machine-learning university-project pytorch transformer course-project seq2seq attention neural-machine-translation attention-mechanism encoder-decoder attention-is-all-you-need huggingface positional-encoding transformer-from-scratch cross-attention masked-attention
-
Updated
Nov 8, 2025 - Python
A showcase repository documenting the complete journey of building a Transformer and Neural Machine Translation framework in pure C—from tensor operations to automatic differentiation, training infrastructure, and end-to-end machine translation.
machine-learning deep-learning machine-translation transformers autograd neural-networks pure-c from-scratch software-architecture transformer-from-scratch training-framework
-
Updated
Jul 21, 2026 - Mermaid
Modular Python implementation of encoder-only, decoder-only and encoder-decoder transformer architectures from scratch, as detailed in Attention Is All You Need.
nlp translation nlu torch transformer generation summarization gpt attention-mechanism attention-is-all-you-need transformer-from-scratch llm generative-ai
-
Updated
Aug 24, 2024 - Python
This repository contains my coursework (assignments & semester exams) for the Natural Language Processing course at IIIT Delhi in Winter 2025.
natural-language-processing word2vec fine-tuning aspect-based-sentiment-analysis wordpiece-tokenization transformer-from-scratch claim-normalization
-
Updated
May 7, 2025 - Jupyter Notebook
PyTorch Transformer for neural machine translation (NMT), inspired by Attention Is All You Need. German→English on OPUS Books: training, inference, and attention visualization.
nlp translation deep-learning pytorch transformer tensorboard opus neural-machine-translation sequence-to-sequence attention-mechanism encoder-decoder attention-is-all-you-need transformer-from-scratch german-to-english
-
Updated
May 14, 2026 - Jupyter Notebook
Simple and Easy Implimentation of Transformer Architecture Introduced in the paper "Attention Is All You Need"
nlp jupyter-notebook pytorch attention-mechanism attention-is-all-you-need pytorch-implementation transformer-architecture transformer-from-scratch llm
-
Updated
Sep 13, 2025 - Jupyter Notebook
A Transformer encoder built from first principles, implementing the core architecture from mathematical foundations to working PyTorch code, including tokenization, embeddings, positional encoding, self-attention, multi-head attention, LayerNorm, feed-forward networks, training, and evaluation.
deep-learning encoder transformers pytorch attention-mechanism self-attention multi-head-attention transformer-from-scratch llm-from-scratch
-
Updated
Sep 22, 2026 - Python
From-scratch ~100k-parameter LLaMA decoder (RMSNorm, RoPE, SwiGLU, GQA), plus an equal-parameter ablation of each choice, a train-short/test-long probe of RoPE against learned absolute positions, and a parameter-free sweep of RoPE's rotation base. CPU-only, multi-seed; every README number renders from a committed artifact and CI byte-compares it.
pytorch llama reproducibility rope ablation-study positional-encoding transformer-from-scratch rmsnorm grouped-query-attention swiglu cpu-only
-
Updated
Sep 26, 2026 - Python
_build a Transformer model from scratch using Pytorch
-
Updated
Oct 27, 2025 - Python
This project aims to build a Transformer from scratch and create a basic translation system from Arabic to English.
-
Updated
Jul 26, 2024 - Jupyter Notebook
Implementation of Transformer:"Attention Is All You Need" in Pytorch
-
Updated
Sep 4, 2024 - Python
An educational implementation of core Transformer architecture concepts built from scratch using Python. This project explores how modern NLP transformer models work internally by implementing attention mechanisms, embeddings, positional encoding, and next-word prediction logic step-by-step.
python nlp transformers embeddings tokens attention-mechanism nlp-machine-learning zero-to-hero transformer-from-scratch
-
Updated
Jul 9, 2026 - Jupyter Notebook
Collection of implementations from scratch (mostly ML)
python machine-learning ai machine-learning-algorithms ml pytorch transformer attention from-scratch machine-learning-from-scratch ml-from-scratch pytorch-implementation implementation-from-scratch ml-papers transformer-from-scratch from-scratch-in-python
-
Updated
Jan 22, 2024 - Jupyter Notebook
Add this topic to your repo
To associate your repository with the transformer-from-scratch topic, visit your repo's landing page and select "manage topics."