deep-learning-theory

A curated list of papers of interesting empirical study and insight on deep learning. Continually updating...

codebase for "A Theory of the Inductive Bias and Generalization of Kernel Regression and Wide Neural Networks"

Code for "Information-Theoretic Local Minima Characterization and Regularization"

PyTorch implementation for "Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers", NeurIPS 2024

From-scratch NumPy implementations of core deep learning architectures (DNN, CNN, RNN). A journey into the first principles of AI.

Code to reproduce the paper "Deconstructing the Goldilocks Zone of Neural Network Initialization"

Resolving the Black Box Interpretability Problem by Proving the Mathematical Necessity of J-space via Hessian Rank Deficiency in Lean 4.

Proof 4: What neural networks cannot see — the Penrose aperiodicity barrier. 3/3 theorems empirically verified.

Technical portfolio for Stanford CS229 (Summer 2020). First-principles approach to ML: Math-heavy derivations, NumPy-from-scratch implementations, and research project.

Proof 7: Why the Cantor function (Devil's Staircase) is structurally hard for ReLU networks. All theorems verified.

LIFE 框架：基于动态同构的自指耗散智能模型 LIFE Framework: Life Is Full Effective — A Dynamic Isomorphic Dissipative Intelligence

Labs from the Master MVA "Fondements théoriques du Deep Learning" class

Proof 5: Why neural networks prefer low frequencies — a ReLU frequency tradeoff theorem. 3/3 theorems empirically verified.

Framework-free Rust replication of arXiv 2510.26745v2: Node2Vec spectral dynamics, TinyNN geometry vs. memorization, and the initializer that decides between them - with measured findings

Representation Group Flow — MSc Mathematics thesis (NISER, 2023): an effective statistical-field-theory of a deep MLP at initialization; how representations flow with depth.