A curated bibliography of cryptanalytic neural-network extraction, structural and parameter recovery, reproducibility code, limitations, and defenses.
中文版 · September progress · Search audit · BibTeX · Machine-readable catalog · Stage comparisons · Original code collection
Updated 2026-09-30 · 52 unique works · 13 categories. Includes preprints, selected adjacent theory, two surveys, and one explicitly labeled course report. This is a broad, source-checked collection, not a claim of exhaustive coverage of all model stealing.
Source status: ePrint 2026/1025 was withdrawn on September 19. Its historical method is retained and labeled. The comparison covers all 52 records and separates stage costs, oracle assumptions and experimental scope.
-
September 30 update: added three preprints on limited architecture knowledge, end-to-end hard-label extraction, and unknown-architecture CNNs. Existing arXiv:2609.14379 is retained once; ePrint 2026/2161 is an alias, not another work. Source audit and limitations.
-
September 18 update: added Normal Alignment for hard-label sign recovery and refreshed ePrint 2026/848, now titled Cryptanalytic Extraction of Neural Networks for Privacy-Preserving Machine Learning. This is one new work plus one major revision. Results, code status and limits.
-
Unknown architecture: guess-and-determine extraction and limited-knowledge recovery address ReLU fully connected networks under different priors. CNN spatial geometry extends architecture recovery to studied convolution/pooling networks, inheriting the parameter attack’s assumptions. These results do not establish arbitrary mixed-network or Transformer recovery.
-
Numerical feasibility: Finite-Precision Error Analysis studies numerical error in cryptanalytic primitives. Output Rounding evaluates an adapted attack on rounded outputs. Neither supports a universal precision threshold for all networks.
-
Beyond ReLU MLPs: the catalog now covers softmax attention, multi-head query learning, isolated GLU blocks, RNNs, GNNs, CNN pooling and smooth activations. Block-level recovery is not full-LLM extraction.
-
Hard-label progress and limits: algebraic signatures include max-pooling CNN experiments, while cross-layer extraction and polynomiality analysis expose important persistent/dead-neuron caveats. Signature recovery, sign recovery and end-to-end executable recovery remain different outcomes.
The cryptanalytic analogy treats weights as hidden parameters and inputs as chosen queries. Differential, geometric or algebraic leakage can expose more than ordinary test-set imitation. It is an analogy, not an assertion that a network is a secure cipher.
- Start with foundations, then CRYPTO 2020, EUROCRYPT 2024 and the raw-output improvements.
- Read hard-label extraction together with its partial-layer and polynomiality limitations.
- Compare CNN/pooling, activation, RNN/GNN and attention results under their exact oracle assumptions.
- Read finite precision and defenses before treating an idealized recovery theorem as a deployable attack.
- Use related theory, partial LLM extraction and side-channel work as explicitly separated context, not interchangeable threat models.
| Axis | Distinctions to retain |
|---|---|
| Oracle | Raw real-valued outputs; probabilities/top-k scores; top-1 label only; explanations; physical leakage |
| Recovery | Architecture; signature up to scale/sign; oriented parameters; canonical equivalent function; partial block; whole executable model |
| Assumptions | Known structure; generic position; identifiable neurons; chosen continuous inputs; finite precision; access to intermediate blocks |
| Evidence | Theorem under stated assumptions; query count; wall-clock time; sampled fidelity; parameter error; certified equivalence |
¹ Year is the archive/report year; venue year is shown separately. Revised titles and cross-listed ePrint/arXiv versions count once. Linked PDFs are archive copies, not necessarily publisher-final layouts. Versions, page counts and SHA-256 hashes of the reviewed PDFs are in the JSON catalog.
² Code means an author/paper-linked repository unless labeled otherwise. 404 means unavailable at checking time, not proof of deletion; anonymous 410 means the service reports expiration. Supplementary code, author forks and reading resources are distinguished from complete attack implementations. Not located means no usable author-linked URL was found, not that no code exists. Accessibility checks are not execution or reproduction of results.
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2019 | Reverse-Engineering Deep ReLU Networks · PDF | ICML 2020 | Real-valued queries Deep ReLU |
Boundary geometry recovers structure and parameters up to network symmetries under assumptions. | Not located |
| 2019 | High Accuracy and High Fidelity Extraction of Neural Networks · PDF | USENIX Security 2020 | Raw outputs / prediction queries Shallow and deep NN |
Separates accuracy from fidelity; functional-equivalent shallow recovery and hybrid attacks. | Not located |
| 2016 | Stealing Machine Learning Models via Prediction APIs · PDF | USENIX Security 2016 | Prediction scores / labels Classical ML / shallow NN |
Equation-solving and early API extraction; not general deep exact recovery. | Official |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Navigating the Deep: End-to-End Extraction on Deep Neural Networks · PDF | EUROCRYPT 2026 | Raw outputs Deep ReLU MLP |
End-to-end extraction with techniques for deeper-layer recovery. | Official |
| 2026 | Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Limited Architecture Knowledge Setting · PDF | Preprint | Raw logits; limited architecture knowledge ReLU MLP; approximate total neuron count assumed |
Detects omitted neurons and backtracks parameter recovery; rough size knowledge remains necessary. Overall runtime/query costs are not reported. | Paper-linked placeholder repository |
| 2026 | Geometric Critical Point Screening: Clustering-Free Cryptanalytic Extraction of Neural Network Models · PDF | Withdrawn preprint; 2026-09-19 | Raw outputs ReLU networks |
Historical geometric screening of early-layer signatures; withdrawn September 19, 2026. Retained for the historical record, not a current unretracted result. | Official |
| 2026 | Cryptanalytic Extraction of Neural Networks Without Known Architecture Assumption · PDF | Preprint | Raw outputs; architecture unknown ReLU fully connected networks |
Guess-and-determine jointly recovers architecture and parameters; not arbitrary CNN/Transformer recovery. | Not located |
| 2024 | Beyond Slow Signs in High-fidelity Model Extraction · PDF | NeurIPS 2024 | Raw outputs ReLU MLP |
Practical sign-recovery improvements; distinguish runtime from query count. | Official |
| 2023 | Polynomial Time Cryptanalytic Extraction of Neural Network Models · PDF | EUROCRYPT 2024 | Raw outputs ReLU MLP |
Polynomial-time sign recovery improves the original cryptanalytic pipeline. | Official |
| 2020 | Cryptanalytic Extraction of Neural Network Models · PDF | CRYPTO 2020 | Raw outputs ReLU MLP |
Differential critical-point extraction; signatures, signs and final parameter recovery. | Official |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Algebraic Cryptanalytic Extraction on Hard-Label Neural Networks · PDF | Preprint | Top-1 label only FCNN / max-pooling CNN |
NRC/ASV accelerates offline clustering with conditional average and worst-case bounds; experiments recover signatures including CNN kernels, not all signs/biases/output layers. | Announced; placeholder URL |
| 2026 | End-to-End Hard-Label Cryptanalytic Model Extraction Using Efficient Sign Recovery · PDF | Preprint | Top-1 label only; known architecture Trained ReLU MLP; width 16, four/six hidden layers |
Cosine/projection sign recovery reuses intersection spaces; no dedicated queries applies to that stage. End-to-end sampled label agreement excludes dead/almost-dead neurons and does not certify exact equivalence. | Not located |
| 2026 | Normal Alignment: Improved Cryptanalytic Sign Recovery on Hard-Label Networks · PDF | Preprint | Top-1 label only; sign-recovery stage Deep ReLU MLP |
Normal-signature alignment plus eSOE recovers 512/512 and 320/320 signs in the reported models; sign recovery, not a demonstrated complete end-to-end attack. | Announced; placeholder URL |
| 2025 | Is the Hard-Label Cryptanalytic Model Extraction Really Polynomial? · PDF | CRYPTO 2026 | Top-1 label only ReLU MLP |
Persistent/dead neurons challenge polynomiality claims; cross-layer extraction addresses limitations. | Paper-linked; 404 |
| 2025 | Extracting Some Layers of Deep Neural Networks in the Hard-Label Setting · PDF | LATINCRYPT 2025 | Top-1 label only ReLU MLP |
Partial/output-layer recovery under structural conditions, not unrestricted end-to-end extraction. | Official |
| 2024 | Polynomial Time Cryptanalytic Extraction of Deep Neural Networks in the Hard-Label Setting (Extended Version) · PDF | EUROCRYPT 2025; extended version | Top-1 label only Deep ReLU MLP |
Dual-point signatures/signs; extended version includes a small random-model black-box end-to-end demonstration, distinct from white-box-assisted CIFAR experiments. | Official |
| 2024 | Hard-Label Cryptanalytic Extraction of Neural Network Models · PDF | ASIACRYPT 2024 | Top-1 label only ReLU MLP |
Label-only cryptanalytic extraction from decision-boundary geometry. | Official |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | End-to-End Polynomial-Time Cryptanalytic Extraction of Convolutional Neural Networks in the Hard-Label Setting · PDF | Preprint | Top-1 label only Average-pooling CNN; known architecture |
End-to-end pipeline; retained-candidate and structural assumptions remain material. | Anonymous attachment |
| 2026 | Model Extraction of Convolutional Neural Networks with Max-Pooling · PDF | ToSC 2026 | Raw outputs Max-pooling CNN |
Pooling-aware extraction and receptive-field structure. | Official |
| 2026 | Algebraic Attack on Convolutional Neural Networks with Max Pooling · PDF | CRYPTO 2026 | Raw outputs Max-pooling CNN |
Algebraic extraction handles pooling switches; raw-output results do not automatically transfer to label-only access. | Paper-linked; 404 |
| 2026 | Cryptanalytic Extraction of Convolutional Neural Networks · PDF | ACISP 2026 | Top-1 label only Average-pooling CNN |
Uses convolutional structure for kernel recovery. | Expired; 410 |
| 2026 | Extracting CNNs in the Unknown-Architecture and Feedback-Agnostic Setting · PDF | Preprint | Raw outputs / top-1 labels; architecture unknown Max-/average-pooling CNN |
Spatial geometry of recovered vectors reveals convolution/pooling structure; inherits upstream parameter-recovery assumptions. Two-convolution end-to-end and three-convolution layer-wise results do not establish arbitrary CNN hard-label recovery. | Not located |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Cryptanalytic Extraction of Deep Neural Networks with Non-Linear Activations · PDF | CRYPTO 2026 | Raw outputs Smooth/non-linear activations |
Higher-order/near-linear geometry enables recovery for studied non-linear activations; not every smooth function. | Official |
| 2026 | Cryptanalytic Extraction of Neural Networks with Various Activation Functions · PDF | ToSC 2026 | Raw outputs / hard labels (variant-dependent) PReLU / LeakyReLU / HardTanh / Step |
Extends extraction to several activation families; oracle assumptions differ by variant. | Official |
| 2026 | Breaking Slope and Structure Restrictions: Broadening Hard-Label Cryptanalytic Extraction of PReLU Neural Networks · PDF | Preprint | Top-1 label only PReLU |
Broadens allowable slopes and architectures for hard-label extraction. | Not located |
| 2025 | Delving into Cryptanalytic Extraction of PReLU Neural Networks · PDF | ASIACRYPT 2025 | Raw outputs / top-m probabilities PReLU |
Recovers PReLU parameters under stated conditions; not a label-only result. | Official |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Polynomial-Time Cryptanalytic Extraction of Graph Neural Networks in the Hard-Label Setting · PDF | Preprint | Top-1 label only Message-passing GNN |
Graph/message-passing structure supports extraction under the stated model. | Official |
| 2026 | Cryptanalytic Extraction of Recurrent Neural Network Models · PDF | Preprint | Raw outputs / top-1 labels RNN |
Exploits recurrence; long unrollings share weights and are not independent deep layers. | Not located |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Cryptanalytic Extraction of Multi-Head Softmax Attention Models · PDF | Preprint | Raw outputs / chosen continuous inputs Multi-head softmax attention |
Recovers a canonical equivalent representation; Q/K/V factors have gauge ambiguity. | Not located |
| 2026 | Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation · PDF | Preprint | Direct queries to an isolated block Bias-free GLU FFN |
Antipodal separation for isolated blocks; not extraction of a complete LLM through its token API. | Not located |
| 2026 | Provably Learning Multi-Head Attention with Queries · PDF | Preprint | Chosen real-valued queries Multi-head attention / restricted one-layer Transformer |
Canonical head recovery; additional assumptions for the Transformer extension. | Not located |
| 2026 | Provably Learning Attention with Queries · PDF | ICML 2026 | Chosen real-valued queries Attention |
Query-learning guarantees under attention-model assumptions. | Not located |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2024 | Logits of API-Protected LLMs Leak Proprietary Information · PDF | COLM 2024 | Logprobs / restricted API LLM output subspace |
Softmax bottleneck reveals hidden dimension and output-space information, not full-network recovery. | Not located |
| 2024 | Stealing Part of a Production Language Model · PDF | ICML 2024 | Restricted logprobs / logit-bias API LLM output projection |
Partial projection recovery up to symmetries, not all model weights; released code is supplementary. | Official supplementary |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Cryptanalytic Extraction of Neural Networks for Privacy-Preserving Machine Learning · PDF | Preprint | Finite-ring/fixed-point inference; variants include top-1 + probability PPML neural inference |
Revised Sep 17: multi-point projection aggregation for expansive networks and optimal sign probes; reports 30%-74% fewer sign-recovery queries than Neuron Wiggle on its benchmark, not a universal reduction or encryption break. | Anonymous attachment |
| 2025 | Activation Functions Considered Harmful: Recovering Neural Network Weights through Controlled Channels · PDF | Preprint | SGX controlled channels DNN activation implementation |
Activation-access leakage supports weight recovery; first-layer and deeper-layer outcomes differ. | Official |
| 2024 | A Divide-and-Conquer Strategy for Hard-Label Extraction of Deep Neural Networks via Side-Channel Attacks · PDF | TCHES 2026; revised title | Top-1 labels + side channel Deep NN / non-FC components |
Divide-and-conquer with physical leakage; stronger than black-box label access. | Official |
| 2020 | SNIFF: Reverse Engineering of Neural Networks with Fault Attacks · PDF | Preprint / IEEE Transactions on Reliability | Fault injection + outputs Neural networks |
Sign-bit fault attacks use a stronger attacker than ordinary API queries. | Not located |
| 2018 | CSI Neural Network: Using Side-channels to Recover Your Artificial Neural Network Information · PDF | USENIX Security 2019; published title differs | Power / EM side channels Embedded neural networks |
Architecture/parameter leakage via physical observations; selected historical context. | Not located |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Finite-Precision Error Analysis of Cryptanalytic Model Extraction · PDF | Preprint; ASIACRYPT 2026 acceptance author-listed | Finite-precision raw outputs Cryptanalytic signature recovery |
Quantifies numerical error in extraction primitives; neither universal impossibility nor a proven generic defense. | Paper-linked; 404 |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2026 | Output Rounding Is Not a Free Defense Against Cryptanalytic Neural Network Extraction · PDF | MIT 6.5610 Spring 2026 course report | Rounded raw outputs Small ReLU MLP |
Step-spacing adapts to rounding; empirical small-model evidence, not a universal security threshold. | Supplement mentioned; URL not located |
| 2025 | Train to Defend: First Defense Against Cryptanalytic Neural Network Parameter Extraction Attacks · PDF | NeurIPS 2025 | Defense against parameter extraction ReLU MLP |
Training-time neuron-similarity regularization; evaluate against adapted attacks. | Official |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2025 | Data Augmentation Techniques to Reverse-Engineer Neural Network Weights from Input-Output Queries · PDF | UniReps 2025 workshop | Input-output queries Teacher-student parameter recovery |
Data augmentation improves Expand-and-Cluster; linked code is the authors' extension fork. | Author extension fork |
| 2024 | Model Stealing for Any Low-Rank Language Model · PDF | Preprint | Conditional queries Low-rank sequence distributions / HMM |
Learns low-rank output distributions, not arbitrary Transformer weights. | Not located |
| 2024 | Provably learning a multi-head attention layer · PDF | STOC 2025 | Random examples (not chosen queries) Multi-head attention |
Learning theory under nondegeneracy; not a practical full-model API extraction demonstration. | Not located |
| 2023 | Reverse Engineering Deep ReLU Networks An Optimization-based Algorithm · PDF | Preprint | Input-output queries Deep ReLU |
Optimization-based reverse engineering; distinguish proposed guarantees from demonstrated scalability. | Not located |
| 2023 | Expand-and-Cluster: Parameter Recovery of Neural Networks · PDF | ICML 2024 | Input-output samples Neural networks |
Overparameterized students plus clustering recover parameters; related to, but not the same as, differential extraction. | Official |
| 2022 | Finite Sample Identification of Wide Shallow Neural Networks with Biases · PDF | Preprint | Finite input-output samples / queries Wide shallow networks with biases |
Identification of directions and biases under model and sampling conditions. | Not located |
| 2021 | An Exact Poly-Time Membership-Queries Algorithm for Extraction a three-Layer ReLU Network · PDF | ICLR 2023 | Membership / real-valued queries Three-layer ReLU |
Exact polynomial-query recovery under depth/generic-position assumptions. | Not located |
| 2018 | Model Reconstruction from Model Explanations · PDF | FAT* 2019 | Gradient/explanation oracle Neural networks |
Reconstruction with explanations uses a stronger oracle than ordinary predictions. | Not located |
| Year¹ | Paper and PDF | Venue / status | Oracle / architecture | Recovery target and limits | Code² |
|---|---|---|---|---|---|
| 2025 | A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives · PDF | Preprint survey | Multiple Multiple |
Broad model-extraction survey; includes surrogate stealing outside this repository's core scope. | Reading list, not attack code |
| 2025 | A Survey on Model Extraction Attacks and Defenses for Large Language Models · PDF | Preprint survey | Multiple LLM APIs LLM |
Contextual survey of LLM extraction and defenses; not an exact-recovery attack. | Reading list, not attack code |
- Unknown general architectures: studied ReLU fully connected and convolution/pooling models now have architecture-recovery results. Mixed operators, residual paths, weak size priors and composition still require separate evidence.
- Hard-label max-pooling: no longer an empty category. Robust full-layer/sign/bias recovery, winner switches and event observability should be evaluated beyond small signature-recovery demonstrations.
- Practical precision and cost: relate numerical stability to query budgets, conditioning, rate limits, abstentions and probability truncation; do not equate real-arithmetic polynomiality with cheap extraction.
- Identifiability: report dead/persistent neurons, equivalent parameterizations and canonicalization explicitly. A failure to recover one parameterization is not automatically security.
- Adaptive defenses: evaluate training regularization and output modifications against adapted attacks, with utility loss and attack budget fixed. Small-model experiments do not establish universal security.
- Composition: recovering attention, output projections or isolated GLU blocks does not establish recovery of their composition through an ordinary token API.
Edit data/papers.json, including official title/authors, source, venue/status, oracle, scope limits, code provenance and PDF version fingerprint. Edit the English template or Chinese template for prose, then run:
python scripts/render_catalog.py
python scripts/render_catalog.py --check
python scripts/render_comparisons.py
python scripts/validate_comparisons.py --strictKeep preprints and course reports visibly labeled. Prefer primary sources and author-linked code; do not vendor third-party implementations or copyrighted PDFs into this repository. A local ref corpus can use the filenames, official PDF URLs and hashes in the catalog. No code release is implied by a paper's promise to publish it.
For academic research and defensive analysis on authorized models and systems. Listing a paper does not independently validate its claims or endorse an attack against a third-party service.