Skip to content

Latest commit

 

History

History
44 lines (34 loc) · 3.1 KB

File metadata and controls

44 lines (34 loc) · 3.1 KB

Architecture

python-source-leak-filter is a one-module, zero-dependency text-filtering library. No I/O, no global state, no framework coupling. You hand it a string; it hands back a redacted string and a boolean.

Module map

Module Responsibility
python_source_leak_filter/filter.py The whole filter: strip_python_source() (the public entry point), the three compiled regexes (_FENCED_PYTHON, _FENCED_GENERIC, _RAW_DEF/_RAW_CLASS), the _looks_like_python() heuristic, and _strip_unfenced() (the indentation-aware def/class block remover).
python_source_leak_filter/__init__.py Re-exports strip_python_source and __version__.

Data flow

model output ──► strip_python_source(text)
                       │
                       ├─ Pass 1: _FENCED_PYTHON.sub(...)        # ```python / ```py fences
                       ├─ Pass 2: _FENCED_GENERIC.sub(...)       # bare ``` fences, gated by _looks_like_python()
                       └─ Pass 3: _strip_unfenced(...)           # unfenced def/class + indented body
                       │
                       └──► (filtered_text, was_stripped)
                                                   │
                                                   └► caller audit-logs if True, then sends filtered_text

Each pass is independent and order-stable: passes 1 and 3 compare before/after text to set was_stripped; pass 2 sets it from inside the substitution callback when a generic fence body trips the heuristic.

Seams (injection points)

Seam How to use it
Syntax signals _looks_like_python() decides whether a generic (untagged) fence is Python. The signals tuple and the >= 2 threshold are the tuning knobs: raise the threshold to be more permissive, add signals to catch more dialects.
Smuggling vectors The base64-exec check inside _looks_like_python() is where you add new encoded-payload patterns (e.g. another decode/exec pairing).
Replacement text _LEAK_REPLACEMENT is the single string substituted for every stripped block. Swap it for your product's wording.
Indent handling _RAW_DEF / _RAW_CLASS match a declaration at any leading indent; _strip_unfenced() then removes every following line more indented than the declaration. This is what catches nested class methods.
Runtime dependencies None. Standard-library re only.
Persistence / logging Caller-owned — the function is pure; the was_stripped flag is your audit hook.

Design invariants

  • Fails closed on model behavior. The filter is deterministic and runs after generation, so it does not depend on the model "choosing" to comply.
  • Selective by construction. Prose, SQL, and JSON must pass through untouched; only Python-shaped content is removed. That selectivity is the whole value and lives in the regexes + heuristic.
  • Never raises. Empty and whitespace-only inputs short-circuit to (text, False).
  • Domain-neutral. Nothing in the module names an industry, product, or data model.