A searchable index over a pile of documents, and an append-only fact chronology that cuts every quote out of the page it cites — and refuses one it cannot find there.
You have more documents than you can read, and every claim you make has to trace back to an exact page. This tool indexes them locally, hands you the most likely relevant material first in batches you can read, and records each fact you take out of them as a row with the document, the page, the line where there is one, and the words themselves — sliced out of the stored page text by the tool, never retyped and never written by a model. It then refuses to export anything you have not personally checked against the original.
It is built for one self-represented person working through a discovery corpus with Claude Code on an ordinary computer, and it fits anyone reviewing a large pile of documents where every claim must trace to a page.
- This is not legal advice, and it is not a substitute for the advice of an attorney. It is software that organizes documents. It does not tell you what any of it means, whether to file anything, or how you are doing.
- It never files and never sends. It has no e-filing credentials, no mail client, no upload path, and no network calls at all: the installed package imports no network library, and you can confirm that by reading its imports.
- It never generates or checks legal authority. No case names, no citations, no propositions of law. Every citation it produces points into your own documents, which are the sources you hold and can open.
- There is no model inside it. The tool is deterministic Python and SQLite. A model reads what you put in front of it, in your own Claude Code session; the tool itself never calls one.
- There is no server, no Docker, no index service, no account. One SQLite file and your corpus directory, both ignored by git from the first commit.
- It is not a production tool. It stamps no Bates numbers, writes no load files, and performs no redaction. Those surfaces are the consequential ones to get wrong, and this build stays out of them.
- Nothing you put into it becomes privileged by virtue of the tool.
An index you can trust; a ledger that refuses a quote it cannot find on the page it cites; a review loop that surfaces the most likely relevant material first and tells you what fraction of the corpus you have seen, what it could not read, and where it stopped; and a discipline that keeps you between the machine and anything that leaves.
What the quote check does not prove, said here and again at the foot of every export: it proves that those characters are in the extracted text layer of the page you cited. It does not prove that the text layer matches the page image, it does not prove who spoke, and it does not prove that the passage survives its own surroundings. Open the document and read it.
And it is measured, not asserted. The typed-quote path was run against 1,823 typed variants of spans taken out of real pages: 26 were refused, 1.4%, and every one of them was a word a typesetter had broken across a line — the case the sliced path does not have. Read that 1.4% carefully, because it is an average over cases that mostly cannot fail: measured over the whole population of that one failing cause — every word broken across a line in all four corpora, 1,094 of them — 861 were refused when typed whole, 78.7%, and none was refused through --span. If you type a quote by hand and it crosses a line break, expect a refusal; that is the number to plan around. In the other direction, 150 of 160 fabricated spans were refused, and the ten that were accepted were passages that genuinely do appear on the page they were cited to, because pages of a regulation repeat their running heads. The counts, the causes and the command that reproduces them are in docs/quote-check-measurement.md.
You are the certifying party. Signing a paper is generally a representation that you made a reasonable inquiry and that your factual contentions have evidentiary support — read your own court's rules for what exactly you are certifying and when. Unverified machine output is judged the way any other unchecked work is judged, and courts apply that standard to self-represented filers and to represented ones alike. This tool exists to make checking cheap and to keep a record of what was searched — not to discharge a duty that is yours.
On the fictional demo corpus the tool builds for itself. Every dd command below ran, in order and exactly as printed, from an empty directory on the build machine on 2026-08-16 — the four install lines above them are the documented install rather than part of that run. A fuller session over the same corpus, with the output and the exit code of every step, is recorded in docs/demo-transcript.md.
git clone https://github.com/MiningTheSkies/discovery-desk
cd discovery-desk
python3 -m venv .venv && . .venv/bin/activate # Windows: py -m venv .venv then .venv\Scripts\activate
pip install -e .
dd doctor # what is installed, what is missing, what that costs you
dd demo # build a fictional corpus and index it
dd reduce # mark duplicate copies before you read: they otherwise fill the queue
dd next-batch --seed "the delivery was cancelled" # the queue, ranked by your phrase, chunked with citable ids
dd add-statement --chunk 4:3:63-295 --fact "Alder testified she wrote to the yard office on 24 February to say the delivery was cancelled." --speaker "Dana Alder" --speaker-basis transcript_qa_role --said-on 2026-03-04 --modality sworn-testimony --topic "delivery cancellation" --run 1
dd add-statement --doc 9 --page 1 --span 172:199 --fact "Eight days earlier Alder wrote to the yard office that nothing had been cancelled." --speaker "Dana Alder" --speaker-basis email_header --said-on 2026-02-24 --modality email --topic "delivery cancellation" --run 1
dd code 4 relevant --note "the sworn account of the cancellation" # relevance is your judgement, and the stopping counter reads it
dd close-run 1 --skipped "14:a produced slipsheet; the native file is refused by name" # every document in the pass accounted for
dd confirm 1 --note "read the transcript page" # a human act: until you do this, the chronology stays empty
dd confirm 2 --note "opened the email" # confirming supersedes: rows 1 and 2 become rows 3 and 4
dd candidates # pairs worth reading — a filter, never a finding
dd add-lead --kind prior-inconsistent-statement --claim "The account of when the yard office was told moves between the two documents." --statements 4:earlier,3:later
dd verify-lead 1 --verdict verified --note "Opened both originals and read the passages." --displayed-at "dd show-lead 1"
mkdir out # exports go here: it is gitignored, and they carry your material
dd export-lead 1 --out out/lead-1.md # a citation pack — and only because you verified it
dd status --export out/manifest.md # the record of the inquiry: what came in, what was refused, what you have seenWhy dd code and dd close-run are in that list. A batch stays open until you close it, and the reading position is counted from closed passes. Walk the loop without them and dd status reports documents seen (read or coded): 0 and extraction runs still open: 1 over a batch you have just read — which is the counter being honest, not a fault. dd code records the relevance judgement the stopping counter reads, and dd close-run makes the pass account for every document in it: read, already coded, or skipped with a reason. After the two lines above, the same store reports 9 of 11 documents seen and no run open.
Why dd confirm is in that list. A row a machine wrote enters as proposed and stays there. dd chronology — the fact chronology this whole tool is built around — prints confirmed facts only, so a reader who walks the loop without confirming anything gets a chronology that says [No confirmed facts yet.] and a Gaps section listing everything. That is the design working, not a fault, but it means confirming is a step and not a formality: dd confirm, dd dispute --by opposition|us|unsure and dd retire are the three human acts that move a row off proposed, and each writes a new row rather than editing the old one.
dd --help lists every command; dd <command> --help explains one. The fifteen-minute walkthrough, with the real output of each step, is in docs/quickstart.md. The design and the reasoning behind it are in docs/design.md.
Six layers. For each one: what the code does, what a model does, and what only you can do.
1. Classify. Before anything is parsed, dd classify decides what shape a delivery is — loose files, an image-plus-load-file production, a mailbox export, a mixture — and prints the evidence and the counts that produced the decision. Guessing wrong here is expensive in both directions: running OCR over a production that already carries the producing party's own text substitutes worse text for better underneath everything you later cite, and treating a page image as a document turns four thousand documents into forty thousand.
Code: all of it. Model: none. You: read the classification and confirm the delivery is what you thought it was.
2. Index. dd ingest walks the tree, records every file by hash and path, extracts text page by page, runs OCR only where a page has no text layer and only where the interlock allows it, and builds a full-text index. Nothing is skipped silently: a file this build cannot read becomes a row marked unsupported with the reason, and a page awaiting OCR is counted as awaiting OCR — never as a page containing nothing. dd search finds a phrase and gives you the document, the page, and the production number on that page.
Code: all of it. Model: none. You: read the refusals list and decide what to do about it.
3. The ledger. One row per fact: the document, the page, the line range where the document has line numbers, the quote, who said it and on what basis, when it was said, in what setting, and what it goes to. The store is append-only — a correction is a new row pointing at the one it replaces — and machine-written rows can only enter as proposed. Confirming, disputing, retiring, and tying a fact to an element of a claim are human acts, and each has a command that writes the superseding row for you: dd confirm <statement>, dd dispute <statement> --by opposition|us|unsure, dd retire <statement>, and dd tick-issue --statement <id> --issue <id>. The first three take a --note, and the note is the part your future self will want.
Code: the write path, the slicing, the refusals. Model: proposes rows. You: everything that turns a proposal into a record.
4. The review loop. dd next-batch --seed "a phrase" ranks the unread documents against your phrase, opens a numbered run, and prints their text in chunks with ids like 4:3:63-295 — document 4, page 3, characters 63 to 295. You (or a session working under briefs/extract-statements.md) name a chunk; dd add-statement --chunk cuts the words out of the stored page. Batches start small, because the first seed phrase is usually a guess, and double as passes close. dd status says what fraction you have seen and how many documents you have coded not relevant since the last relevant one.
Code: ranking, chunking, counting. Model: reads the batch and proposes statements. You: launch each batch, judge relevance, close the run.
5. Candidates. dd candidates returns pairs of statements by the same person on an overlapping topic, made in different settings or far apart in time. That is a filter you can restate in one sentence, not a conclusion. The word "contradiction" is never something this tool concludes: the one place it writes the word on its own account is the empty result, which says "no candidates under this tagging" and then, in the same breath, that this is "not a finding that the corpus holds no contradictions." It is also one of the five kinds you may give a lead — dd add-lead --kind contradiction — and there it is a label you chose, not a finding it reached. dd timeline, dd changes, dd first-said, dd chronology and dd cast are the same kind of thing — orderings of what you recorded, never reconstructions of what happened.
Code: all of it — plain rules over the ledger. Model: none. You: read the pairs and decide whether anything is there.
6. Verify and export. A lead is a claim about two or more statements, and it starts as proposed. dd show-lead puts the surrounding context in front of you; dd verify-lead records your verdict — verified, rejected, or could_not_check, which is a verdict and not a missing one — along with what you were looking at when you gave it. Only a verified lead exports, and dd export-lead prints a citation pack carrying the pin cite, the quote, the context, the source file and its hash, how the quote was checked, and the disclaimer. dd where-cited DOC:PAGE runs the index backwards: everything that already rests on a page, so that when a source turns out not to say what you thought, you can find every place it reached.
Code: prints the pack, and refuses an unverified export. Model: none — this gate does not delegate. You: the whole stage.
Nobody can tell you how many documents are in your corpus without measuring it, and the tool measures yours rather than quoting a constant. The published figures are worth showing anyway, because they say how wide the uncertainty is.
Every figure in this section has a source. The URL each came from, a verbatim excerpt of the sentence that carries it, the date it was read, and — for the numbers nobody published, like the band endpoints and the hours totals — the arithmetic that produced them, are in docs/corpus-arithmetic.md. Quoted and derived are marked separately there, because they are not the same kind of claim.
How many documents in a gigabyte. There is no standard. The industry body whose project set out to establish the number has published none. Practitioner data by file type runs from about 505 documents per gigabyte for one presentation format to about 10,000 per gigabyte for email; one widely used vendor rule of thumb blends to 7,500; two published cost models differ from each other by 7.5× on the same quantity, at 2,500 and 18,750. The one measured number against a named public research corpus — 836,165 messages and attachments distributed as 73.7 GB — is 11,345 documents per gigabyte, and even that is an upper-ish bound, because the distributed rendering inflates relative to native mail. A published study of one very large regulatory production, 2.3 million documents in 1.3 TB, measures about 1,784 per gigabyte — four times fewer documents per gigabyte than the blended rule of thumb, because 600,000 of those documents were scans, and a scanned page is a fat file.
So, for a 20 GB corpus: roughly 36,000 to 375,000 documents, a 10× spread, decided by composition rather than by size. The two endpoints are the two blended, whole-corpus rates — 1,784 per gigabyte at the low end, measured on that image-heavy production, and 18,750 per gigabyte at the high end, a published cost model's assumption. The type-specific rates above are not endpoints: 505 per gigabyte describes one presentation format and 10,000 describes email, and no real delivery is made of one file type. Assume nothing; run dd status and read your own count.
How many survive deduplication. Exact deduplication is published as removing "as much as 90 percent, and, on average, 30 or 40 percent"; the one measured production above dropped 31%, from 2,319,346 to 1,600,047. Take the blended rule of thumb at 20 GB — 150,000 documents — and 65% survival: about 97,500 to read. Across the whole band that is roughly 23,000 to 244,000. dd reduce computes the two exact methods (byte-identical files, and the standard email field hash) and prints your own rate; it does no near-duplicate grouping and no thread suppression, because those have tunables and every tunable is a decision about what you will never see.
Why the loop is seeded rather than a sweep. At the industry budgeting rate of 50 documents an hour, reading 97,500 documents is 1,950 hours — over a person-year of full-time work. Reading them in ten-document batches is 9,750 separate launches; at twenty seconds apiece just to launch, that alone is 54 hours. A ranked queue changes the shape: published continuous-active-learning results reach 96–97% recall at a review volume that — at the responsiveness rate measured in the study behind the low endpoint — works out to about a fifth of the surviving documents (derived; see docs/corpus-arithmetic.md §7), and in one large study 67–87% recall came from reading 0.5% to 4.1% of an 836,165-document collection. Effort scales with how much of the corpus matters, not with how big it is.
And the ceiling on any of it. The most exhaustive process ever measured — 225 attorneys reading 1.6 million documents at about $8.50 each — scored 49–53% recall against its own production when a sample was re-reviewed. Reading everything did not find everything. This tool does not promise you every relevant statement. It promises an index that finds what is in it, a ledger that refuses a quote that is not on the page, and a count of what you have seen and what could not be read.
Where your text goes. The tool makes no network calls. Text reaches a model when a command prints your material into a session you are running a model in — or when you paste that output into one. That is more than one or two commands, so here is the whole list, split by what kind of text it prints:
| Command | What it prints out of your corpus |
|---|---|
dd next-batch |
page text nobody selected — every page of every document in the batch, in chunks |
dd search |
page text nobody selected — a short window of the page around each hit |
dd show-lead, dd export-lead |
the stored quotes, plus 200 characters of the page either side of each |
dd candidates, dd timeline, dd changes, dd first-said |
the stored quote of each statement, in full |
dd chronology |
the stored quote of each confirmed fact, in full |
dd where-cited |
the opening of each stored quote resting on that page |
dd cast |
names and aliases as recorded — no page text |
dd status, dd status --export, dd classify, dd doctor, dd reduce, dd code, dd close-run |
file names, paths and document titles, custodians, the designation text you typed, seeds, counts — no page text |
Every other command prints counts, ids, states and refusals, and no corpus text at all. That covers the fourteen not named above — dd init, dd demo, dd ingest, dd add-entity, dd add-issue, dd add-statement, dd add-lead, dd confirm, dd dispute, dd retire, dd tick-issue, dd verify-lead, dd designate and dd clear-designation — and the manifest the tool writes says the same sentence about its own build, so the two records agree.
The distinction worth holding on to is the one in the first column: a stored quote is a passage a person already cut out of a page and recorded, and page context is text nobody selected — the chunks in a batch, the window around a search hit, the 200 characters either side of a quote in a lead. Everything else the tool prints is metadata about your files.
Read your provider's terms for what happens to what you send; this tool makes no claim about what any subscription, plan, or protective order permits.
What the record says about it. The tool keeps a log of its own invocations — which command ran, and when — and the manifest reports that log rather than asserting what any model did or did not see. (A store created before that log existed does not have the table; dd init on an existing store adds it, and until then every run says out loud that it was not recorded rather than leaving a silent gap in the record.) It cannot know what you pasted where, and it does not pretend to: what it can say honestly is which commands that print corpus text were run during the inquiry, and it says exactly that.
Designations are yours to set and yours to clear, and they gate one surface. dd designate records — in your words, as you read them off the document — that a document carries a designation. A document carrying one is refused from every batch until you clear it with dd clear-designation, and the designation is carried into any citation pack that rests on it. What it does not do is hold the document out of everything else: dd search, dd show-lead, dd where-cited, dd timeline and the chronology views will still print material from a designated document. A designation holds a document back from the reading loop, not from the session — so if a document must not be shown to a machine at all, the answer is to keep it out of the corpus directory you ingest, not to designate it. The tool reads no stamp off a page and infers nothing from a production's name: a designation it invented would hold back material nobody withheld, and one it missed would put material in front of a machine that you had not decided to show it.
The manifest is your record of the inquiry. dd status --export out/manifest.md writes what came in, how each delivery was classified, which files this build refused and why, how many documents hold text, how many you have seen, where the reading stopped, which designations are outstanding, every reading pass with its seed and its skips, and a plain paragraph naming the tool and its version and reporting which of its commands were run. Its last line says it describes an inquiry and certifies nothing, because that is the truth about it.
The duties are yours. Courts differ on what they require and change it often: some want a disclosure on the first page under a named heading, some at the end or above the signature block, some require a certification even when no such tool was used, and in at least one the absence of a disclosure is itself a certification that none was used. Read your own court's local rules and its judges' standing orders before you file anything. This tool keeps the record; it does not know your court, and this README is not maintained as legal advice.
In: born-digital and scanned PDF · .eml, .mbox, Maildir, .emlx · .msg through the optional extract-msg package · loose images (.png, .jpg, .tif), including screenshots of text threads · image-plus-load-file productions (Opticon .opt, Concordance .dat, per-document text) · deposition and hearing transcripts in the standard 25-lines-to-the-page format, with condensed four-up pages detected · plain PDF dumps and email printed to PDF, whose synthesized identifiers are marked as synthesized everywhere they appear · .pst through the external readpst program when it is on your PATH.
Refused, by name and counted: native chat, Slack and Teams exports · spreadsheets · Lotus Notes / Domino .nsf · proprietary transcript-viewer formats · anything unrecognized. A refusal is printed with the reason and a suggestion, appears in dd status, and appears in the manifest. It is never a silent skip.
Measured rather than estimated, and small measurements: they were taken on the demo corpus and on two throwaway corpora made for the measurement, on the build machine, on 2026-08-16, with a clock read either side of each run. Read them as ceilings and as shapes, not as forecasts for your material. Nothing here uses a GPU; there is no model in the tool to run on one.
Extraction is fast, and it is not what will cost you time. dd demo — building the fictional corpus and indexing all 21 of its pages — took 0.78 seconds. Ingesting 300 copies of its four-page born-digital transcript, 1,200 pages, took 1.30 seconds: about 900 pages a second. Those pages are small and synthetic; a real born-digital page carries several times the text and a graphics-heavy one takes longer still, so treat that rate as an upper bound and measure your own — time your first dd ingest and divide by the page count dd status prints.
The store is roughly three times the extracted text. One thousand generated plain-text pages, carrying 3.05 million characters between them, produced a 9.4 MB store — about 3× the text they hold. The store keeps the page text, a normalized copy of it for matching, and the full-text index (whose detail='full' configuration costs about 45% of the extracted text on the SQLite project's own email benchmark). Extracted text is a small fraction of a corpus's bytes — a scanned page is a large file and a few thousand characters — so for a corpus of scans the store will be far smaller than the corpus; for plain text and email it will be a few times the text.
OCR is the one that takes days, and it was not measured here, because the machine these numbers came from has no OCR engine installed — dd doctor reports tesseract blind and the pages stay counted as pending rather than pretending to be read. What can be said without measuring: OCR runs single-threaded, one page at a time, with a per-page ceiling of 300 seconds (dd ingest --ocr-timeout, which is a ceiling, not an expectation). Run it in sittings rather than in one pass — dd ingest --ocr-limit N stops after N pages and leaves the rest pending, and the next run picks them up where the last one stopped. Time twenty pages with --ocr-limit 20 before committing an evening to five thousand.
Python 3.10 or newer. The tool uses the sqlite3 module that ships with Python; dd doctor checks at runtime that this build has FTS5 and says so either way. Install pointers below were read at each project's own source — the URLs, the excerpts and the dates are in docs/_verified_pointers.md.
If you are going to work in Claude Code, clone it. That is the documented path, and it is not the same as installing the package:
git clone https://github.com/MiningTheSkies/discovery-desk
cd discovery-desk
python3 -m venv .venv && . .venv/bin/activate # Windows: py -m venv .venv then .venv\Scripts\activate
pip install -e .
pip install -e ".[msg]" # optional: Outlook .msg supportThen open Claude Code in that folder. Cloning is what gets you the harness — CLAUDE.md's session rules, a hook that refuses direct writes to the store, a session-start hook that prints dd status, the four slash commands, and the worker briefs. A pip install from the git URL delivers the dd command and none of that, and the harness is precisely what stops a session from writing rows behind your back or running batch after batch on its own.
If you only want the command-line tool, install it into an environment of its own:
pipx install "git+https://github.com/MiningTheSkies/discovery-desk" # pipx makes the environment for you
# or:
python3 -m venv ~/.venvs/dd && . ~/.venvs/dd/bin/activate && pip install "git+https://github.com/MiningTheSkies/discovery-desk"If you would rather not use git at all, download the repository as a ZIP, unpack it, and run pip install -e . inside the folder with a virtual environment active.
Debian and Ubuntu: the virtual environment was not created successfully because ensurepip is not available. This is the very first line of the documented path, and on those distributions it fails by default: they ship the venv module without the pieces that seed a new environment with pip, in a separate package. Install it and run the line again — sudo apt install python3-venv, or sudo apt install python3.N-venv for your version (the error message names the exact package for the Python you ran, and it is the one to use).
error: externally-managed-environment. Your operating system is working, not broken. Debian, Ubuntu, Fedora and Homebrew mark the system Python as managed by their own package manager, and pip refuses to install into it — the specification behind that refusal (PEP 668) says tools should "guide the end user towards using a virtual environment", which is what the message is doing. Use one of the two commands above. --break-system-packages also exists; it is the answer that eventually breaks something the operating system is relying on, so reach for it last rather than first.
Windows PowerShell: cannot be loaded because running scripts is disabled on this system. PowerShell is the default terminal on Windows 10 and 11, and its default execution policy on those editions refuses to run script files — including activate.ps1, which is what .venv\Scripts\activate resolves to there. Two answers: open cmd instead and run .venv\Scripts\activate.bat, which is not a PowerShell script and is unaffected; or allow scripts for that one window with Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass and activate again. The -Scope Process part matters — it lasts until you close the window and changes nothing about the machine.
Windows: 'dd' is not recognized as an internal or external command. The package installed and its console script went into a Scripts directory that is not on your PATH. Two answers: activate the environment you installed into (.venv\Scripts\activate), which puts it on PATH for that shell; or find out where it went and use the full path —
py -c "import sysconfig; print(sysconfig.get_path('scripts'))"
which prints the Scripts directory of the Python that ran it. The same one-liner works on macOS and Linux.
dd doctor prints something like unrecognized operand 'doctor'. dd is also the name of a standard byte-copying program that ships with macOS and Linux, and you are running that one — check which dd (macOS, Linux) or where dd (Windows) and see which comes first on your PATH. Activating the virtual environment you installed into fixes it; so does calling the script by its full path. The quick test: this tool's dd --version prints dd 0.1.0 and nothing else. If you get anything else — a package name in parentheses, or a complaint about an unknown operand — you are talking to the copying program.
Windows. Install Python from python.org (the official installer's SQLite is built with FTS5 enabled). For OCR, install Tesseract with the UB Mannheim installer from https://github.com/UB-Mannheim/tesseract/wiki, or winget install -e --id UB-Mannheim.TesseractOCR, or choco install tesseract; the vendor's own installer is usually newer than the package managers'. There is no Windows build of readpst to point you at — if you need PST, install Windows Subsystem for Linux, use the Debian instructions below inside it, and ingest what it writes out.
macOS. brew install tesseract for OCR; brew install libpst, which installs readpst.
Debian and Ubuntu. sudo apt install tesseract-ocr for OCR; sudo apt install pst-utils, which is the package readpst lives in.
Both external programs are optional and are detected at runtime. Without Tesseract, pages with no text layer stay marked pending-ocr and are counted. Without readpst, .pst files are counted and reported as unread. Neither is ever a silent skip — run dd doctor and it will tell you exactly what you are missing and what that costs you.
This tool is MIT. What it stands on is not all MIT, and you should be able to see that rather than discover it.
| What it is | License, as its own project states it | |
|---|---|---|
pypdfium2 |
primary PDF text extraction and page rendering; installed with the tool | "BSD-3-Clause, Apache-2.0, dependency licenses" — the project's own summary string in its package metadata, read 2026-08-16 |
pypdf |
pure-Python fallback extractor for machines where no native wheel installs; installed with the tool | BSD-3-Clause, as the SPDX expression in its package metadata, read 2026-08-16 |
extract-msg |
optional, for Outlook .msg files: pip install "discovery-desk[msg]" |
GPL-3.0 — a stronger copyleft than this tool's own license. Using it on your own documents is unremarkable; redistributing a combined work is a different question, and you should see the license rather than find it later. |
tesseract |
optional external program for OCR; not bundled | Apache-2.0 |
readpst (libpst / pst-utils) |
optional external program for .pst mailboxes; not bundled |
GNU GPL, version 2 or later — the Debian source package's copyright file and the Homebrew formula's GPL-2.0-or-later, both read 2026-08-16 |
Every row above was read at that project's own source rather than recalled; the URLs, the verbatim excerpts and the dates are in docs/_verified_pointers.md, which also records the extractor that was considered and not used, and why. Version numbers move and are deliberately not pinned here: the licenses and the package names are the durable part.
No LLM SDK. No web framework. No search service. No JVM. No embeddings.
Named out loud, because a tool that does not say where it stops invites you to assume it goes further.
No production module — no Bates stamping, no load-file writing, no redaction. No service of any kind: no Elasticsearch, no PostgreSQL, no Docker, no accounts. No embeddings, vector search, entailment models, or clustering. No automatic cross-document timeline presented as an answer. No PST parser inside the process, and no .nsf reader. No chat, Slack, Teams, or spreadsheet ingest — those are classified and refused. No claim of interoperability with any other document platform. No docket or deadline tracking. No detection of personal information. An exhibit list and an issue outline are natural next exports and are not in this build.
The neighbours, described accurately, since the honest question is whether you need this at all. Datashare is a free desktop application from the International Consortium of Investigative Journalists that indexes, OCRs, extracts named entities and searches, and genuinely runs on one laptop — the closest existing platform to this operating point, under AGPL-3.0. OpenAleph is the organization-scale answer, and its own FAQ says it "is meant to run on a server, it is not intended to be used on a personal computer." Microsoft's GraphRAG builds "a community hierarchy, generating summaries for these communities" and answers broad questions from those summaries — a good trade for understanding a body of material, and the wrong one when you need every relevant statement tied to a page. This tool does not integrate with any of them and makes no claim that it does.
discovery_desk/ the package
schema.sql the whole store: core, addendum, full-text index, refusals
cli.py the `dd` command line and the extension point every module registers through
db.py normalize.py dates.py bates.py connections, the normalization ladder, dates with precision, production numbers
shapes.py ingest.py pdf.py ocr.py email_ingest.py loadfile.py transcript.py reduce.py search.py
classification, the walk, extraction, OCR, mail, load files, transcripts, deduplication, search
ledger.py writing rows, following supersessions, printing citation packs
batch.py analysis.py manifest.py the review loop, the four views, the corpus manifest
demo.py a wholly fictional corpus, built on demand
tests/ the suite, including a test for every refusal
docs/
design.md why it has this shape, what is in the store, what it refuses
quickstart.md the fifteen-minute walkthrough, with the real output of each step
quote-check-measurement.md what the typed-quote check refuses, measured
corpus-arithmetic.md every sizing figure in this README, with its source and its arithmetic
_verified_pointers.md install and license claims, each read at its own project's source
demo-transcript.md one end-to-end run of the demo, every command with its exit code
_build_receipts_*.md the build records
scripts/ the measurement harness for the quote check — not installed by pip, and
the only thing in the repository that touches the network: it downloads
public documents to measure the check against
briefs/ worker briefs for a subagent given a batch, a candidate list, or one document
templates/handoff.md what one session leaves the next
.claude/ the Claude Code harness: hooks, slash commands
.github/workflows/ci.yml the checks, on two Python versions, with an OCR engine installed
CLAUDE.md the rules a session in this repository works under
What is gitignored, and why it matters if you work inside a clone. Your corpus directory and the store (desk.db) have been ignored since the first commit. Ignored alongside them are the three other places the tool writes your material into the working directory: derived/, where a mailbox container is unpacked into its messages; manifest.md, if you write it in the default place; and out/, which is where this README tells you to send exports. A manifest names file paths, custodians and designations, and a citation pack carries quotes out of your documents — neither belongs in a commit, and the ignore rules exist so that git add -A in a moment of tiredness cannot take them. None of it ever leaves your machine on its own: the tool makes no network calls.
Built by one operator working with Claude Code, and reviewed independently before it was published — four reviews, each reading only the artifact and its rubric, not the conversation that produced it; what they found was repaired before this went out. The refusal tests were watched failing before the refusals existed, which is the only way to know a check is wired to the thing it checks. The end-to-end run behind the quickstart, with every command and its exit code, is in docs/demo-transcript.md, including the section where it records the defects that run found. Continuous integration runs the suite on Ubuntu, on two Python versions, with Tesseract installed, so the OCR path has an oracle that a machine without Tesseract cannot provide; local runs skip those tests and print the reason. The demo corpus is entirely invented — every name, date, matter and document in it — and says so in its own README.
The method this tool applies is written up separately in the methodology kit at https://github.com/MiningTheSkies/agentic-company-seed.
MIT licensed. See LICENSE.
This is not a substitute for the advice of an attorney.