Created: 16-01-2014 · Last updated: 28-07-2026
Monier Monier-Williams, Sir; A Sanskrit-English Dictionary. Oxford, 1899.
This repository holds corrections, enhancements, and tooling for the Cologne digitization of the MW dictionary. The canonical source data (mw.txt in SLP1 encoding) lives in csl-orig; the build system is in csl-pywork. Text corrections are never made directly to the source — they are expressed as change files applied by scripts, following the canonical correction workflow. Issues and corrections are tracked at the MWS GitHub issue tracker; the dictionary is served online through the Cologne interface, and this repository publishes a GitHub Pages landing page.
| Doc | What it covers |
|---|---|
| docs/PIPELINE_MANUAL.md | Operator manual — the universal correction loop, every re-runnable pipeline with exact commands, what is frozen, symptom→cause→cure |
| DATA_DICTIONARY.md | Tag and field reference for mw.txt |
| ENTRY_GUIDE.md | How to read an MW entry |
| DICT_PROFILE.md | The dictionary itself — editions, history, when to use |
| ANALYSIS.md | The five read-only 2026 analysis modules |
| ROADMAP.md | 2026 H2 plan (workstreams W1–W4) |
| CONTRIBUTING.md | Correction workflow for contributors |
| CHANGELOG.md | Dated maintenance snapshots |
| Directory | Description |
|---|---|
history/ |
Original MONIER.ALL from Thomas Malten (2004); cp1252→UTF-8 conversion |
homophone/ |
Homophone markup corrections and enhancements; Java and Python pipelines |
mwtranscode/ |
Transcoding between SLP1, IAST, and Devanagari |
mwsissues/ |
Per-issue correction workflows and documentation |
mwabbreviations/ |
Analysis of abbreviations used in the digitization |
mwauthorities/ |
Works and authors cited in MW |
mwverbs/ |
Verb and preverb extraction from the digitization |
botbio/ |
Botanical (<bot>) and biographical (<bio>) tag extraction |
k1k2/ |
Headword key1/key2 clash analysis |
accent_diff/ |
Accent markup discrepancy analysis |
Lithuanian/ |
Lithuanian word list comparison against MW |
mwsupplement/ |
MW supplement entries |
greek_andhrabharati/ |
Greek words comparison between Cologne digitization and Andhrabharati |
CORRECTIONS_issue_362/ |
Language-tag corrections from Nagabhushana Rao (@Andhrabharati) (csl-corrections issue #362) |
verbs01/ |
Verb merge and cross-reference tooling |
transcodeExample/ |
Example transcoder PHP/Python scripts and SLP1→IAST table |
basic04a/ |
Simple two-dictionary web display sample |
list02php/ |
PHP-based display sample |
prefaces/ |
Front-matter OCR (title page, Preface, Introduction §§ I–V) with Russian translation — see Front matter below |
Faithful OCR of the 29 front-matter scan pages of the New Edition (Oxford, at the Clarendon Press, 1899) — the title page, the Preface to the New Edition (pp. v–x, including the Postscript added by the author's son M. F. Monier-Williams), and the Introduction in five sections (pp. xi–xxxii) — each in the English source plus a Russian translation, with consolidated single-file editions and a prefaces/README.md index. Source: the Cologne csldoc preface scans. Because the source is already English there are no per-page .en.md files (the base .md is the English edition); the consolidated outputs are mwpref_all.en.md and mwpref_all.ru.md. Devanāgarī/Sanskrit is kept verbatim with full diacritics; the p. xxvii comparative-alphabet table (Phœnician → Greek/Roman/English and Brāhma → Nāgarī) is preserved (archaic glyphs marked, Nāgarī column in Unicode); digitizer header/footer stamps are omitted. The Preface is signed “Indian Institute, Oxford, 1899. — Monier Monier-Williams”; the Postscript records the author's death at Cannes, in the south of France, on 11 April 1899.
OCR run notes (2026-06-17) — process retrospective, not part of the deliverable. Produced by the
/cologne-preface-ocrskill (vision OCR + translation subagents). Reusable technical lessons:
- MW's csldoc scans are
.jpg, not.png, and low-resolution (1200×1686). 1200 px is the maximum available; native-resolution crop bands beat a full-page read, but final fidelity is capped by the scan itself.- Trust the toctree order over filename sort. Pages 26/27 embed scans
mw010034/mw010033(swapped).- Verify subagent output against the scan; don't trust self-reports. One agent silently truncated page 27 (dropping the SECTION V heading and its first two paragraphs) and mis-split the p. 24/25 boundary. A scripted continuity check (every page tail → next-page head) plus verifying each page's true first line against the scan top surfaced both.
- A subagent can die mid-run on a 400 content-filter error and produce nothing; redo those pages in the foreground with proper crops.
- Heading levels matter for the consolidated build. In-body section headings (POSTSCRIPT, SECTION n) must be
##not#, orbuild_combined.py's H2 sanity-count overshoots.- Encoding. All
.mdfiles written UTF-8 no BOM; git's LF→CRLF warning on Windows is cosmetic (committed blobs are LF).
| Date | Milestone |
|---|---|
| 2004 | Thomas Malten provides MONIER.ALL — original digitization in cp1252 encoding |
| Jan 2014 | Repository initialized; early data analysis (hiatus entries, avagraha, space in keys) |
| Sep 2014 | Transcoder example added (issue #5) |
| Apr–Jun 2015 | Homophone corrections: ~6,500 removeHom + 10,913 artificial homophones assigned |
| Dec 2015 | k1k2 headword clash analysis |
| Jul 2016 | mwauthorities XML structure established |
| Feb 2017 | Web display samples added (basic04a, list02php) |
| Nov 2017 | mwabbreviations analysis added |
| Jan 2020 | Greek words file received from Nagabhushana Rao (@Andhrabharati) (issue #89) |
| Jun 2020 | botbio tag extraction and mwverbs pipeline added |
| Jan 2021 | mwtranscode: SLP1 ↔ IAST ↔ Devanagari pipeline; Lithuanian comparison; Andhrabharati (AB) version work |
| 2024 | Issues 141–181: accent corrections, Grassmanizing, AB3 (Andhrabharati) alternate format, supplement revisions |
| Aug–Nov 2025 | Issue 190: recovery of 16+ lost headwords (with Nagabhushana Rao (@Andhrabharati) and Scott Rhodes (@aumsanskrit)) |
| Feb 2026 | History folder: MONIER.ALL archived and documented |
| Jun 2026 | Front-matter OCR + Russian translation of the 1899 prefaces (prefaces/) |
Work is organised into four GitHub Projects (org-level kanban boards), each mirroring a milestone:
| Project | Milestone | Open | Closed | Scope |
|---|---|---|---|---|
| Dictionary to Book | milestone | 5 | 8 | Link targets and link splitting |
| Digitization Quality | milestone | 6 | 74 | Scan quality, encoding, bug fixes, text corrections |
| Structured Data | milestone | 11 | 63 | Markup normalisation, structured data improvements, editorial questions |
| Major Enhancements | milestone | 15 | 14 | Display upgrades, new data, large structural additions |
Issue counts as of 10-07-2026 (live figures shift as issues are opened and closed).
pie title Closed issues by milestone
"Digitization Quality" : 74
"Structured Data" : 63
"Major Enhancements" : 14
"Dictionary to Book" : 8
pie title Open issues by milestone
"Major Enhancements" : 15
"Structured Data" : 11
"Digitization Quality" : 6
"Dictionary to Book" : 5
Issues track two broad concerns: enriching the XML (markup, link targets) and improving the digitization (encoding, scan quality, text corrections).
pie title Issues by type label
"markup" : 53
"text-correction" : 46
"content-enhancement" : 29
"question" : 22
"bug" : 19
"link-target" : 12
"encoding" : 10
"scan-quality" : 3
"link-splitting" : 1
| Type | Description | Examples |
|---|---|---|
| Link targets | Building clickable references from <ls> abbreviations to scanned PDF pages (7 issues). |
Hyperlinking MW to Panini #77, Gram. link #159, VIKRAMORVAŚĪ #188, Ramayana #151 |
| Link splitting | Splitting combined N,N refs into per-page links (1 issue). |
Dhātup. links #126 |
| Markup | Normalising XML tag content and structure: <ls>, <ab>, <lex>, <bot>, <hom>, <bio> (45 issues). |
Missing ls tags #112, RV <ls> cleanup #134, <hom> consistency #131, 6800 missing ¦ #132 |
| Text corrections | Corrections to headwords, definitions, accent marks, and orthography (46 issues). | Accent correction phase 1–4 #141, #142, #145, typos #102 |
| Content enhancement | Display upgrades, new data, structural additions (14 issues). | Cross-entry links #64, supplement #83, verbs01 #75 |
| Encoding | SLP1/IAST/Devanagari transcoding, Greek and Lithuanian rendering (8 issues). | Lithuanian IAST #79, Greek text #153, German encoding #42 |
| Scan quality | Replacing missing or poor-quality scan pages (3 issues). | Two missing pages #81, scan review #144 |
| Bug fixes | Broken display, XML errors, broken links (17 issues). | Abnormal <root/> tag #27, punctuation in ls refs #54, nikāya lost #128 |
| Questions resolved | Scholarly and editorial questions researched and answered (17 issues). | MW missing feminine data #84, ŚivaPurāṇa refs #125, Zend language #114 |
| Type | Description | Examples |
|---|---|---|
| Link targets | Sources still needing index and links installed (5 open issues). | PAÑCATANTRA #185, ŚĀKUNTALA #186, MĀLAVIKĀGNIMITRA #187, missing links #129 |
| Markup | XML tag normalisation still in progress (8 open issues). | New markup for alternates #147, tag inventory #168, titular abbreviations #172 |
| Content enhancement | Display upgrades and new data (15 open issues). | Display variant part 2 #73, resolving idems #98, grouped entries #163, web font #170 |
| Encoding | Transcoding edge cases (2 open issues). | IAST to ISO 15919 #155, recoding shortlong #164 |
| Bug fixes | Known display and link errors (2 open issues). | &c. abbreviation #86, MW links to Mn. #189 |
| Questions / interpretation | Open scholarly questions (5 open issues). | cf. accord. to some #45, Ka or KA #93, genders in bold #108 |
Every issue carries one type label and one severity label.
| Label | Meaning |
|---|---|
link-target |
Building a click-through from a <ls> abbreviation to scanned PDF pages |
link-splitting |
Splitting combined SOURCE N,N refs into individual per-page links |
markup |
Normalising XML tag content or structure (<ls>, <ab>, <lex>, <bot>, etc.) |
text-correction |
Corrections to headwords, definitions, accent marks, or orthography |
content-enhancement |
New material, display upgrades, or structural additions beyond correction |
encoding |
SLP1/IAST/Devanagari transcoding, Greek/Lithuanian rendering, character normalisation |
scan-quality |
Replacing blurry, skewed, or missing scan pages |
bug |
Broken display, XML structure errors, broken links |
question |
Scholarly or editorial questions requiring research before any code change |
| Label | Meaning |
|---|---|
minor |
Targeted, self-contained fix — a handful of entries or a single file |
medium |
Standard unit of work — one link-target index, a batch of corrections |
hard |
Large effort spanning many entries, files, or dictionaries |
- Thomas Malten — provided the original MONIER.ALL digitization (2004)
- Peter Scharf — designed the rational extension of homophone markup (2013); requested Python reimplementation
- Pawan Goyal — co-designed homophone markup with Scharf (2013)
- Jim Funderburk (@funderburkjim) — primary repository maintainer; tooling and correction workflows
- Mārcis Gasūns (@gasyoun) — initial commit and early data analysis
- drdhaval2785 (@drdhaval2785) — k1k2 clash analysis; AB/Cologne comparison tooling
- Nagabhushana Rao (@Andhrabharati) — Greek words file; AB version analysis; extensive issue contributions
- Scott Rhodes (@aumsanskrit) — issue analysis and corrections (issue #190 and others)
- Darius — Lithuanian word list comparison
See the readme.txt in the homophone directory.
Dr. Mārcis Gasūns