Skip to content

Commit b964357

Browse files
committed
docs: add political spectrum section to verification standard (Phase 5/5)
1 parent ccf8b7d commit b964357

1 file changed

Lines changed: 106 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -466,6 +466,112 @@ updating its submodule reference.
466466

467467
---
468468

469+
## Political Spectrum & Source Bias
470+
471+
### The Problem
472+
473+
AI podcast scripts can inadvertently reflect a single political perspective when the source
474+
material fed into generation is ideologically one-sided. A script built entirely from sources
475+
rated "Left" by media-bias researchers will skew its framing, word choice, and which facts
476+
it emphasises — even if every cited reference passes verification.
477+
478+
**Concrete example:** A podcast about immigration policy sourced exclusively from outlets
479+
rated Left-Center produces accurate but one-sided content. Every URL resolves (✅ VERIFIED),
480+
yet a listener expecting balanced treatment is misled. Reference verification alone cannot
481+
catch this — it is orthogonal to the question of ideological balance.
482+
483+
### Approach
484+
485+
Static media-bias lookup at **content-extraction time**, not at verification time.
486+
487+
The lookup runs once per source URL when content is first extracted, before the script is
488+
generated. It annotates the extraction context with bias metadata. The script-generation
489+
prompt then receives conditional guidance — only when the topic is political — to seek
490+
balance or flag one-sidedness to the user.
491+
492+
This keeps bias detection cleanly separated from reference verification: the verification
493+
standard scores whether a reference is real; bias metadata informs whether the generation
494+
prompt should seek additional perspective.
495+
496+
### How It Works
497+
498+
```
499+
Source URLs (from content extraction)
500+
│
501+
▼
502+
Domain extraction
503+
(strip protocol, path, query)
504+
│
505+
▼
506+
MBFC dataset lookup ──► { bias, credibility, country }
507+
│ e.g. { bias: "left-center", credibility: "high" }
508+
▼
509+
Political topic detection
510+
(keyword match on topic + extracted content)
511+
│
512+
political? non-political?
513+
│ │
514+
▼ ▼
515+
Inject bias No bias
516+
guidance into guidance
517+
script-gen injected
518+
prompt
519+
```
520+
521+
Bias categories surfaced per source:
522+
523+
| Value | Meaning |
524+
|-------|---------|
525+
| `left` | Far-left leaning |
526+
| `left-center` | Center-left leaning |
527+
| `center` | Least-biased / centrist |
528+
| `right-center` | Center-right leaning |
529+
| `right` | Far-right leaning |
530+
| `conspiracy-pseudoscience` | Promotes conspiracy theories or pseudoscience |
531+
| `satire` | Satire — content should not be treated as factual |
532+
| `fake-news` | Known misinformation outlet |
533+
534+
When all detected sources share the same non-center rating and the topic is political, the
535+
script-generation prompt is augmented with guidance to note the ideological lean to the
536+
listener and, where possible, incorporate contrasting framing.
537+
538+
### Data Source
539+
540+
**Dataset:** [`drmikecrowe/mbfcext`](https://github.com/drmikecrowe/mbfcext) — a
541+
community-maintained mirror of [Media Bias / Fact Check (MBFC)](https://mediabiasfactcheck.com/)
542+
ratings, licensed MIT.
543+
544+
- **Size:** 9,773 sources (as of dataset release)
545+
- **Update cadence:** Auto-updated daily from MBFC ratings via the upstream repository's CI
546+
- **Fields used:** `domain`, `bias`, `credibility`, `country`
547+
548+
No network call is made at generation time. The dataset is bundled as a static JSON lookup.
549+
550+
### What This Does NOT Do
551+
552+
- **Does not reject sources.** A source rated `right` or `left` is not excluded from the
553+
script. The verification standard continues to assess whether the reference is real.
554+
- **Does not editorialize.** The system does not label content "biased" to the end user
555+
unprompted. Guidance is injected into the generation prompt, not the podcast transcript.
556+
- **Does not apply to non-political topics.** Technology tutorials, science explainers,
557+
cooking guides — bias guidance is suppressed entirely when political topic detection
558+
returns negative.
559+
560+
### Limitations & Transparency
561+
562+
| Limitation | Detail |
563+
|------------|--------|
564+
| One framework among several | MBFC is widely cited but not the only media bias rating system. [AllSides](https://www.allsides.com/) and [Ad Fontes Media](https://adfontesmedia.com/) use different methodologies and sometimes reach different conclusions for the same outlet. |
565+
| US-centric dataset | MBFC coverage is strongest for US English-language media. Non-US sources are rated but coverage is uneven; many regional outlets are absent from the dataset entirely. |
566+
| Source-level ≠ article-level | A center-rated outlet can publish a one-sided op-ed. A left-rated outlet can publish a balanced investigative piece. The lookup reflects outlet-level ratings, not individual article analysis. |
567+
| Static snapshot | The bundled dataset reflects ratings at a point in time. Outlets change ownership and editorial stance; the dataset may lag real-world shifts by weeks or months. |
568+
| No confidence score | MBFC ratings are categorical, not probabilistic. The dataset does not expose reviewer agreement or confidence — every rating is treated with equal weight regardless of how contested it may be. |
569+
570+
Community contributions that extend coverage to non-US sources, integrate a second bias
571+
framework, or add article-level analysis are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md).
572+
573+
---
574+
469575
## Prior Art & Theoretical Basis
470576

471577
The Bayesian log-odds algorithm is a well-established pattern in statistics and evidence-based

0 commit comments

Comments
 (0)