국회 출석부

국회 출석부 is a public web app and data pipeline for tracking plenary vote activity and asset disclosure history in the National Assembly of Korea. The project collects official upstream data, normalizes it into stable datasets, publishes lightweight JSON exports, and renders those exports in a static client.

The current product is organized around two main views. The home view focuses on recent vote activity and accountability signals, while the calendar view helps readers inspect participation patterns, compare individual lawmakers over time, and review published asset disclosure history for current members of the 22nd Assembly.

What The App Shows

• Accountability leaderboard for lawmakers in the current assembly
• Latest vote feed with official tallies, highlighted no or abstain votes, and verified absences
• Weekly accountability trend summaries for the assembly
• Member search from the published ranking data
• Member activity calendar with per-day participation states
• Compare mode for viewing two members side by side in the calendar experience
• Shareable deep links for member and compare views
• Lazy member detail loading so detailed vote records are fetched only when needed
• Member asset disclosure history in the calendar view for current 22nd Assembly lawmakers
• Asset summary cards with latest total, first disclosure date, latest disclosure date, and cumulative delta
• Category toggles for asset subtotal series plus a real-estate-focused readout for land and building categories
• Scope toggle for family-included totals versus self-only totals when item-level disclosure parsing supports it
• Member history analysis at #analysis: source-backed career periods, role/activity timelines, period-adjusted peers, cross-party voting relationships, topic continuity, and asset/role chronology. See the analytical model and verification notes.

Repository Layout

• apps/web: Vite + React frontend deployed to GitHub Pages
• packages/ingest: ingestion scripts, parsers, normalization, export builders, source monitoring, and document mirroring
• packages/schemas: shared Zod schemas and TypeScript types for normalized records and published contracts
• tests/fixtures: fixed upstream payload fixtures used by pipeline and parser tests
• .github/workflows: scheduled and manual workflows for ingest, build, deployment, monitoring, and document mirroring

Data Flow

1. ingest-live fetches official Assembly API payloads and writes a raw snapshot artifact for vote, member, committee, and meeting data.
2. mirror-property-disclosures queries the official Assembly file-service JSON listing for property disclosure PDFs, mirrors the files into the public data repository, and refreshes a JSON-only property member context cache.
3. build-data reads the raw ingest snapshot plus the mirrored property inputs from the public data repository, builds normalized datasets, and publishes public exports plus a manifest.
4. deploy-web builds the frontend against the published data base URL and deploys the static site to GitHub Pages.
5. monitor-sources checks official upstream payloads for changes that could break parsing assumptions.
6. mirror-documents mirrors other public Assembly documents into the public data repository and updates mirror state metadata.

This repository owns the application code, schemas, tests, and workflow orchestration. Published datasets and exports are meant to live in a separate public data repository referenced by the web app at runtime.

The asset disclosure path is intentionally split from the XML-derived vote snapshot path. Property mapping uses a JSON-only cached roster and tenure context built from official Assembly OpenAPI responses and does not depend on the XML member snapshot used by the main vote pipeline.

Published Data

The frontend reads public JSON exports and validates them with the schemas package.

• exports/latest_votes.json: home feed payload for recent plenary votes
• exports/accountability_summary.json: ranked lawmaker summary used by the leaderboard and search
• exports/accountability_trends.json: weekly trend points and mover windows for the overview charts
• exports/member_activity_calendar.json: calendar summary payload for the current assembly
• exports/member_activity_calendar_members/<memberId>.json: lazily loaded member vote record details
• exports/member_statement_summaries/index.json: members with published bill-related minutes summaries and their export paths
• exports/member_statement_summaries/<memberId>.json: locally generated summaries of each member's officially attributed bill-related statements with source evidence
• exports/member_assets_index.json: current-member asset disclosure index with latest summary and per-member history pointers
• exports/member_assets_history/<memberId>.json: lazily loaded member asset disclosure history and category subtotal series
• exports/member_sponsorship_accounts.json: official sponsorship committee registration and online donation routes without direct-deposit account details
• curated/asset_disclosures.parquet: mirrored disclosure file metadata
• curated/asset_disclosure_records.parquet: parsed disclosure record summaries per named filer
• curated/asset_disclosure_categories.parquet: parsed category subtotal rows per disclosure record
• curated/asset_disclosure_items.parquet: parsed asset item rows with relation and detail text fields when available
• manifests/latest.json: manifest with dataset and export metadata, checksums, and current assembly information

Shared contracts for these files are defined in @lawmaker-monitor/schemas.

Local Development

Install dependencies and run the shared checks:

npm install
npm run build
npm test
npm run typecheck

Useful commands during development:

npm run dev:web
npm run ingest:live --workspace @lawmaker-monitor/ingest
npm run build:data --workspace @lawmaker-monitor/ingest
npm run sync:property-member-context --workspace @lawmaker-monitor/ingest
npm run test:ui

UI flow coverage and screenshot outputs are documented in docs/testing/ui-test-cases.md.

Additional operational scripts:

npm run mirror:documents --workspace @lawmaker-monitor/ingest
npm run sync:property-member-context --workspace @lawmaker-monitor/ingest
npm run monitor:sources --workspace @lawmaker-monitor/ingest

GitHub Actions Workflows

• ingest-live: runs the live ingest pipeline and uploads a raw snapshot artifact
• build-data: consumes a raw snapshot, builds normalized outputs, materializes public exports, and syncs them into the public data repository
• deploy-web: builds the frontend and deploys it to GitHub Pages
• monitor-sources: checks upstream source stability and opens an incident issue when parser assumptions fail
• mirror-documents: mirrors public Assembly documents into the public data repository
• summarize-minutes: runs a local lightweight model after document mirroring and publishes member-level bill statement summaries
• mirror-property-disclosures: mirrors official property disclosure PDFs, refreshes property member context JSON caches, and publishes those raw inputs into the public data repository

Supported Ingest Environment Variables

Only the following Assembly ingest variables are part of the supported surface and should be documented or configured for the live ingest workflow:

• ASSEMBLY_API_KEY
• ASSEMBLY_PAGE_SIZE
• ASSEMBLY_BILL_FEED_CONCURRENCY
• ASSEMBLY_VOTE_DETAIL_CONCURRENCY
• ASSEMBLY_BILL_VOTE_SUMMARY_CONCURRENCY
• ASSEMBLY_CONNECT_TIMEOUT_MS
• ASSEMBLY_FETCH_TIMEOUT_MS
• ASSEMBLY_FETCH_RETRIES

ASSEMBLY_API_KEY is required for production-like ingest runs. The remaining values tune page size, concurrency, timeout, and retry behavior for official API collection.

ASSEMBLY_CONNECT_TIMEOUT_MS controls connection establishment to open.assembly.go.kr (default 15000 ms), separately from the full request/body deadline ASSEMBLY_FETCH_TIMEOUT_MS (default 20000 ms). Keep the full request deadline greater than the connection deadline. ASSEMBLY_FETCH_RETRIES=0 disables retries; transient transport errors and HTTP 408/429/500/502/503/504 are retried with bounded exponential backoff and jitter, while authentication and validation errors fail immediately.

The property mirror's file-list POST uses MIRROR_TIMEOUT_MS and MIRROR_FETCH_RETRIES, not the ASSEMBLY_FETCH_* settings. Its workflow defaults are 45000 ms and 3 retries (at most 4 attempts). Increasing a timeout is not a substitute for diagnosing persistent source or network failures.

Ingest, build and property-mirror workflows retain a separate diagnostic artifact on failure. It records this attempt's timestamps, outcome, run ID and sanitized error. It is not a successful source snapshot; failed collection never advances published mirror state, and incomplete raw/build outputs are not promoted.

Operational Settings

The project also uses repository or local settings for data publication and document mirroring.

Data publication settings:

• DATA_REPO
• DATA_REPO_BRANCH
• DATA_REPO_BASE_URL
• DATA_REPO_PAT
• COLLECT_SPONSORSHIP_ACCOUNTS
• SPONSORSHIP_FETCH_TIMEOUT_MS
• SPONSORSHIP_FETCH_CONCURRENCY

Web analytics settings:

• GA_MEASUREMENT_ID: GA4 web stream measurement ID configured as a GitHub Actions repository variable. The deploy workflow exposes it to Vite as VITE_GA_MEASUREMENT_ID.

The web app does not load Google Analytics when the measurement ID is absent or invalid. When enabled, it records one page view for the initial screen and each distinct hash route, retains public member and region identifiers while dropping other hash parameters from analytics URLs, disables advertising signals, and defaults all Consent Mode v2 storage settings to denied.

Public document mirror settings:

• MIRROR_MODE
• MIRROR_SOURCE_ID
• MIRROR_START_URL
• MIRROR_READY_SELECTOR
• MIRROR_ROW_SELECTOR
• MIRROR_TITLE_SELECTOR
• MIRROR_LINK_SELECTOR
• MIRROR_LINK_ATTRIBUTE
• MIRROR_DATE_SELECTOR
• MIRROR_NEXT_SELECTOR
• MIRROR_MAX_PAGES
• MIRROR_MAX_DOWNLOADS
• MIRROR_PAGE_DELAY_MS
• MIRROR_TIMEOUT_MS
• MIRROR_FETCH_RETRIES
• MIRROR_TIME_ZONE
• MIRROR_RECENT_DAYS
• MIRROR_MIN_RECENT_DAYS
• MIRROR_BACKFILL_START_DATE
• MIRROR_BACKFILL_DAYS
• MIRROR_BACKFILL_WINDOWS_PER_RUN
• MIRROR_CATCHUP_WINDOWS_PER_RUN
• MIRROR_INCLUDE_APPENDICES

The scheduled minutes mirror always applies MIRROR_MIN_RECENT_DAYS as a freshness floor, even when MIRROR_RECENT_DAYS is configured to a smaller value. This rolling overlap captures minutes that are published after their meeting date. Historical coverage advances independently from MIRROR_BACKFILL_START_DATE; successful runs dispatch bounded catch-up runs without repeating the recent window until the backfill reaches current coverage. Mirror state reports the latest discovered and mirrored meeting dates, whether backfill remains, and transcript parser refresh counts.

Minutes summary settings:

• LLAMA_CPP_RELEASE
• MINUTES_SUMMARY_MODEL
• MINUTES_SUMMARY_CHAT_TEMPLATE_URL
• MINUTES_SUMMARY_MAX_DOCUMENTS
• MINUTES_SUMMARY_MAX_GROUPS
• MINUTES_SUMMARY_CONCURRENCY
• MINUTES_SUMMARY_REQUEST_TIMEOUT_MS

The summary workflow runs an open-weight GGUF model directly on the GitHub Actions runner. The default is the official EXAONE 4.0 1.2B Q8 GGUF with its pinned chat template. The model ranks important source sentences, but published summary text is copied verbatim from the collected minutes. Generated guidance is rejected when it introduces unsupported names, institutions, numbers, or sentences with insufficient source overlap. Long agenda-level statement groups are also segmented by size and intervening speaker turns before ranking. Legacy abstractive artifacts are excluded by the active prompt version and remain hidden until regenerated. It does not send minutes to a hosted inference API. The Assembly search endpoint is used only as a document catalog to discover minutes IDs and dates. Every summary input must come from a separately downloaded official minutes viewer document at record.assembly.go.kr/assembly/viewer/minutes/xml.do. The pipeline splits those documents into member statements before the model is invoked. Speaker and agenda attribution come from that structured minutes document.

Collection and summarization are intentionally incremental. A mirror run downloads at most 20 documents and advances at most two catch-up windows. A summary run processes one transcript and up to eight member-agenda groups with two concurrent local-model requests. Each model request times out after 60 seconds, completed work is committed after every run, and the next scheduled run continues remaining work. Successful runs do not recursively dispatch more summaries.

The daily pipeline is fully chained in this order:

1. mirror-documents searches the rolling recent window and writes official transcript artifacts.
2. A successful mirror explicitly dispatches summarize-minutes.
3. summarize-minutes publishes one checkpointed batch. Unattempted work gets priority over earlier attempts; transient failures receive exponential cooldowns, and source groups without a publishable sentence are explicitly recorded as skipped. Partial progress is committed before reporting any transient failure.
4. A successful summary run triggers deploy-web.

Recent-window collection, historical backfill, summary processing, and the other data builders share a FIFO concurrency queue. The queue retains up to 100 pending runs so a newly dispatched summary batch cannot displace the daily recent-window collection. All generated-data writers publish through scripts/publish-data-repo.sh, which stages only the declared paths and retries fetch, rebase, and push operations. Failed mirror or summary batches automatically dispatch up to three recovery runs. Manual cancellation is not retried.

Use docs/operations/minutes-reclassification-verification.md to verify workflow completion, published coverage, source grounding, regression samples, and the final production deployment after a reclassification run.

Property disclosure mirror settings use the same document mirror infrastructure plus Assembly file-service specific inputs:

• MIRROR_SERVICE_INF_ID
• MIRROR_SERVICE_INF_SEQ
• MIRROR_INDEX_PATH
• MIRROR_STATE_PATH
• PROPERTY_SOURCE_ID
• PROPERTY_MEMBER_CONTEXT_MANIFEST_PATH
• PROPERTY_MEMBER_INFO_PATH
• PROPERTY_MEMBER_HISTORY_PATH

See .env.example for a concrete local configuration template.

Source Policy

Lawmaker Monitor is built around official Assembly sources for vote, meeting, member, committee, and related plenary data. The asset disclosure pipeline also stays on official Assembly surfaces: the property PDF list comes from the public file-service JSON endpoint, and property member mapping comes from cached official JSON OpenAPI responses for current member roster and member history. Sponsorship collection matches the current member roster against the National Election Commission Political Donation Center and publishes only official committee registration and online donation routes. Direct-deposit bank, account-number, and account-holder details are neither collected for the public export nor rendered in the product or external share cards. The ingest pipeline, source monitoring checks, and public export contracts are all designed around keeping that official-source path stable over time.

Reference Material

• docs/references/assembly-openapi-reference.md: concise human-readable notes about the Assembly OpenAPI surface used by this project
• docs/references/assembly-openapi-endpoints.json: machine-readable endpoint registry for prompts, tooling, and automation

Open Source Dependencies

Lawmaker Monitor depends on open-source tools and libraries across the web application, geospatial visualization, data ingestion, and test pipeline.

• UI and visualization: React, Recharts, and Phosphor Icons
• Social preview rendering: resvg-js and Noto Sans KR via Expo Google Fonts
• Geospatial processing: H3, TopoJSON, and Proj4js
• Data ingestion: PDF.js, Playwright, Cheerio, and fast-xml-parser
• Contracts and tooling: Zod, Vite, Vitest, TypeScript, ESLint, and Prettier

See NOTICE for the dependency license index, required upstream attributions, and the current project-license status.

Build memory and failure diagnostics

build-data runs with an explicit 4096 MiB V8 old-space budget; this is not a larger-heap workaround. Normalized/property NDJSON is written with backpressure, with checksums computed from the exact written bytes. After DuckDB materializes Parquet, scripts/finalize-data-manifest.mjs replaces dataset checksums with hashes of the final published Parquet bytes. JSON export hashes are unchanged. Existing NDJSON seed, newline, and row-count conventions are preserved.

The calendar reuses one Korean date formatter and computes each roll-call date once, rather than allocating ICU formatters for every member/roll-call pair. Geography writes one province's boundary and hexmap artifacts at a time, retaining only index metadata after each province. The full source GeoJSON is still parsed once; the build does not claim memory independent of the source dataset size.

Property PDFs are parsed one document at a time. Each page is cleaned after text extraction and each loading task is destroyed even on errors. Raw ingest payloads are not passed into the publish stage.

bash scripts/run-data-build.sh retains code/data SHA provenance, the source ingest run ID, process exit code and peak RSS (/usr/bin/time on Linux and Python 3 standard library child resource usage on macOS). build-memory.ndjson contains synchronous stage checkpoints (heapUsed, rss, external/ArrayBuffer memory and row/byte counts). Its sampled peaks are not continuous heap profiling. No heap snapshot, environment dump, source text or API key is included. An unfinished running diagnostic plus a non-zero process exit is a failure, not a success. BUILD_DIAGNOSTICS_DIR optionally redirects the diagnostic files. Publication and deployment still require the build to succeed.

Operational verification

Scheduled monitor-sources validates the latest successful live ingest artifact, checks that it and the published manifest are at most 96 hours old, probes the live official member API, and checks mirror/summary failure counters. Local runs without SOURCE_MONITOR_LIVE=true remain fixture regression checks and report that mode explicitly. The monitor workflow opens or updates an incident on failure.

Minutes collection overlaps the preceding 30 days and resumes historical gaps from the last mirrored meeting date when the old same-day mode cleared the cursor. A seven-day revalidation interval prevents unchanged documents from consuming every bounded batch. PROPERTY_MIRROR_TIMEOUT_MS controls the property workflow timeout (default 60000 ms), independently of legacy MIRROR_TIMEOUT_MS repository values. Playwright request timeouts are retried with the same bounded transient-error policy. PR checks now run the responsive browser suite and retain screenshot evidence.