Skip to content

fix: regenerate state configs from dev; table-view wait matched an icon, not the table - #46

Merged
saqibmanan merged 9 commits into
mainfrom
fix/state-configs-from-dev
Sep 25, 2026
Merged

saqibmanan merged 9 commits into
mainfrom
fix/state-configs-from-dev

Conversation

@saqibmanan

@saqibmanan saqibmanan commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Part of #45 (get the pre-merge full regression green). Refs #44.

Why

Every automatic run targets dev: the PR gate, main.yml (pointed at dev until the prod move) and nightly. The committed config/states/*.yaml were discovered on 2026-06-17 and no longer match dev. The multistate tests select each configured indicator by its exact name, so every renamed or moved indicator fails its whole section.

A fresh baseline on main (Dispatch Tests, regression on dev, run 36150264203) failed on exactly these names:

  • Assam exposure / gov response: Elderly population, Children population (moved to vulnerability on dev), and the six Total Value of Flood Tenders… indicators (renamed to Flood Tenders : …)
  • Odisha exposure / vulnerability: Population*, Total Number of Households*, …flood and cyclone names

Change

Regenerated from dev with scripts/discover_state_indicators.py (2026-09-25), one commit per state. Himachal Pradesh is unchanged; its only diff was the timestamp.

State committed dev
Assam 30 32
Bihar 7 15
Odisha 19 22
Uttar Pradesh 6 22
Himachal Pradesh 32 32
Total 94 123

Evidence

  • Before, on main in CI: test_section_all_indicators_map_view[assam-Assam-exposure] failed with "2/4 indicators FAILED: Elderly population, Children population", and [assam-Assam-government-response] with "6/6 indicators FAILED".
  • After, locally against dev with this branch's configs: the same two tests give 2 passed.
  • This PR's own Full Regression (pre-merge) run is the full check.

Prod

Prod (drr.open-contracting.in) still serves Assam only: the other four states 404 on 2026-09-25, so #43 holds and the prod move hasn't happened. Prod isn't targeted by any automatic run, so these dev-based configs don't break anything there. When prod catches up with dev they will already match it. Same plan as before, with no environment-specific config added.

Second fix: every table-view test was failing (test bug)

Baseline shard 3: 15 of 15 TestMultiStateIndicatorsTableView tests failed in every state, including Himachal Pradesh, which has no drift. Each indicator failed with Table validation failed: Message: (a blank timeout).

The cause: the wait used EC.visibility_of_element_located((By.CSS_SELECTOR, "table, .table, [class*='table']")). Selenium takes the first match in the document, and that first match is a Tabler icon (<svg class="tabler-icon …">, since "tabler" contains "table"), not the table. Waiting on that icon's visibility timed out while the real table was rendered. I checked in the browser with the test's exact flow (Assam → Table View → Sivasagar / Sibsagar → Total Number of Households): a <table> with the right column showed within 1 ms.

The fix waits for any displayed <table>.

  • Red, locally with --reruns off: [uttar-pradesh-Uttar pradesh-hazard] → 1 failed, "Table validation failed" on both indicators
  • Green, after the fix: [uttar-pradesh-Uttar pradesh-hazard] and [assam-Assam-exposure] → 2 passed

The other two shard 3 failures (TestSectionCoverageByState Odisha exposure, Assam government response: "No indicators successfully tested") are the same indicator-name drift as above.

Third change: data gaps are reported, not failed

The first rerun of this PR (36165577942) left 4 failures, all of them the app working correctly with nothing to show:

  • Table view shows "No data available." HP vulnerability (the four Vulnerability Index indicators, sanitation and one more), Odisha "…Affected by Flood and Cyclone" (2), Assam "Immediate Measure Flood Tenders : Total Value". For HP Social Vulnerability Index I checked by hand: no data for either May or June 2026, at state or district level.
  • Chart view doesn't offer 2 UP government-response indicators ("Flood Tenders Under SDRF : Total value", "State Disaster Relief Funds Allocated District-wise During SEC meetings : Total Value"), while map and table view list all 8.

Decision (agreed): these are reported, not failed.

  • AnalyticsPage.is_no_data_displayed() and is_indicator_listed().
  • Table view: an explicit "No data available." → warnings.warn.
  • Chart view: an indicator the view doesn't list → warnings.warn. Map and table view still fail if an indicator really disappears.
  • pytest.ini: dropped --disable-warnings so the warnings summary is actually printed. The only other warnings in the suite were my own local -p no:rerunfailures config warnings.

Verified locally against dev: UP chart view plus the Assam and Odisha table views → 3 passed. The UP run's summary names exactly the 2 expected indicators.

A data card listing every "No data available." indicator will follow from this run's warnings summary.

Follow-up (not in this PR)

The chart and map waits (canvas, svg, .chart, [class*='chart'] and canvas, svg) are satisfied by any visible SVG icon, so they pass even if no chart or map renders. That's the same substring-selector trap as the table wait, in the other direction. It needs the real chart and map containers; to be tracked as a separate card.

Baseline summary (main, run 36150264203)

  • Shards 4–8 passed.
  • Shards 1 and 2: 9 failures. 8 are name drift; 1 is Assam hazard table view (the icon bug).
  • Shard 3: 17 failures. 15 are table view (the icon bug); 2 are section coverage (drift).

@saqibmanan saqibmanan changed the title fix(config): regenerate state indicator configs from dev fix: regenerate state configs from dev; table-view wait matched an icon, not the table Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Test Results (all)

368 tests  +6   341 ✅ +40   2h 16m 31s ⏱️ - 3h 57m 59s
  8 suites ±0    27 💤  -  8 
  8 files   ±0     0 ❌  - 26 

Results for commit 90dad34. ± Comparison against base commit d0b99a2.

♻️ This comment has been updated with latest results.

@github-actions

Copy link
Copy Markdown

QA Tests ✅ passed

  • Markers: all
  • Sharding: 8 shards × 2 workers
  • Target env: dev

Download the consolidated-test-report artifact for the merged report.html.

@saqibmanan
saqibmanan merged commit 27c9267 into main Sep 25, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant