Skip to content

fix(sitemap): remove getProvider() fan-out — pre-filter slugs in worker - #2155

Merged
Flotapponnier merged 1 commit into
mainfrom
hotfix/sitemap-no-getprovider
Aug 27, 2026
Merged

fix(sitemap): remove getProvider() fan-out — pre-filter slugs in worker#2155
Flotapponnier merged 1 commit into
mainfrom
hotfix/sitemap-no-getprovider

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Problem

After the slim-blob sitemap shipped, buildFullSitemap() called getProvider(slug) 505 times in parallel to validate provider slugs. Each call internally triggers loadAggregateFromBlob() which cannot be cached (>2MB unstable_cache limit), so each of the 505 calls fetched the 4.25MB blob independently — OOM crash in ~65s, bypassing both the 75s JS timeout and the route handler try/catch.

Fix

Worker (publish-aggregate.ts): pre-filter providerSlugs at publish time to exclude chain slugs, HL builder slugs, perp venue slugs (except polymarket), and removed slugs. These are O(1) set lookups using shared constants.

Vercel (sitemap-builder.ts): replace the Promise.all(getProvider()) async fan-out with a sync .filter() using the same set checks (safety net). Remove the dead getProvider import. COMPARE_PAIRS routes are also emitted directly (curated list, no validation needed).

Result: buildFullSitemap() makes zero async provider calls — fetches only the 50KB slim blob + answers + alternatives (~15ms total).

@Flotapponnier
Flotapponnier merged commit 46b03e7 into main Aug 27, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant