Skip to content

waitForIpniProviderResults: surface per-CID outcomes for diagnostic visibility #417

Open

Description

@SgtPooki

Problem

waitForIpniProviderResults returns Promise<boolean> and on failure throws a single Error whose message references one CID (the last one checked). Consumers cannot tell, after the fact, which CIDs in [rootCid, ...childBlocks] were verified vs. timed out vs. missing-expected-provider vs. fetch-errored.

This matters most on the abort/timeout path: when an outer AbortSignal.timeout(...) fires mid-walk, the in-flight fetch throws AbortError and we lose which CID was current and which CIDs had already been verified successfully.

Concrete example from the dealbot consumer (FilOzone/dealbot):

  • Deals are uploaded with rootCID + ~12 childBlocks. Outer timeout 60s, delayMs 2500ms, so each CID gets ~25 attempts.
  • 7 of 8 SPs verify cleanly at p50 ~2.4s. SP6 (Hangzhou CN) verifies at p50 ~31s with 39% slow_verification rate; ~22 deals/24h time out at the 60s budget.
  • All timeouts surface as a single generic message. We currently cannot distinguish "rootCID never indexed" from "rootCID indexed but childBlock N never indexed."
  • Investigation: IPNI xxx root CID not verified FilOzone/dealbot#473 (comment)

Why this is currently low priority

  1. In observed dealbot data, all timeouts are wall-clock SLO breaches, not selective "missing expected provider" failures. Per-CID granularity would not have shifted root-cause attribution (path quality between filecoinpin.contact / Reston VA and SP6 / Hangzhou CN, AS4134 ChinaNet, ICMP packet loss 24% from a peer test vantage).
  2. At dealbot's current SLO budget (60s with 2.5s polls and 12 CIDs), only the first 4-5 CIDs get any attempts before the outer abort fires anyway — so child-CID granularity is not measurable end-to-end without also rebudgeting.
  3. PR fix(ipni): preserve failure reason in deal-flow rethrow FilOzone/dealbot#490 + feat(ipni): expose failureReason as top-level field on ipni_tracking_failed FilOzone/dealbot#491 close the immediate visibility gap that prompted dealbot#473 by carrying the existing single failureReason via Error.cause.

Proposal (when prioritized)

Change return type from Promise<boolean> to something like:

```ts
type IpniValidationOutcome = {
verified: Array<{ cid: CID; matchedAt: number; attempts: number }>;
failed: Array<{
cid: CID;
reason:
| { type: 'timeout'; attempts: number }
| { type: 'missingProviders'; missingServiceUrls: string[]; actualMultiaddrs: string[] }
| { type: 'fetch'; message: string }
| { type: 'parse'; message: string }
| { type: 'http'; status: number; statusText?: string };
}>;
ipniIndexerUrl: string;
};
```

Existing single-error throw can be preserved as a thin wrapper for callers that prefer the boolean shape, or replaced with always returning the outcome and letting callers decide whether to throw.

Acceptance criteria

  • waitForIpniProviderResults returns or exposes per-CID verified/failed lists.
  • Abort during setTimeout(delayMs) (currently signal-unaware at https://github.com/filecoin-project/filecoin-pin/blob/main/src/core/utils/validate-ipni-advertisement.ts#L387) is preserved or fixed in the same change. Optional related fix: make the inter-attempt sleep signal-aware so abort overshoot is bounded by the in-flight fetch rather than by delayMs.
  • Existing consumers that only need pass/fail can keep that behavior via a wrapper or a boolean accessor on the new return type.
  • Tests cover: timeout-mid-walk, partial verification (some CIDs verified, some failed), all-CIDs-missing, single-CID success.

Consumer signal

Open this when:

  • A real dealbot incident appears where timeout reason "some childBlock never indexed" needs to be distinguished from "rootCID never indexed."
  • Or filecoin-pin gains another consumer that needs per-CID outcomes for its own product semantics.

Activity

  1. added
    team/filecoin-pin"Filecoin Pin" project is a stakeholder for this work.
    team/fs-wgFOC working group is a stakeholder for this work, and thus wants to track it on their project board.
    on Apr 28, 2026
  2. moved this to 📌 Triage in FOCon Apr 28, 2026
  3. moved this to 📌 Triage in FilOzon Apr 28, 2026
  4. moved this from 📌 Triage to 🐱 Todo in FOCon Apr 29, 2026
  5. self-assigned this
    on May 1, 2026
  6. moved this from 🐱 Todo to ⌨️ In Progress in FOCon May 5, 2026
  7. moved this from ⌨️ In Progress to ⌚️ Issue awaiting PR merge in FOCon May 7, 2026
  8. added
    enhancementNew feature or request
    ready-for-workTriaged: scope, plan, and DoD are clear; contributor can pick up
    on May 19, 2026
  9. moved this from ⌚️ Issue awaiting PR merge to 🐱 Todo in FOCon Jun 29, 2026
  10. BigLep commented on Jul 10, 2026

    @BigLep
    Member

    Triage comment from 2026 FOC Board Triage (Post GA): fairly straightforward fix that will also be useful for dealbot. not high priority, but straightforward

  11. added a commit that references this issue on Jul 28, 2026
    6190879
  12. moved this from 🐱 Todo to 🔎 Awaiting review in FOCon Jul 29, 2026
  13. moved this from 🔎 Awaiting review to ⌚️ Issue awaiting PR merge in FOCon Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestready-for-workTriaged: scope, plan, and DoD are clear; contributor can pick upteam/filecoin-pin"Filecoin Pin" project is a stakeholder for this work.team/fs-wgFOC working group is a stakeholder for this work, and thus wants to track it on their project board.

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions