Skip to content

waitForIpniProviderResults: surface per-CID outcomes for diagnostic visibility #417

Description

@SgtPooki

Problem

waitForIpniProviderResults returns Promise<boolean> and on failure throws a single Error whose message references one CID (the last one checked). Consumers cannot tell, after the fact, which CIDs in [rootCid, ...childBlocks] were verified vs. timed out vs. missing-expected-provider vs. fetch-errored.

This matters most on the abort/timeout path: when an outer AbortSignal.timeout(...) fires mid-walk, the in-flight fetch throws AbortError and we lose which CID was current and which CIDs had already been verified successfully.

Concrete example from the dealbot consumer (FilOzone/dealbot):

  • Deals are uploaded with rootCID + ~12 childBlocks. Outer timeout 60s, delayMs 2500ms, so each CID gets ~25 attempts.
  • 7 of 8 SPs verify cleanly at p50 ~2.4s. SP6 (Hangzhou CN) verifies at p50 ~31s with 39% slow_verification rate; ~22 deals/24h time out at the 60s budget.
  • All timeouts surface as a single generic message. We currently cannot distinguish "rootCID never indexed" from "rootCID indexed but childBlock N never indexed."
  • Investigation: IPNI xxx root CID not verified FilOzone/dealbot#473 (comment)

Why this is currently low priority

  1. In observed dealbot data, all timeouts are wall-clock SLO breaches, not selective "missing expected provider" failures. Per-CID granularity would not have shifted root-cause attribution (path quality between filecoinpin.contact / Reston VA and SP6 / Hangzhou CN, AS4134 ChinaNet, ICMP packet loss 24% from a peer test vantage).
  2. At dealbot's current SLO budget (60s with 2.5s polls and 12 CIDs), only the first 4-5 CIDs get any attempts before the outer abort fires anyway — so child-CID granularity is not measurable end-to-end without also rebudgeting.
  3. PR fix(ipni): preserve failure reason in deal-flow rethrow FilOzone/dealbot#490 + feat(ipni): expose failureReason as top-level field on ipni_tracking_failed FilOzone/dealbot#491 close the immediate visibility gap that prompted dealbot#473 by carrying the existing single failureReason via Error.cause.

Proposal (when prioritized)

Change return type from Promise<boolean> to something like:

```ts
type IpniValidationOutcome = {
verified: Array<{ cid: CID; matchedAt: number; attempts: number }>;
failed: Array<{
cid: CID;
reason:
| { type: 'timeout'; attempts: number }
| { type: 'missingProviders'; missingServiceUrls: string[]; actualMultiaddrs: string[] }
| { type: 'fetch'; message: string }
| { type: 'parse'; message: string }
| { type: 'http'; status: number; statusText?: string };
}>;
ipniIndexerUrl: string;
};
```

Existing single-error throw can be preserved as a thin wrapper for callers that prefer the boolean shape, or replaced with always returning the outcome and letting callers decide whether to throw.

Acceptance criteria

  • waitForIpniProviderResults returns or exposes per-CID verified/failed lists.
  • Abort during setTimeout(delayMs) (currently signal-unaware at https://github.com/filecoin-project/filecoin-pin/blob/main/src/core/utils/validate-ipni-advertisement.ts#L387) is preserved or fixed in the same change. Optional related fix: make the inter-attempt sleep signal-aware so abort overshoot is bounded by the in-flight fetch rather than by delayMs.
  • Existing consumers that only need pass/fail can keep that behavior via a wrapper or a boolean accessor on the new return type.
  • Tests cover: timeout-mid-walk, partial verification (some CIDs verified, some failed), all-CIDs-missing, single-CID success.

Consumer signal

Open this when:

  • A real dealbot incident appears where timeout reason "some childBlock never indexed" needs to be distinguished from "rootCID never indexed."
  • Or filecoin-pin gains another consumer that needs per-CID outcomes for its own product semantics.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestready-for-workTriaged: scope, plan, and DoD are clear; contributor can pick upteam/filecoin-pin"Filecoin Pin" project is a stakeholder for this work.team/fs-wgFOC working group is a stakeholder for this work, and thus wants to track it on their project board.

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions