Skip to content

fix: resolve @bitgo/* siblings into npm-shrinkwrap.json - #9404

Open
lokesh-bitgo wants to merge 1 commit into
masterfrom
WCI-1200-fix-bitgo-module-not-found
Open

fix: resolve @bitgo/* siblings into npm-shrinkwrap.json#9404
lokesh-bitgo wants to merge 1 commit into
masterfrom
WCI-1200-fix-bitgo-module-not-found

Conversation

@lokesh-bitgo

Copy link
Copy Markdown
Contributor

Ticket: WCI-1200

Issue

Reported : running npm install bitgo (the bundled all-coins SDK
install, not modular) reports a successful install, but none of the @bitgo/*
sibling packages (@bitgo/sdk-api, @bitgo/sdk-core, all @bitgo/sdk-coin-*,
etc. — 89 packages total) land in node_modules. require('bitgo') then fails
immediately with:

Error: Cannot find module '@bitgo/sdk-core'
code: 'MODULE_NOT_FOUND'

Modular install (@bitgo/sdk-api + @bitgo/sdk-coin-{name} + register())
is unaffected — this only breaks the bundled bitgo package install path.

Confirmed reproduction

Installed the real, published bitgo@52.4.1 from the real npm registry in an
empty directory:

  • node_modules/@bitgo/ does not exist
  • require('bitgo') throws MODULE_NOT_FOUND: Cannot find module '@bitgo/sdk-core'

Root cause

scripts/generate-bitgo-shrinkwrap.ts runs at pack time (prepack for the
bitgo module, gated by BITGO_GENERATE_SHRINKWRAP=true in the release
workflow) and generates the npm-shrinkwrap.json shipped inside the published
bitgo tarball. The previous implementation:

  1. Built an isolated temp package.json with all @bitgo/* sibling
    dependencies filtered out, to avoid requiring sibling versions that
    might not yet be published to the registry at release time.

  2. Ran npm install --package-lock-only + npm shrinkwrap against that
    stripped manifest. Since siblings were never in the manifest, npm never
    resolved them — no packages["node_modules/@bitgo/<name>"] entries were
    generated for them.

  3. "Restored" the siblings afterward, but only into:

    • top-level shrinkwrap.dependencies, and
    • shrinkwrap.packages[''].dependencies

    Both of these are just metadata ("the root package declares a
    dependency on X") — not install instructions. The actual install plan npm
    uses to populate node_modules in an npm v7+ lockfile lives in the
    packages map keyed by node_modules/<name> paths, and no such entries
    were ever created for the 89 siblings.

On the real npmjs.com registry, published versions carry _hasShrinkwrap: true
in their packument metadata. When that flag is set, npm treats the bundled
npm-shrinkwrap.json as strictly authoritative for that subtree — any
dependency listed only in packages[''].dependencies without a matching
resolved packages[...] entry is silently dropped from the install rather
than falling back to normal resolution. That's the exact mechanism causing the
reported MODULE_NOT_FOUND.

(Confirmed locally: local tarball installs and a local Verdaccio registry do
not reproduce this, because neither sets _hasShrinkwrap: true — only the
real npmjs.com registry does. This matches an explicitly unresolved
open question already flagged in the original ticket, WCN-604, when the
shrinkwrap generation was first introduced.)

Origin

Introduced in commit 8885c03a87 "fix(bitgo): generate npm-shrinkwrap.json at
pack time" (WCN-604 — "Ship npm-shrinkwrap.json + overrides to fix npm audit
gap for consumers"). That ticket's goal was legitimate and security-driven:
yarn resolutions pin safe transitive dependency versions internally, but npm
consumers never got those pins, so npm audit on the published bitgo
package showed 51 vulnerabilities (3 critical, 13 high) that internal
yarn-based CI didn't catch. This fix is a side effect of the workaround chosen
for "siblings aren't published yet at pack time" — it never accounted for
npm's lockfile requiring full packages entries for every listed dependency.

Fix

Minimal, targeted change to generate-bitgo-shrinkwrap.ts:

  1. Stopped stripping @bitgo/* siblings from the temp package.json before
    resolution. They now resolve as part of the same npm install --package-lock-only + npm shrinkwrap run as every other dependency,
    against the real registry. lerna publish publishes packages in
    dependency-topological order, so by the time bitgo (which depends on
    every sibling) is packed, the sibling versions it references are already
    live on the registry.
  2. Removed the broken "restore" step that only patched sibling names into
    dependencies / packages[''].dependencies metadata — no longer needed
    since nothing is stripped anymore.
  3. Added a fail-loud safety check: after generation, verifies every
    @bitgo/* sibling has a real packages["node_modules/@bitgo/<name>"]
    entry in the resulting shrinkwrap. If any are missing, the script throws
    and the release fails — correct behavior, since a failed release beats a
    silently broken shrinkwrap shipped to every consumer.

Validation performed

  • Ran the fixed script live against the real npm registry:
    npm-shrinkwrap.json generated successfully, all 89 @bitgo/* siblings
    received full version / resolved / integrity / dependencies entries,
    and the new safety check passed with zero unresolved siblings.
  • Confirmed the safety check actually fires: set a sibling to a nonexistent
    version (999.999.999) and reran — script failed with exit code 1
    instead of silently producing a broken shrinkwrap.
  • End-to-end reproduction against the real published package: swapped the
    newly-generated (fixed) shrinkwrap into a locally-unpacked copy of the real
    published bitgo@52.4.1 tarball, ran npm install, and confirmed:
    • All 108 @bitgo/* packages installed (up from 0 before the fix)
    • require('@bitgo/sdk-core') and require('@bitgo/sdk-api') resolve and
      load successfully
    • The original MODULE_NOT_FOUND: Cannot find module '@bitgo/sdk-core'
      error is completely gone
  • No new TypeScript or lint errors introduced (only pre-existing
    no-console / no-sync style warnings remain, unchanged in kind and count
    from before this change).

Known unrelated issue found during testing

After this fix, require('bitgo') against the real published tarball
progresses much further but then hits an unrelated, pre-existing error deep in
a transitive dependency:

Error [ERR_REQUIRE_ESM]: require() of ES Module .../rpc-websockets/node_modules/uuid/dist-node/index.js
from .../rpc-websockets/dist/index.cjs not supported.

This is a separate ESM/CJS packaging incompatibility in rpc-websockets's own
dependency on uuid, unrelated to shrinkwrap generation. Worth a follow-up
ticket; out of scope for this PR.

References

  • Original ticket: WCN-604 — Ship npm-shrinkwrap.json + overrides to fix npm
    audit gap for consumers
  • Bug ticket: WCI-1200 — npm install bitgo fails with MODULE_NOT_FOUND
  • Root cause commit: 8885c03a87 "fix(bitgo): generate npm-shrinkwrap.json at
    pack time"
  • Follow-up commit: cb0124ab34 "fix(scripts): skip overrides that are
    already direct bitgo deps"
  • File changed: scripts/generate-bitgo-shrinkwrap.ts

@linear-code

linear-code Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

WCI-1200

@lokesh-bitgo lokesh-bitgo self-assigned this Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants