tachyon: make setup recover from unfamiliar GPTs and fix 24.04 onboarding

[mrlambchop]

What

Nine commits that make particle tachyon setup work on 24.04, and make
tachyon backup / tachyon restore layout-independent and verified.

commit	change
3acc9124	write the config blob to the post-flash misc; verify NV restores
da0ccaa0	log the real UFS LUN geometry read back from the device GPT
d8431859	identify a device whose layout has no boot_a/boot_b
e19bb122	offer the headless variant on 24.04, drop the beta label
9dccf641	fetch eSIM profiles during 24.04 setup, as 20.04 already does
54bd8d90	tell 24.04 users the modem will activate, as 20.04 does
ba0dea1f	print the durable backup path, not only its basename
7111fea6	make the outgoing GPT advisory for setup; EDL identity is sufficient
640ece58	resolve region/board by explicit input → device → cloud → prompt; remove silent NA/DVT defaults

No dependency changes. package.json is untouched — in particular this does not
add @particle/tachyon-image, which stays dormant until the A/B work lands.

Why

1. The config blob was written using the wrong GPT

createConfigBlobStep resolved the misc LBA from the pre-flash GPT and
flashOSAndConfigStep programmed that XML after the OS was written. On a 20.04 → 24.04
setup that means the blob went to the old misc offset — i.e. to whatever the new image
put on those sectors. On a device already running 24.04 it failed outright with
Partition misc not found in device partition table, because 24.04 had no misc at all
(fixed by tachyon-composer #76).

Reordered to:

1. create the blob (createBlobFile, unchanged);
2. preflight the image — verifyConfigPartitionStep reads the partition table out of
   the image about to be flashed and fails if it has no misc, if misc is smaller than
   256 sectors, or if the blob would not fit. This runs before the multi-GB write, so a
   mismatch costs seconds rather than a full flash;
3. flash the OS/GPT, keeping the device in EDL;
4. read the newly flashed live GPT off the device;
5. resolve misc by label and generate the QDL program XML.

flash() already ran flashTachyon then flashTachyonXml as two qdl invocations, so
step 4 slots between them. createConfigBlobStep now returns only { configBlobPath }.
Step 5 (resolveConfigPartition) is the authoritative gate: if misc is absent from the
post-flash GPT it fails with a message naming the image and stating that the OS was
installed but the configuration was not applied, rather than silently writing the blob to
whatever now occupies those sectors. keepInEdl = skipReset || Boolean(configBlobPath).

Reset behaviour is unchanged from master: flashTachyonXml({ skipReset }) is called
exactly as before. Only the internal name changed (shouldResetOS → keepInEdl).

2. restore()'s retry path called backup()

backup-restore-tachyon.js:151-153 caught a restore failure and retried with
this.backup({ 'input-dir': … }) — wrong method, and backup() does not take
input-dir. Now retries this.restore(...) with restore's own arguments.

Also: backups stored durably under the Particle data directory with the path always
printed; a backup.json sidecar (schema version, device ID, source LUN/LBA geometry,
byte lengths, SHA-256 per file); pre-write validation that each file fits its target
partition; and post-write read-back hash verification. Legacy archives without
backup.json are still accepted.

3. identify threw on any device without boot_a

partitionDefinitions threw Partition boot_a not found in device partition table for
any layout lacking it. 20.04 has boot_a/boot_b; 24.04 does not — so tachyon setup
died at "Unable to get device info." on a 24.04 board. Adds an optionalPartitions list
(threaded through prepareFlashFiles) so a missing optional partition yields null and
is filtered out instead of throwing, and uses the absence of both boot partitions
together with a single non-A/B system to report Ubuntu 24.04.

4. 24.04 offered no headless variant

The ubuntu24 workflow had a desktop variant only, so step 10 printed
undefinedLearn more about Tachyon… — a missing setupCompletedMessage. Adds the
headless variant with its own completion message, and drops (beta) from the 24.04
name plus the now-stale selectionWarning.

5. 24.04 setup never fetched the eSIM profiles

The ubuntu24 workflow was missing getCountryStep and getESIMProfilesStep.
ubuntu20 has both. So 24.04 setup never asked for a country, never called
PUT /v1/products/:productId/devices/:deviceId/target_profile, and wrote the config blob
with no esim key at all.

The consumer has been there the whole time — particle-linux's bootstrap declares

esim?: { profiles: Array<{ iccid, activation_code }>, target_profile: string | null }

and calls storeEsimBootstrap() whenever the blob carries it. 24.04 devices simply never
got one, so a device worked only if its eUICC happened to already be provisioned.

Both steps are OS-agnostic: getCountryStep prompts (or takes country from a loaded
config in silent mode), and getESIMProfilesStep is non-fatal — it warns and returns
{ esim: null } if the lookup fails. createBlobFile snapshots the whole context minus a
small exclusion list and drops nulls, so country and esim reach the blob with no extra
plumbing.

Ordering matters and is now asserted: the country must resolve before the profile lookup
(it is the countryCode argument), and both before the blob is built, since the blob is a
snapshot of the context.

6. 24.04's completion message did not mention the modem

Both 24.04 variants listed "Connect to the Particle Cloud" but not "Activate the built-in
5G modem"; 20.04 lists both. That was accurate while ubuntu24 had no
getESIMProfilesStep, and stopped being accurate the moment it gained one. Added to the
desktop and headless variants, matching ubuntu20's wording and punctuation.

Tied to behaviour rather than left as prose: a test asserts a workflow claims modem
activation if and only if its step list contains getESIMProfilesStep, across every
variant — so the copy and the capability cannot drift apart again the way they just did.

⚠️ Not verified end to end on hardware. head2's eUICC is already provisioned, so
re-running setup there does not exercise a device that arrives without a profile —
which is exactly the case this message now promises. Needs an unprovisioned unit before
release.

7. The outgoing GPT is no longer a setup precondition

Commits 7111fea6 and 640ece58 fix the underlying recovery contract rather than
adding another known layout to the allow-list.

Why setup read the old GPT in the first place

The original structure appears to have been a convenience: getTachyonInfo()
produced one object containing the EDL device ID plus values inferred from storage:

• region from a modem marker in fsg;
• EVT/DVT board type from the LUN containing nvdata1;
• installed OS from boot_a/boot_b;
• whether manufacturing data looked present.

Region and board then selected the matching image automatically, while the remaining
fields were printed to the user. That is useful metadata, but setup treated failure to
obtain any of it as proof that flashing could not work. The conclusion does not follow:
the EDL enumeration already supplies the identity used by setup, and the selected image
carries prog_firehose_ddr.elf, its rawprogram XMLs, patch XMLs, and Primary/Backup
GPT writes.

New behavior

The Particle Cloud lookup starts as soon as USB enumeration supplies the EDL device ID,
in parallel with the slower best-effort storage read. A blank, corrupt, or unfamiliar GPT
now emits a warning and setup continues. Region and board are resolved independently, in
this precedence:

1. an explicit command-line option;
2. a loaded setup configuration;
3. a value read from the physical device;
4. Particle Cloud metadata;
5. an interactive prompt.

The physical device wins over the cloud because the cloud record is only as fresh as its
last connection. Today the cloud supplies region from modem_firmware_version; it does
not report EVT/DVT board type, so an unreadable board still prompts unless the operator
provided --board or saved it in the setup configuration. The selected source is shown
in the device information. There is no silent NA/DVT fallback.

Effect on the onboarding flow

1. Login and EDL enumeration run as before; EDL supplies the device ID without reading storage.
2. The cloud lookup starts immediately from that ID while the existing GPT read runs. Cloud
   failure is non-fatal.
3. Setup resolves region and board separately using the precedence above. A source may answer
   one field without being trusted for the other.
4. The resolved values and any non-device provenance are printed before image selection. If a
   value remains unknown, setup asks the operator instead of guessing.
5. The normal variant, account, product, Wi-Fi, eSIM, image preflight, flash, post-flash GPT
   read, and misc configuration steps then continue unchanged.

A real TachyonConnectionError is deliberately different: setup still offers a retry
and stops if the user declines, because the device is no longer satisfying the one
hardware precondition -- being reachable in EDL.

Downsides and remaining constraints

• The cloud record may be stale, so a readable physical value deliberately wins. If the
  device cannot answer, the output labels cloud-derived values.
• A loaded config is now treated as explicit intent, including board, and overrides
  discovery. Reusing a config created for different hardware can therefore select the wrong
  image; explicit --region / --board remain the final override.
• When no source knows a value, interactive setup pauses for a region or board choice. This
  is intentional: guessing NA/DVT could install an image for the wrong cellular bands or
  hardware revision.
• Keeping identification as best-effort adds one firehose/read attempt before a recovery
  flash. Removing it entirely would be faster, but would regress automatic image
  selection for the normal readable-device case.
• EDL presence is the only required device state, not a guarantee that every image is
  compatible. The package must still contain a valid programmer/GPT and fit the target
  UFS geometry.
• The post-flash misc read remains intentional. It occurs only after the image has
  installed its own GPT and resolves where the configuration blob belongs. If that new
  image lacks a valid misc, the OS is installed but setup reports that configuration
  could not be applied.

The setup tests cover recognized and unsupported layouts, cloud lookup ordering, every
precedence tier, prompt fallback, loaded board configuration, connection failure, and
connection retry. Utility tests cover cloud firmware parsing, non-fatal lookup failure,
and refusing to classify an unfamiliar GPT as DVT.

Hardware regression: 24.04 single-slot to 20.04 A/B

Tested on head2, device 422a060000000000d0c7965f, using the exact command:

pi@head2-pi:~/images $ /opt/particle/bin/particle tachyon setup --version /home/pi/images/tachyon-ubuntu-20.04-NA-headless-formfactor_dvt-1.0.190.zip

Before installing the PR build, the old /opt binary stopped before any write:

Starting Process. See logs at: .../tachyon_flash_422a060000000000d0c7965f_1788376808082.log
Tachyon setup failed: Unable to get device info. Please restart the device and try again.

After installing the PR build at the same path, the command:

• passed identification from the outgoing 24.04 single-slot layout;
• preflighted the supplied image's misc;
• wrote the supplied 20.04 A/B GPT and OS;
• re-read that new GPT;
• programmed the configuration into misc;
• completed all 12 setup steps.

The booted board reports Ubuntu 20.04.6 LTS, kernel 5.4.219, root
/dev/sda11, Wi-Fi associated, and the Particle, GNSS, and RIL services active.
misc resolves to /dev/sda5, is exactly 1 MiB, and hashes as an all-zero 1 MiB
buffer after first boot, showing that bootstrap consumed and erased the configuration.

The older 20.04 image reports systemd degraded; observed failures across these boots
were persist.mount, sfsconfig.service, and thermal-engine.service. Those boot-time
image issues were recorded but are outside this CLI change.

Hardware verification of the final precedence resolver

The 087b8c8d ARM64 build exercised the final behavior at /opt/particle/bin/particle. The final 640ece58 adds only the numbered precedence comment in source; its exact ARM64 artifact was built and installed afterward. The same
20.04 package was run again, with the saved setup configuration streamed after removing
its region and board keys so discovery—not the config—had to resolve both fields.

The live output was:

 -  Device ID: 422a060000000000d0c7965f
 -  Board: DVT or later
 -  Region: NA (from Particle Cloud)
 -  OS Version: Ubuntu 20.04

This exercises the per-field merge rather than an all-or-nothing fallback: the physical
GPT supplied DVT, while the physical fsg partition was readable but contained no
SG560D-NA / SG560D-EM region marker, so the cloud filled only region. fsg itself
and its manufacturing-data EFS header are present; this is not modem-data damage.

Setup again completed all 12 steps. The board booted Ubuntu 20.04.6, associated to Wi-Fi,
and the Particle, GNSS, and RIL services are active. The post-flash misc is 1 MiB and
hashes as an all-zero buffer after bootstrap consumed the configuration.

Tests

1117 passing, 10 pending, lint clean. New coverage in
steps.test.js, tachyon-utils.test.js and backup-restore-tachyon.test.js for: the
GPT being read after the OS flash and the new misc LBA being used; an image with no
misc, or an undersized one, failing at preflight before any flash; a device with no
misc post-flash failing at the second gate; restore retrying restore; restore
rejecting a file that would overrun its target; read-back hash verification; legacy
archives with no backup.json; and partitionDefinitions tolerating absent optional
partitions.

Hardware

head2, against composer build 1.2.14-dev+build.64a0cda (tachyon-composer #76). The
board has been on 24.04 since 27 Aug, so the first two runs are 24.04 → 24.04 — which is the
case defect 1 broke outright. The 20.04 → 24.04 direction is not exercised here.

Run 1

• setup ran to step 10 with the headless completion message rendered, no undefined
• device boots to a login on Ubuntu 24.04.4 LTS, kernel 6.8.0-1058-particle
• misc -> sda3, exactly 1048576 bytes, and entirely zero after first boot — the blob
  was written to the correct post-flash offset, consumed by the particle-linux bootstrap
  and wiped
• the configured password authenticates (sha512crypt match for particle in /etc/shadow)
• Wi-Fi associated to the configured SSID and took a DHCP lease
• modem NV byte-identical to the pre-flash capture across all six partitions
  (modemst1 bf3b6f09…, modemst2 a00c952e…, fsc fa43239b…, fsg abccf416…,
  nvdata1/nvdata2 106f0647…), with nvdata1/2 still at the 20.04 1.5 MiB sizes

Run 2 — repeat setup, again on 24.04

Both runs exercise the case that used to fail outright with
Partition boot_a not found in device partition table → "Unable to get device info."
That failure is itself the evidence the board was on 24.04: the very first attempt in
this session died there, because 24.04 has no boot_a.

• Device info: now reports OS Version: Ubuntu 24.04 and setup continues
• Step 8 — "Checking that the operating system image can store your configuration…" —
  the new image preflight ran and passed before any flashing
• ran through to step 10, device boots to a login again on root=PARTLABEL=system / sda4
• misc again exactly 1 MiB and again entirely zero after first boot
• Wi-Fi reassociated, same DHCP lease
• modem identity intact — particle-tachyon-ril-ctl info reports IMEI-1 8644300100010 91,
  IMEI-2 …92, ICCID 8935711800001245077, EID 89044045…, firmware SG560DNAPAR60A03

fsg, fsc, nvdata1 and nvdata2 are byte-identical across both flashes.
modemst1/modemst2 differ between the two runs, which is expected and not
flash-related — they are the modem's live working EFS partitions. Verified directly:
with no flashing at all, modemst2 changed hash within 90 s of idle runtime
(f05c5fc1… → 99ff9042…). After the first flash they matched the pre-flash capture
exactly, which is what shows the flash itself does not touch them.

On-image suites (run 2 device, TMPDIR=/tmp)

suite	result	baseline
particle-linux-test	21 passed / 0 failed / 2 skipped	20 / 0 / 3
particle-tachyon-gnss-test	41 passed / 2 failed / 9 skipped	43 / 0 / 9
particle-tachyon-rild-test	74 passed / 3 failed	78 / 0 / 0

The failures are pre-existing image issues, not regressions from this PR — see the
composer PR for the reasoning. Note the suites must be run with a valid TMPDIR: under
adb shell, TMPDIR defaults to the Android path /data/local/tmp, which does not
exist on this image, and the suites' output redirects fail in a way that looks like a
CLI fault ('particlectl systemdoc' exited 2).

Landing note

Land after tachyon-composer #76 and after an image containing misc is published —
this PR's post-flash gate fails by design against an image that has no misc.

[Copilot]

Pull request overview

This PR fixes two defects in the Particle CLI's Tachyon setup and backup/restore flows. In setup, the first-boot configuration blob was being addressed from the pre-flash GPT and written after the OS was flashed, so on 20.04→24.04 flashes the blob landed on the wrong sectors (or failed outright on 24.04, which had no misc). The fix splits blob creation from address resolution: a new pre-flight step verifies the target image declares an adequate misc partition, and the address is now resolved from the newly-flashed live GPT. In backup/restore, the retry path mistakenly called backup() instead of restore(); the fix corrects the retry, adds a backup.json sidecar (schema version, device ID, geometry, sizes, SHA-256), performs up-front bounds/integrity checks and post-restore read-back verification, moves backups under the Particle data directory (with legacy-location fallback), and logs real UFS LUN geometry from the device GPT.

Changes:

• Setup: add verifyConfigPartitionStep (image preflight) and resolve misc from the post-flash GPT via resolveConfigPartition, keeping the device in EDL between OS and config writes.
• Add image/GPT parsing helpers (readPartitionsFromImage, parseRawProgramPartitions, readLunCapacities) and shared constants to tachyon-utils.js.
• Backup/restore: fix retry to call restore(), add backup.json metadata plus pre-write validation and post-write read-back verification, and store backups durably under the Particle data directory.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/lib/tachyon/workflow.js	Inserts verifyConfigPartitionStep into the ubuntu20/ubuntu24 workflows.
src/lib/tachyon/steps.js	Splits config creation, image preflight, and post-flash misc resolution; keeps device in EDL for the config write.
src/lib/tachyon/steps.test.js	New tests for verifyConfigPartitionStep and flashOSAndConfigStep ordering/failure messaging.
src/lib/tachyon-utils.js	Adds image/GPT partition parsing, LUN capacity reads/logging, and exported config-partition constants.
src/lib/tachyon-utils.test.js	New tests for rawprogram parsing, image partition reads, and LUN capacities.
src/cmd/backup-restore-tachyon.js	Fixes retry to restore(), adds metadata sidecar, up-front verification, read-back verification, and durable backup dir.
src/cmd/backup-restore-tachyon.test.js	New tests covering metadata, verification, and the retry fix.
src/cmd/tachyon-factory-restore.js	Aligns _backupDir with the backup command default and broadens hasBackups().
src/cli/tachyon.js	Updates backup/restore option help text for the new default directory.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[monkbroc]

Note that there's no headless variant for latest for 24.04

? Select the OS Type to setup in your device Ubuntu 24.04

===================================================================================

Step 1:

Select the variant of the Tachyon operating system to set up.
The 'desktop' includes a GUI and is best for interacting with the device with a keyboard, mouse, and display.
The 'headless' variant is accessed only by a terminal out of the box. 

? Select the OS variant: Headless (command-line only)
Tachyon setup failed: No builds found for this variant headless, board formfactor_dvt and version latest

[monkbroc]

Added DO-NOT-MERGE to make it clear that this needs to wait until the changes to Ubuntu 24.04 in Tachyon composer are finalized.