Skip to content

Commit 1da33cb

Browse files
authored
Release v0.34.0 (#304)
* Foundation for the staged-apply fix: a pmxcfs writer and a pvesh that answers like the real one First commit of the series designed in diagnostics/design-staged-apply-pmxcfs-2026-09-02.md (fable-check bugs 1-5, verified live on a PVE 9.1.9 node). pmxcfsWriteFile writes one file under /etc/pve behind the same mounted guard restore_ha.go and restore_access_control_ui.go already enforce (isMounted via a seam var, root seamed for tests): with pmxcfs not mounted a write would be a shadow file on the root filesystem that the real mount hides forever, so the helper refuses. No rollback capture, on the maintainer's call: the workflow's existing safety backup already archives everything these writes can touch. TDD: three tests written first (write-under-root, not-mounted refusal with no shadow file, mount-check error surfaced), red on the missing symbol, green on the implementation. schemaAwarePvesh is the test-side half: a CommandRunner that answers the way the live node answered, verbatim - "No 'set' handler defined for '/cluster/config'", "meta: property is not defined in schema", "storage ID 'local' already defined" - and stays permissive for the two pvesh callers the series does not touch. The exact-key FakeCommandRunner could only echo what a test taught it, which is how endpoints that do not exist stayed green for months. A self-test pins the fake to the live strings so nobody can un-teach it the rejections. Nothing wired yet: no production call site changes in this commit. * datacenter.cfg is restored by writing pmxcfs, not by calling an endpoint that does not exist Both rails - the staged arm (applyPVEDatacenterCfgFromStage) and the SAFE-apply UI arm (confirmAndApplyDatacenterCfg) - called `pvesh set /cluster/config -conf <file>`. A live PVE 9.1.9 node answers that with `No 'set' handler defined for '/cluster/config'` (probed 2026-09-02), so datacenter.cfg was NEVER restored on any non-recovery restore; the failure just landed in failedItems as a warning. fable-check bug 1. The API has no whole-file endpoint (options live per-key under /cluster/options), and /etc/pve IS pmxcfs: writing the staged file replicates it cluster-wide exactly the way the call would have. Both rails now go through pmxcfsWriteFile - mounted guard included - and pvesh is out of this arm entirely, together with its which-check. The UI confirmation still gates the SAFE rail. TDD, red first against the schema-aware fake, which reproduced the LIVE failure verbatim in a unit test for the first time: staged datacenter.cfg apply failed against the real pvesh surface: pvesh [set /cluster/config -conf /stage/etc/pve/datacenter.cfg] failed: No 'set' handler defined for '/cluster/config' Green: both rails land the staged bytes at <pmxcfs>/datacenter.cfg, zero /cluster/config calls, and the unmounted guard refuses with no shadow file. The two tests that PINNED the dead endpoint (pve_staged_apply_additional_test, restore_coverage_extra_test) now pin the file write instead. * vzdump.cron finally comes back from a staged restore Collected under pve_jobs, documented in RESTORE_GUIDE.md as staged-applied, and never read back by any restore path: legacy cron backup jobs silently vanished from every staged restore (fable-check bug 5; a repo-wide grep finds zero restore-side consumers before this commit). There is no pvesh endpoint for vzdump.cron. The file lives in pmxcfs, so applyPVEVzdumpCronFromStage writes the staged bytes through pmxcfsWriteFile - mounted guard included - exactly the way datacenter.cfg is now handled, and is wired into the pve_jobs branch beside jobs.cfg, gated off cluster RECOVERY like its sibling (config.db owns the file there). A failure lands in failedItems as "vzdump.cron", warning-weight like every other arm. TDD, red first (missing symbol, then the wiring pin): write-through test, absent/empty silent skips that leave pmxcfs untouched, unmounted-guard refusal, and a wiring test that fails if the arm ever becomes dead code again. This makes RESTORE_GUIDE.md:156 true instead of amending it downward. * storage.cfg apply updates existing definitions instead of failing on them applyStorageCfg only ever ran `pvesh create /storage`. `dir: local` exists on every node and in every backup, so every staged/SAFE storage apply on a live node failed on it - the live node answers "create storage failed: storage ID 'local' already defined" (probed 2026-09-02) - ended "with warnings", and an existing definition was never updated (fable-check bug 4). The proof the pattern was known sits twenty lines away: the backup-jobs apply has had create-then-set all along. On create failure the arm now retries as `pvesh set /storage/<id>` with the same args minus --storage and --type (create-only keys; `pvesh usage /storage/local` confirms the set handler exists). Both failing keeps today's warning, now carrying both causes. No file fallback, on the maintainer's call: per-block API preserves the definitions an operator added after the backup. TDD, red first against the schema-aware fake: a two-block storage.cfg (existing `local` + new `backup_ext`) scored applied=1 failed=1; green it scores 2/0, the fallback call is `set /storage/local` without create-only keys, and the new block still goes through create. A second test pins the both-fail arm at 0/1. * Guest configs apply through the API minus its blind spots, with the conf file as the net Two proven failures in one arm (fable-check bugs 2-3, probed live 2026-09-02 on PVE 9.1.9): - Every VM created on PVE >= 7.2 carries `meta:` in its conf, the qemu update schema has no --meta ("meta: property is not defined in schema"), and one rejected key fails the WHOLE `pvesh set`: no modern guest ever got its config applied. - A missing LXC could never be created: pveshCreateGuestArgs demanded an ostemplate that is create-time-only and never persists into a conf. The passing unit test hand-fed it - an input no real backup produces. applyVMConfigs now works in the shapes the maintainer chose: - Existing guest: `pvesh set` first, minus the create-only keys the schemas refuse (filterGuestCreateOnlyArgs: --meta, --ostemplate). If the set still fails, the staged conf is written byte-for-byte into /etc/pve/nodes/<node>/<dir>/<vmid>.conf - /etc/pve is pmxcfs, the write IS the cluster-wide apply - UNLESS status/current answers "running": a config race with a live guest is the one case where the file must not win, so the arm warns naming the guest and the reason and counts the failure. - Missing guest: no pvesh create at all; writing the conf into pmxcfs is how a guest comes into existence, and the log states it restores the CONFIG, not the disks. pveshCreateGuestArgs and its hand-fed test are deleted. TDD, red first against the schema-aware fake (extended with guests, running state and a failSet seam): meta-carrying guest scored 0/1, missing LXC 0/1, both green at 1/0 with the API attempt stripped of the refused keys and the file landing only when the API lost. The running-guest guard and the stopped-guest fallback each have their own pin. The one legacy test pinning create-then-set now pins register-via-pmxcfs. * The restore docs and prompts stop describing the endpoint that never existed Companion to the staged-apply series (fable-check bugs 1-5): four passages documented `pvesh set /cluster/config -conf <file>` as the datacenter apply - RESTORE_TECHNICAL.md's pseudocode, CLUSTER_RECOVERY.md's SAFE list, RESTORE_GUIDE.md's post-restore actions and its datacenter section. All four now describe what the code does since this series: datacenter.cfg written into pmxcfs (no whole-file endpoint exists), storage create with the set-fallback for existing ids, guests via set minus create-only keys with the conf file as the net, missing guests registered by writing the conf. The two datacenter prompts ("Apply datacenter.cfg via pvesh?") and the SAFE header line said "via pvesh" too; a prompt must not name a mechanism the code no longer uses, so the mechanism is simply gone from the words. Sample transcripts in RESTORE_GUIDE updated to match. Release notes 0.33.0 gain the two operator-facing lines: staged restore now really applies datacenter.cfg/vzdump.cron/existing storage, and guest configs survive on modern PVE. * The live node caught two more schema truths; the arms now handle both Ran the whole staged-apply matrix on a real PVE 9.1.9 node (livepve_test.go, build tag livepve: idempotent by construction - identical bytes, identical values, and the one thing it creates, config-only CT 990, it deletes again). First run: 3/5. The two failures were fresh live-only facts the exact-key fakes had always masked: - A missing guest never read as missing: pvesh prints "Configuration file '...' does not exist" on its OUTPUT while the Go error is a bare "exit status 2", so isPveshNotFoundError never matched on a real node and pveshGuestExists answered "check failed" instead of "absent" - also under the OLD create path, which was therefore unreachable live. pveshGuestExists now reads the output too (isPveshNotFoundText). - `path` is create-only for storage as well: the set fallback died with "Unknown option: path". The set schema varies by storage type and PVE version, so a whitelist would drift; pveshSetStorageDroppingCreateOnly instead drops exactly the key pvesh names and retries, converging or failing with the real cause. All settable keys already matching is a success, not a failure. The schema-aware fake learned both live shapes (reason-on-output with bare exit status; --path refused on storage set), which turned the two unit tests red first; both fixes green; full suite green. Second live run: 5/5 PASS - datacenter.cfg written to pmxcfs and byte-identical after the apply; vzdump.cron the same; storage 'local': create refused, --path dropped, set applied ("Updated existing storage definition local"); VM 101 (stopped, meta-carrying): full set applied minus --meta, meta line preserved in the conf; CT 990: registered by writing the conf, answers on the API, deleted after. One benign nuance recorded: the API path is not byte-idempotent - PVE canonicalizes storage.cfg content lists and re-setting an empty cdrom appended ide2 to VM 101's boot order (restored by hand). Fidelity-relevant only for byte-for-byte expectations, which the file-write arms already serve. * The staged-restore notes move to 0.34.0: 0.33.0 shipped while they were written The maintainer merged PR #303 and tagged v0.33.0 while the staged-apply series was in flight, so the release-notes lines this series had added to the 0.33.0 entry described a release that had already shipped without them - and the registry is append-only over FINAL versions. The 0.33.0 entry is restored byte-identical to the tag (zero deletions against v0.33.0), and the staged-restore story gets its own 0.34.0 entry: datacenter.cfg/vzdump.cron/ storage really applied, guest configs surviving modern PVE, missing guests registered from their config file, plus the one operator pointer at the restore log. * Cloud retention attribution is floored and the log delete speaks single-object rclone Two rclone faults from the fable-check sweep, both on the cloud cleanup path: resolveRetentionOwners ran one `rclone cat` per archive on the raw cancel-only run ctx and waited on ALL of them, so a single wedged cat hung the unattended cron/daemon run forever - while cloud.go's header promises the retention paths are floored. The whole attribution phase now runs under boundManagementCtx: archives left unresolved when the budget expires keep an empty Hostname and degrade to the filename token, the documented fallback. deleteAssociatedLog removed ONE file with the directory-oriented `delete`: a merely-absent log came back as "directory not found" on directory-listing backends, which the error branch misread as the whole CLOUD_LOG_PATH being gone (false WARNING, exit 1, logPathMissing poisoning cleanup for every remaining backup), and as exit 0 on prefix backends, counting a deletion that never happened. Both reproduced live with rclone v1.75.0. The verb is now `deletefile` and every not-found wording lands in the benign already-removed branch; the real path-missing detection stays with countLogFiles' lsf. * A crashed bundle's temp file no longer counts as a backup forever The bundle build stages <archive>.bundle.tar.tmp-<rand> in BACKUP_PATH (fs.CreateTemp with pattern "<base>.tmp-*"), with no leading dot - but isBackupTempArtifact only recognized a leading-dot .tmp- prefix. A crash during bundling therefore left a file every List counted as a backup: it matches the backup glob, triggers the missing-.metadata WARNING each pass (pinning exit 1), inflates counts, and nothing ever deletes it. The temp marker now matches anywhere in the base name. * Mount points own a path by boundary, not by string prefix getMountPoint matched /proc/mounts entries with a bare strings.HasPrefix, so with mounts / and /mnt/nas a BACKUP_PATH of /mnt/nas2 was attributed to /mnt/nas - and a dead /mnt/nas then turned into a critical StorageError aborting the backup of a perfectly healthy sibling path. The match now requires a path boundary (root, exact, or prefix + "/"), and the selection loop moved into bestMountPointFor so the rule is testable against synthetic mount tables. * A second --daemon refuses to start instead of burying the first There was no single-instance guard: a second hand-run daemon (the exact ExecStart line DAEMON.md invites operators to paste) overwrote .daemon.pid/.daemon_info.json with its own identity and, on exit, DELETED both, leaving the still-running unit daemon undiscoverable - standalone backup handoffs find no live daemon and outcomes go unpinged until a unit restart - and the daily backup was double-scheduled while both lived. run() now probes the recorded pid before publishing anything, with the same liveness + /proc cmdline gate the standalone handoff already trusts, and refuses to start when a live proxsave --daemon owns it, leaving the incumbent's files untouched. A stale file (dead pid, recycled pid belonging to something else) never blocks a restart, so kill -9 + systemd restart still boots. The refusal exits ExitBackupSkipped for the same reason the concurrent-backup skip does: nothing is wrong with the host, the work is already owned. * The in-progress lock speaks once: the bare pre-log drops to Debug All four benign-concurrency arms of CheckLockFile (three formatInProgress returns and the lost O_EXCL create race) logged the bare message at ERROR right before returning it - and the caller's logResult then rendered the same fact again as the one red "✗ Lock File (BACKUP_IN_PROGRESS)" line. One skip recapped errors=2 with two red lines saying the same thing. The ERROR level on the ✗ line is intended and stays; the bare pre-log stays too, for debugging, at Debug. * The tier closing line reports the worst thing the adapter saw After a non-critical Store failure with a clean retention pass, the closing branch read only hasWarnings and printed the green "✓ ... operations completed" at INFO - the log's last word on the backend contradicted the notification's "error" status and the two "Backup was not saved" warnings a few lines above. Reproduced end to end on the real secondary backend (read-only destination) and the real cloud backend (rclone failing copyto); the local arm never reaches this branch (IsCritical aborts the run first). The closing line now mirrors finalizeStorageStatus: errors outrank warnings, "✗ ... operations completed with errors" at WARNING. Deliberately not ERROR: the store failure itself is logged as recoverable and the exit contract stays warning-weight (maintainer call, 2026-09-02). Nothing pinned any of the three arms before; all three are pinned now. * Critical-file and custom-path copy failures finally speak, and an aborted recipe names its hole Since the initial Go port (#44), a REAL copy failure of /etc/passwd, /etc/shadow, /etc/sudoers, /etc/network/interfaces or a CUSTOM_BACKUP_PATHS entry logged only at Debug: WarningCount, exit code, notifications and healthchecks stayed green while the archive silently lacked the file. The same recipe already warned loudly for fstab, logrotate and mount units, so these were the mute odd ones out, not a doctrine - no commit ever touched those levels on purpose. The interfaces arm also lied: safeCopyFile answers a missing source with nil, so its err was always a real failure, yet the message said "No /etc/network/interfaces found". Real failures now warn naming the file and the cause; not-found stays silent by design. And when a brick error aborts a recipe, the tail used to vanish without a trace at any level (bricks that never ran record nothing, not even in the manifest): the abort now warns naming the recipe, the failed brick and how many bricks never ran, with a Debug line per skipped brick. Reproduced live before fixing: unreadable /etc/shadow on this host rendered nothing at INFO while FilesFailed counted 4; a read-only staging tree aborted the system recipe at the commands brick with everything after it missing untraced. * The prefilter touches only what it claims to touch: CRLF pairs in plain text normalizeTextFile did bytes.ReplaceAll(data, "\r", nil) on every small .txt/.log/.md/.conf/.cfg/.ini in the whole staging tree, CUSTOM_BACKUP_PATHS payload included, while CONFIGURATION.md sells the step as safe, semantic-preserving removal of \r from CRLF text files. Measured live before fixing: a UTF-16LE "hi\r\nyo" lost the 0x0D byte of its 0x0D 0x00 pair, went odd-length and decoded to garbage from that point on; a lone-\r progress log had its lines silently merged. The corruption lands in the staged copy, so it is the backup - and any restore of it - that carries the damage. The rewrite now collapses only \r\n pairs, and only in plain single-byte text: a UTF-16/UTF-32 BOM or any NUL byte (the cheapest reliable tell for wide encodings and binary payloads alike) leaves the file byte-identical. The existing pure-CRLF fixtures pass unchanged - CRLF-only input normalizes to the same bytes as before. * config.db is captured through sqlite3 .backup, not a raw read of a live WAL database The pmxcfs backing store - the artifact RECOVERY writes back verbatim - runs in WAL mode, and the capture was a plain file copy of the base file: torn pages on a mid-copy write, and on a standalone node (which skips the directory snapshot) every write since the last checkpoint simply absent. Measured on the live test node: base file 40KB, hours old; config.db-wal 4.1MB, current - the raw capture was missing most of the recent state even with no race at all. The capture now runs sqlite3 -cmd '.timeout 5000' <db> '.backup <target>': the online-backup API takes a shared lock and reads base+WAL as one coherent snapshot, verified live under the running pmxcfs on PVE 9.1.9 (integrity_check ok, WAL content included; sqlite3 ships on the node). On any failure - sqlite3 absent, database locked past the busy timeout - the old raw copy still runs, now behind a WARNING that names the torn-page and missing-WAL consequence. The snapshot lands after the clustered directory copy on the same target path, so the consistent artifact is the one that wins in both arms. * The parked bundle-pair double count is pinned as a characterization Bug 6 of the fable-check sweep is real but parked (maintainer call: the both-forms state needs a removeAssociatedFiles failure right after bundling and has never been seen in the wild). These tests keep the state measured instead of remembered: with BUNDLE_ASSOCIATED_FILES=true - the default everyone runs - the standalone+bundle pair collapses to one logical backup, which is exactly why no user has ever seen a double count; with it false, List still reports two entries for one archive while deleting either entry removes both forms, so simple retention keeps fewer real backups than MAX_LOCAL_BACKUPS promises. If either side changes, it now changes a failing test - consciously. A deliberate fix of bug 6 flips the OFF-side assertions. * The log-level threshold mutes the console, not the truth One gate sat before the counters, the issue capture and the file sink alike, so --log-level error made a warning vanish from every channel at once: HasWarnings painted the footer green, the shipped log had no line for the re-parse to find, and the run exited 0 - for a run that DID warn. Maintainer call (2026-09-02): the threshold is a CONSOLE filter. Warning-weight lines and above are always counted and always reach the log file - the artifact notifications ship keeps the evidence for the exit code - and only their display (console and mirror tap) is muted. Below warning the threshold keeps its full meaning everywhere. CLI_REFERENCE.md states the contract. * A notify failure exits 1 on every install, not only when Prometheus is on The post-notification re-parse (finalizeSuccessIssueStats) decides the PROCESS exit code - exportBackupMetrics runs as a defer inside RunGoBackup and mutates the same stats the cmd layer returns - yet it sat behind the shouldExportBackupMetrics gate: the same notify failure exited 1 with METRICS_ENABLED=true and 0 on a default install. Both halves of the contradiction shipped in the same release (90338af): NOTIFICATIONS.md promised the exit code frozen before any send, while the helper's comment and test promised promotion to 1. Maintainer call (2026-09-02): a notification failure is warning-weight ALWAYS. The re-parse now runs before the metrics gate on every successful non-dry run, and NOTIFICATIONS.md states the decided contract: warning-weight, exit 1, never an error code - monitoring learns notifications are broken exactly when email cannot say so. * The mount table answers with its last word, not its first getFilesystemType took the FIRST /proc/mounts entry for the mount point. Under a systemd automount the autofs placeholder precedes the triggered real filesystem at the same path - the live PVE test node shows the exact pair for binfmt_misc, and stacked-mount experiments there confirm the kernel appends in mount order, so the LAST entry is what a path actually resolves to. With the first entry winning, a backup path behind an automount parsed as autofs -> FilesystemUnknown: not a network filesystem, so the ownership write-probe never ran, and SetPermissions silently skipped chown/chmod 0600 on every stored backup. The lookup now keeps the last matching entry, extracted into lastMountEntryFor and pinned with the node's verbatim lines. * The 0.34.0 notes cover the whole fable-check batch The pending entry described only the staged-restore series; everything the sweep fixed since - collection failures that warn, the prefilter that no longer corrupts, the coherent config.db snapshot, the retention hang, the exit-code contracts, the daemon guard - lands in the same release and belongs on Screen 0. Eight lines, the gate's ceiling, so the most niche fixes (mount boundary details, closing-line wording) stay in the log and the release notes draft instead. * Ignore local Superpowers planning artifacts * Fail closed on uncertain PVE guest status * Guard PVE guest apply with cluster inventory * Harden live PVE guest safety checks * Confine pmxcfs writes to the verified mount * Make daemon ownership atomic * Honor dry-run for SQLite snapshots * Centralize advisory file locking * Reuse advisory locks across runtime services * Preserve NUL-free binary backup data * Keep valid tmp-named backups visible * Report optional network copy failures * Use live storage values in PVE test * Keep muted log records in the mirror * Cover failed-run notification reparse guard * Label pmxcfs restore confirmation accurately * Clarify notification exit-code release note * Align restore docs with payload-driven recovery * Clean up PVE guest fallback test * fix: detect interactive auto-confirmed upgrades by tty * fix: reopen dashboard with upgraded binary * test: cover unattended upgrade notes warning * fix: clarify trusted personal script paths * fix: serialize PVE guest config restores * fix: resolve dashboard lint failures * refactor: share daemon status diagnostics * refactor: share personal script inspection * feat: report personal script daemon diagnostics * docs: explain daemon script diagnostics * fix: bound daemon uid conversion * fix: normalize statfs block size * docs: clarify amd64 syscall assumption * fix: clear daemon diagnostics CI regressions * fix: execute inspected personal script path
1 parent b6765c7 commit 1da33cb

84 files changed

Lines changed: 5898 additions & 549 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@
77
# GSD workspace — local-only planning projects, never version-controlled
88
/gsd/
99

10+
# Superpowers design and planning artifacts — local-only, never version-controlled
11+
/docs/superpowers/
12+
1013
# Build artifacts — compiled binary (root build) and Makefile output dir
1114
/proxsave
1215
/build/

cmd/proxsave/daemon.go

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,11 +311,39 @@ func runDaemon(rt *appRuntime) int {
311311
}
312312

313313
func (d *daemon) run(ctx context.Context) int {
314+
ownershipDone := logging.DebugStart(d.logger, "daemon ownership", "base_dir=%s", d.cfg.BaseDir)
315+
releaseOwnership, err := acquireDaemonLock(d.cfg.BaseDir)
316+
if err != nil {
317+
if errors.Is(err, errDaemonLockHeld) {
318+
if pid, pidErr := health.ReadDaemonPID(d.cfg.BaseDir); pidErr == nil && pid > 0 {
319+
logging.Warning("daemon: another proxsave daemon is already running (pid=%d) - refusing a second instance so its pid file survives", pid)
320+
} else {
321+
logging.Warning("daemon: another proxsave daemon already owns %s - refusing a second instance", d.cfg.BaseDir)
322+
}
323+
ownershipDone(err)
324+
return types.ExitBackupSkipped.Int()
325+
}
326+
logging.Error("daemon: cannot establish single-instance ownership: %v", err)
327+
ownershipDone(err)
328+
return types.ExitGenericError.Int()
329+
}
330+
ownershipDone(nil)
331+
defer releaseOwnership()
332+
314333
// The trusted-path gate for the operator scripts, once, before any tick can start
315334
// one: a refused path is blanked here with a loud reason (validatePersonalScripts),
316335
// so the silent starters below never see it.
317336
validatePersonalScripts(d.cfg)
318337

338+
// The flock above is authoritative between new daemons. Keep this PID probe while
339+
// holding it for rolling compatibility with an older incumbent that publishes its
340+
// identity but does not take the lock. The same liveness + /proc cmdline gate the
341+
// standalone handoff trusts ensures a stale or recycled PID never blocks a restart.
342+
if pid, err := health.ReadDaemonPID(d.cfg.BaseDir); err == nil && pid > 0 && pid != os.Getpid() && daemonAliveProbe(pid) {
343+
logging.Warning("daemon: another proxsave daemon is already running (pid=%d) - refusing a second instance so its pid file survives", pid)
344+
return types.ExitBackupSkipped.Int()
345+
}
346+
319347
// CRITICAL: install the SIGUSR1 handler BEFORE publishing the pidfile below. Go's DEFAULT action
320348
// for SIGUSR1 is to TERMINATE the process, and the pidfile is exactly what a standalone backup
321349
// run uses to discover us and send SIGUSR1 to hand off its outcome. If the pid became

cmd/proxsave/daemon_diagnostics.go

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"fmt"
6+
"os"
7+
"strconv"
8+
"strings"
9+
"time"
10+
11+
"github.com/tis24dev/proxsave/internal/config"
12+
"github.com/tis24dev/proxsave/internal/health"
13+
"github.com/tis24dev/proxsave/internal/logging"
14+
"github.com/tis24dev/proxsave/internal/orchestrator"
15+
"github.com/tis24dev/proxsave/internal/ui/components"
16+
)
17+
18+
type daemonUIDDiagnostic struct {
19+
Value int
20+
Source string
21+
FallbackReason string
22+
}
23+
24+
// daemonDiagnostics is the single presentation-neutral snapshot consumed by
25+
// both the plain CLI and the dashboard daemon-status renderers.
26+
type daemonDiagnostics struct {
27+
Mode string
28+
Unit string
29+
Active string
30+
State health.DaemonState
31+
Level orchestrator.HealthcheckSetupLevel
32+
Keyword string
33+
Explanation string
34+
DaemonUID daemonUIDDiagnostic
35+
Scripts personalScriptsDiagnostics
36+
}
37+
38+
var (
39+
daemonDiagnosticsCollector = collectDaemonDiagnostics
40+
daemonStatusUnitInstalledProbe = daemonUnitInstalled
41+
daemonStatusActiveStateProbe = daemonUnitActiveState
42+
daemonStatusNow = time.Now
43+
daemonProcStatusReadFile = os.ReadFile
44+
daemonCurrentEUID = os.Geteuid
45+
daemonUIDResolver = resolveDaemonEffectiveUID
46+
personalScriptsInspector = inspectPersonalScripts
47+
)
48+
49+
// collectDaemonDiagnostics owns every probe and verdict used by the two
50+
// daemon-status frontends. Renderers receive facts; they never recompute them.
51+
func collectDaemonDiagnostics(ctx context.Context, cfg *config.Config, baseDir string) daemonDiagnostics {
52+
mode := "unknown"
53+
var interval time.Duration
54+
if cfg != nil {
55+
mode = cfg.SchedulerMode
56+
interval = cfg.HealthcheckHeartbeatInterval
57+
if configured := strings.TrimSpace(cfg.BaseDir); configured != "" {
58+
baseDir = configured
59+
}
60+
}
61+
if strings.TrimSpace(baseDir) == "" {
62+
baseDir, _ = detectedBaseDirOrFallback()
63+
}
64+
65+
unit := "not installed"
66+
if daemonStatusUnitInstalledProbe() {
67+
unit = "installed"
68+
}
69+
active := daemonStatusActiveStateProbe(ctx)
70+
if active == "" {
71+
active = "unknown"
72+
}
73+
74+
state := health.CheckDaemonState(health.DaemonStateInput{
75+
BaseDir: baseDir,
76+
SchedulerMode: mode,
77+
HeartbeatInterval: interval,
78+
Now: daemonStatusNow(),
79+
Presence: daemonPresenceProbe(ctx),
80+
ProcAlive: probeProxsaveDaemonAlive,
81+
ProcStale: procBinaryStaleProbe,
82+
})
83+
level, keyword, explanation := daemonStatusStyle(state)
84+
daemonUID := daemonUIDResolver(state)
85+
scripts := personalScriptsInspector(cfg, daemonUID.Value)
86+
return daemonDiagnostics{
87+
Mode: mode,
88+
Unit: unit,
89+
Active: active,
90+
State: state,
91+
Level: level,
92+
Keyword: keyword,
93+
Explanation: explanation,
94+
DaemonUID: daemonUID,
95+
Scripts: scripts,
96+
}
97+
}
98+
99+
// parseProcEffectiveUID reads the second numeric value from Linux's Uid line:
100+
// real, effective, saved-set, then filesystem UID.
101+
func parseProcEffectiveUID(data []byte) (int, error) {
102+
for _, line := range strings.Split(string(data), "\n") {
103+
fields := strings.Fields(line)
104+
if len(fields) == 0 || fields[0] != "Uid:" {
105+
continue
106+
}
107+
if len(fields) < 3 {
108+
return 0, fmt.Errorf("malformed Uid line: %q", line)
109+
}
110+
uid, err := strconv.Atoi(fields[2])
111+
if err != nil {
112+
return 0, fmt.Errorf("parse effective uid %q: %w", fields[2], err)
113+
}
114+
if uid < 0 {
115+
return 0, fmt.Errorf("effective uid %q must not be negative", fields[2])
116+
}
117+
return uid, nil
118+
}
119+
return 0, fmt.Errorf("uid line not found")
120+
}
121+
122+
func readProcessEffectiveUID(pid int) (int, error) {
123+
if pid <= 0 {
124+
return 0, fmt.Errorf("invalid daemon pid %d", pid)
125+
}
126+
path := fmt.Sprintf("/proc/%d/status", pid)
127+
data, err := daemonProcStatusReadFile(path)
128+
if err != nil {
129+
return 0, fmt.Errorf("read %s: %w", path, err)
130+
}
131+
uid, err := parseProcEffectiveUID(data)
132+
if err != nil {
133+
return 0, fmt.Errorf("parse %s: %w", path, err)
134+
}
135+
return uid, nil
136+
}
137+
138+
func resolveDaemonEffectiveUID(state health.DaemonState) daemonUIDDiagnostic {
139+
if state.ProcessAlive && state.PID > 0 {
140+
uid, err := readProcessEffectiveUID(state.PID)
141+
if err == nil {
142+
return daemonUIDDiagnostic{Value: uid, Source: "running daemon /proc"}
143+
}
144+
return daemonUIDDiagnostic{
145+
Value: daemonCurrentEUID(),
146+
Source: "current process",
147+
FallbackReason: err.Error(),
148+
}
149+
}
150+
return daemonUIDDiagnostic{
151+
Value: daemonCurrentEUID(),
152+
Source: "current process",
153+
FallbackReason: "daemon process is not live or has no PID",
154+
}
155+
}
156+
157+
// logDaemonDiagnostics is the plain CLI renderer for the shared snapshot. The
158+
// entire visible block is bracketed by the standard debug start/end markers.
159+
func logDaemonDiagnostics(logger *logging.Logger, diagnostics daemonDiagnostics) {
160+
if logger == nil {
161+
logger = logging.GetDefaultLogger()
162+
}
163+
done := logging.DebugStart(logger, "daemon diagnostics", "daemon_uid=%d uid_source=%s",
164+
diagnostics.DaemonUID.Value, daemonDiagnosticText(diagnostics.DaemonUID.Source))
165+
defer done(nil)
166+
167+
logger.Info("Daemon status: %s", daemonDiagnosticText(diagnostics.Keyword))
168+
logger.Info("Scheduler mode: %s", daemonDiagnosticText(diagnostics.Mode))
169+
logger.Info("Daemon service (%s): %s", daemonUnitName, daemonDiagnosticText(diagnostics.Unit))
170+
logger.Info("Service state (systemctl is-active): %s", daemonDiagnosticText(diagnostics.Active))
171+
if diagnostics.State.HaveInfo {
172+
logger.Info("Running version: %s (%s)", daemonDiagnosticText(diagnostics.State.Version), daemonDiagnosticText(diagnostics.State.Commit))
173+
}
174+
if diagnostics.State.HaveInfo || diagnostics.State.AlignChecked {
175+
alignment := "unknown"
176+
if diagnostics.State.AlignChecked {
177+
if diagnostics.State.Aligned {
178+
alignment = "aligned"
179+
} else {
180+
alignment = "BEHIND (restart needed)"
181+
}
182+
}
183+
logger.Info("Binary alignment: %s", alignment)
184+
}
185+
186+
logPersonalScriptDiagnostic(logger, "Personal pre-run script", diagnostics.Scripts.Pre)
187+
logPersonalScriptDiagnostic(logger, "Personal post-run script", diagnostics.Scripts.Post)
188+
logging.DebugStep(logger, "daemon diagnostics", "daemon uid: value=%d source=%q fallback_reason=%q",
189+
diagnostics.DaemonUID.Value,
190+
daemonDiagnosticText(diagnostics.DaemonUID.Source),
191+
daemonDiagnosticText(diagnostics.DaemonUID.FallbackReason))
192+
logPersonalScriptEvidence(logger, "pre-run", diagnostics.Scripts.Pre)
193+
logPersonalScriptEvidence(logger, "post-run", diagnostics.Scripts.Post)
194+
}
195+
196+
func logPersonalScriptDiagnostic(logger *logging.Logger, label string, diagnostic personalScriptDiagnostic) {
197+
path := daemonDiagnosticText(diagnostic.Path)
198+
reason := daemonDiagnosticText(diagnostic.Reason)
199+
switch diagnostic.State {
200+
case personalScriptReady:
201+
logger.Info("%s: READY (%s)", label, path)
202+
case personalScriptRefused:
203+
if path == "" {
204+
logger.Warning("%s: REFUSED: %s", label, reason)
205+
return
206+
}
207+
logger.Warning("%s: REFUSED (%s): %s", label, path, reason)
208+
default:
209+
logger.Info("%s: NOT CONFIGURED", label)
210+
}
211+
}
212+
213+
func logPersonalScriptEvidence(logger *logging.Logger, label string, diagnostic personalScriptDiagnostic) {
214+
logging.DebugStep(logger, "daemon diagnostics", "%s: key=%q state=%s path=%q daemon_uid=%d",
215+
label,
216+
daemonDiagnosticText(diagnostic.Key),
217+
diagnostic.State,
218+
daemonDiagnosticText(diagnostic.Path),
219+
diagnostic.DaemonUID)
220+
for _, component := range diagnostic.Components {
221+
logging.DebugStep(logger, "daemon diagnostics", "%s component: path=%q uid=%d mode=%04o",
222+
label, daemonDiagnosticText(component.Path), component.UID, component.Mode.Perm())
223+
}
224+
}
225+
226+
func daemonDiagnosticText(value string) string {
227+
return strings.TrimSpace(components.SanitizeText(value))
228+
}
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
package main
2+
3+
import (
4+
"context"
5+
"testing"
6+
"time"
7+
8+
"github.com/tis24dev/proxsave/internal/config"
9+
"github.com/tis24dev/proxsave/internal/health"
10+
"github.com/tis24dev/proxsave/internal/logging"
11+
"github.com/tis24dev/proxsave/internal/types"
12+
)
13+
14+
func TestCollectDaemonDiagnosticsBuildsOneSharedSnapshot(t *testing.T) {
15+
origInstalled := daemonStatusUnitInstalledProbe
16+
origActive := daemonStatusActiveStateProbe
17+
origPresence := daemonPresenceProbe
18+
origNow := daemonStatusNow
19+
t.Cleanup(func() {
20+
daemonStatusUnitInstalledProbe = origInstalled
21+
daemonStatusActiveStateProbe = origActive
22+
daemonPresenceProbe = origPresence
23+
daemonStatusNow = origNow
24+
})
25+
26+
daemonStatusUnitInstalledProbe = func() bool { return true }
27+
daemonStatusActiveStateProbe = func(context.Context) string { return "active" }
28+
daemonPresenceProbe = func(context.Context) health.DaemonPresence {
29+
return health.DaemonPresence{Probed: true, Installed: true, Active: true}
30+
}
31+
daemonStatusNow = func() time.Time { return time.Unix(1_700_000_000, 0) }
32+
33+
cfg := &config.Config{
34+
SchedulerMode: "daemon",
35+
BaseDir: t.TempDir(),
36+
HealthcheckHeartbeatInterval: time.Minute,
37+
}
38+
got := collectDaemonDiagnostics(context.Background(), cfg, cfg.BaseDir)
39+
40+
if got.Mode != "daemon" || got.Unit != "installed" || got.Active != "active" {
41+
t.Fatalf("incomplete shared snapshot: %+v", got)
42+
}
43+
if !got.State.Probed || !got.State.Installed || !got.State.Active {
44+
t.Fatalf("health state did not use the shared presence probe: %+v", got.State)
45+
}
46+
if got.Keyword == "" || got.Explanation == "" {
47+
t.Fatalf("shared snapshot is missing its verdict: %+v", got)
48+
}
49+
}
50+
51+
func TestRunDaemonStatusUsesSharedDiagnosticsCollector(t *testing.T) {
52+
origCollector := daemonDiagnosticsCollector
53+
t.Cleanup(func() { daemonDiagnosticsCollector = origCollector })
54+
55+
called := 0
56+
daemonDiagnosticsCollector = func(context.Context, *config.Config, string) daemonDiagnostics {
57+
called++
58+
return daemonDiagnostics{
59+
Mode: "daemon",
60+
Unit: "installed",
61+
Active: "active",
62+
State: health.DaemonState{HaveInfo: true, Version: "1.2.3", Commit: "abc", AlignChecked: true, Aligned: true},
63+
Level: 1,
64+
Keyword: "running",
65+
}
66+
}
67+
68+
origLogger := logging.GetDefaultLogger()
69+
logger := logging.New(types.LogLevelDebug, false)
70+
logging.SetDefaultLogger(logger)
71+
t.Cleanup(func() { logging.SetDefaultLogger(origLogger) })
72+
73+
rt := &appRuntime{
74+
ctx: context.Background(),
75+
cfg: &config.Config{SchedulerMode: "daemon", BaseDir: t.TempDir()},
76+
logger: logger,
77+
}
78+
if code := runDaemonStatus(rt); code != 0 {
79+
t.Fatalf("runDaemonStatus exit = %d, want 0", code)
80+
}
81+
if called != 1 {
82+
t.Fatalf("shared collector calls = %d, want 1", called)
83+
}
84+
}

cmd/proxsave/daemon_lock.go

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
package main
2+
3+
import (
4+
"errors"
5+
"fmt"
6+
"os"
7+
"path/filepath"
8+
9+
"github.com/tis24dev/proxsave/internal/filelock"
10+
)
11+
12+
const daemonLockFileName = ".daemon.lock"
13+
14+
var errDaemonLockHeld = errors.New("daemon ownership lock is already held")
15+
16+
func acquireDaemonLock(baseDir string) (release func(), err error) {
17+
dir := filepath.Join(baseDir, "identity")
18+
if err := os.MkdirAll(dir, 0o750); err != nil {
19+
return nil, fmt.Errorf("create daemon lock directory %s: %w", dir, err)
20+
}
21+
path := filepath.Join(dir, daemonLockFileName)
22+
releaseLock, err := filelock.TryAcquire(path)
23+
if errors.Is(err, filelock.ErrHeld) {
24+
return nil, fmt.Errorf("%w: %s", errDaemonLockHeld, path)
25+
}
26+
if err != nil {
27+
return nil, fmt.Errorf("acquire daemon lock %s: %w", path, err)
28+
}
29+
return func() {
30+
_ = releaseLock()
31+
}, nil
32+
}

0 commit comments

Comments
 (0)