[Bug]: container cp fails for every container created before a package upgrade (stale per-container vminitd); upload direction fails silently with exit 0

[sembsa]

I have done the following

• I have searched the existing issues

Steps to reproduce

1. On an older container release, create and run a container:

   container run -d --name demo alpine:latest sleep 3600
   

2. Upgrade container to 1.4.1 (.pkg installer), then restart the service so
   the CLI and apiserver are both 1.4.1:

   container system stop && container system start
   container system status | grep -E 'client.version|server.version'
   # client.version 1.4.1 / server.version 1.4.1
   

3. Start the pre-existing container again and try to copy anything out of it:

   container start demo
   container cp demo:/etc/hostname ./hostname
   

Observed:

Error: failed to copy from container demo (cause: "internalError: "failed to copy
from container demo (cause: "unknown: "notFound: "copy: path not found
'/etc/hostname'""")"")

The file is plainly there — container exec demo ls -l /etc/hostname prints it,
and container exec demo tar -cf - -C /etc hostname streams it out fine. It is
not path-specific: every path fails, including /, /etc, /tmp/<file just created via exec>.

4. Now create a new container on 1.4.1 and repeat — it copies normally:

   container run -d --name fresh alpine:latest sleep 300
   container cp fresh:/etc/hostname ./hostname   # works
   

So the discriminator is when the container was created, not its image, and
not the CLI/apiserver versions.

Problem description

container cp is broken for every container that was created before the
upgrade, and keeps being broken after the service is restarted and both sides
report 1.4.1. Recreating the container is the only thing that fixes it.

The error originates in the guest. The container's own
vminitd.log names it:

error vminitd: direction: copyOut, error: notFound: "copy: path not found
'/etc/hostname'", path: /etc/hostname copy failed

(~/Library/Application Support/com.apple.container/containers/<id>/vminitd.log)

Likely cause. Each container has its own initfs.ext4 in that same
directory, written when the container is created (getInitBlock fetches
containerSystemConfig.vminit.image during creation). Upgrading the package
does not refresh it, so a pre-existing container keeps booting the older
vminitd while the host side is 1.4.1. The mtimes line up exactly:

initfs.ext4 of an affected container: 2026-09-04 20:32
/usr/local/bin/container (1.4.1):     2026-09-09 03:41
initfs.ext4 of a container created
after the upgrade:                    2026-09-11 11:54   # copies fine

On this machine 14 of 15 existing containers are affected.

The upload direction is worse, because it is silent. Copying into an
affected container exits 0 and echoes the destination, while writing nothing:

$ container cp ./probe.txt demo:/tmp/probe.txt
demo:/tmp/probe.txt
$ echo $?
0
$ container exec demo test -e /tmp/probe.txt; echo $?
1

Anything scripting container cp — or any GUI wrapping it — reports success to
the user for a copy that never happened. Even if refreshing the guest agent on
upgrade is not feasible, it would help a lot if a failed copyIn returned a
non-zero exit status, and if a guest/host mismatch produced an error that says
so rather than path not found for files that exist.

Environment

• OS: macOS 26.6.2 (25G83)
• Xcode: not used (installed via the .pkg release)
• Container: container CLI version 1.4.1 (build: release, commit: 9a8917c), apiserver 1.4.1 (same commit)

Code of Conduct

• I agree to follow this project's Code of Conduct