I have done the following
Steps to reproduce
The defect is in the source, so the shortest reproduction is to hand kubeadm inside a 1.34 node image the config the plugin generates for it (initConfigYAML in Sources/ContainerK8s/Support/K8sHelper+Bootstrap.swift, whose kubernetesVersion comes out as v1.35.5 for any --node-image):
container run --rm --entrypoint /bin/bash docker.io/kindest/node:v1.34.11 -c '
cat > /tmp/kubeadm-config.yaml <<EOF
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.64.9
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.35.5
networking:
podSubnet: 10.244.0.0/16
EOF
kubeadm version -o short
kubeadm config images list --config /tmp/kubeadm-config.yaml
kubeadm init phase preflight --config /tmp/kubeadm-config.yaml 2>&1 | grep KubernetesVersion'
Output:
v1.34.11
registry.k8s.io/kube-apiserver:v1.35.5
registry.k8s.io/kube-controller-manager:v1.35.5
registry.k8s.io/kube-scheduler:v1.35.5
registry.k8s.io/kube-proxy:v1.35.5
registry.k8s.io/coredns/coredns:v1.12.1
registry.k8s.io/pause:3.10.1
registry.k8s.io/etcd:3.6.5-0
[WARNING KubernetesVersion]: Kubernetes version is greater than kubeadm version. Please consider to upgrade kubeadm. Kubernetes version: 1.35.5. Kubeadm version: 1.34.x
I could not run container k8s create --node-image docker.io/kindest/node:v1.34.11 to the end on the shipped 1.3.1 because node prep stops earlier (#2120), so the end state below is expected rather than observed.
Problem description
kubernetesVersion() in K8sHelper+Bootstrap.swift takes no argument: it parses the tag out of the static K8sHelper.nodeImage constant (docker.io/kindest/node:v1.35.5@sha256:...). The --node-image value flows to LinuxNodeProvisioner, which pulls and boots it, but never to bootstrapControlPlane. So every cluster's kubeadm config says kubernetesVersion: v1.35.5.
kubeadm treats a newer control-plane version as a warning, and the image list above is what it would pull, so a user who asks for a 1.34 node would be expected to get a 1.35.5 control plane on a 1.34 kubelet. For older images the gap grows until it exceeds the documented n-3 skew (kubelet may be up to three minor versions older than kube-apiserver): a v1.32 node lands exactly at that limit, v1.31 one minor past it. Below 1.31 kubeadm rejects the v1beta4 config outright (experimental API spec: "kubeadm.k8s.io/v1beta4" is not allowed, checked in kindest/node:v1.30.8; v1.31.14 accepts it), so 1.31 is the effective floor for --node-image regardless of this bug.
Expected: the kubeadm config carries the version of the image actually booted. The simplest fix is to omit kubernetesVersion entirely: kubeadm then resolves it itself, to the latest patch of its own minor when it can reach dl.k8s.io and to its own exact version when it cannot (both checked inside kindest/node:v1.34.0, which resolved to v1.34.11 online and to v1.34.0 with the fetch blocked, warning only). Parsing the resolved image tag also works but kubeadm config validate hard-errors on a non-semver custom tag, so it needs a fallback; asking the node with kubeadm version -o short is a third option. Happy to attempt a PR if you would prefer, though I would be relying on CI to compile it (Command Line Tools only here).
Related: PR #2244 documents --node-image for running several Kubernetes versions side by side, which is what surfaced this.
Environment
- OS: macOS 26.2 (25C56), Apple silicon
- Xcode: Command Line Tools only
- Container: container CLI version 1.3.1 (Homebrew);
Sources/ContainerK8s read at main
Code of Conduct
I have done the following
Steps to reproduce
The defect is in the source, so the shortest reproduction is to hand kubeadm inside a 1.34 node image the config the plugin generates for it (
initConfigYAMLinSources/ContainerK8s/Support/K8sHelper+Bootstrap.swift, whosekubernetesVersioncomes out asv1.35.5for any--node-image):Output:
I could not run
container k8s create --node-image docker.io/kindest/node:v1.34.11to the end on the shipped 1.3.1 because node prep stops earlier (#2120), so the end state below is expected rather than observed.Problem description
kubernetesVersion()inK8sHelper+Bootstrap.swifttakes no argument: it parses the tag out of the staticK8sHelper.nodeImageconstant (docker.io/kindest/node:v1.35.5@sha256:...). The--node-imagevalue flows toLinuxNodeProvisioner, which pulls and boots it, but never tobootstrapControlPlane. So every cluster's kubeadm config sayskubernetesVersion: v1.35.5.kubeadm treats a newer control-plane version as a warning, and the image list above is what it would pull, so a user who asks for a 1.34 node would be expected to get a 1.35.5 control plane on a 1.34 kubelet. For older images the gap grows until it exceeds the documented n-3 skew (kubelet may be up to three minor versions older than kube-apiserver): a v1.32 node lands exactly at that limit, v1.31 one minor past it. Below 1.31 kubeadm rejects the
v1beta4config outright (experimental API spec: "kubeadm.k8s.io/v1beta4" is not allowed, checked inkindest/node:v1.30.8;v1.31.14accepts it), so 1.31 is the effective floor for--node-imageregardless of this bug.Expected: the kubeadm config carries the version of the image actually booted. The simplest fix is to omit
kubernetesVersionentirely: kubeadm then resolves it itself, to the latest patch of its own minor when it can reach dl.k8s.io and to its own exact version when it cannot (both checked insidekindest/node:v1.34.0, which resolved to v1.34.11 online and to v1.34.0 with the fetch blocked, warning only). Parsing the resolved image tag also works butkubeadm config validatehard-errors on a non-semver custom tag, so it needs a fallback; asking the node withkubeadm version -o shortis a third option. Happy to attempt a PR if you would prefer, though I would be relying on CI to compile it (Command Line Tools only here).Related: PR #2244 documents
--node-imagefor running several Kubernetes versions side by side, which is what surfaced this.Environment
Sources/ContainerK8sread atmainCode of Conduct