Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,9 @@ Build pipelines run as Kubernetes Jobs with init containers. The container names
3. **compile** (optional, `CompileContainer`): Runs custom build commands
4. **publish** (`PublishContainer`): Builds and pushes Docker image using Buildah, with the `overlay` storage driver over the `container-storage` emptyDir that `ContainerStorageVolume` mounts at `/var/lib/containers`. Both halves matter: the store defaults into the container's writable layer, where every write pays an overlayfs copy-up, and `vfs` (used until the volume existed) copies the entire tree per layer — together they made a build of any size stall for minutes with no log output after its last line. Buildah's own image declares a `VOLUME` for this path, but Kubernetes ignores image `VOLUME` declarations, so the mount has to be explicit.

Two source types exist: `GIT` (default) and `ZIP`. ZIP uploads use presigned S3 URLs via `BuildSourceObjectStorageService` — the frontend gets a presigned PUT URL from `POST .../deployments/source-upload`, uploads the file, then triggers the pipeline. ZIP builds use `oops.pipeline.image.zip` (defaults to `alpine/curl:8.17.0`) to download the archive.
Three source types exist: `GIT` (default), `ZIP` and `IMAGE`. ZIP uploads use presigned S3 URLs via `BuildSourceObjectStorageService` — the frontend gets a presigned PUT URL from `POST .../deployments/source-upload`, uploads the file, then triggers the pipeline. ZIP builds use `oops.pipeline.image.zip` (defaults to `alpine/curl:8.17.0`) to download the archive.

**`IMAGE` runs no build at all.** The build config stores the image name without a tag (`ImageSourceConfig.image`, a field of its own rather than a reuse of the Git `repository`, so an application switching between the two sources keeps both values instead of offering a Git URL as an image name — `ApplicationBuildConfig.repository()` and `.image()` each answer for one source and `null` for the others), the publish names the tag (`ImageDeployStrategyParam.tag`, validated against the OCI tag grammar so it cannot smuggle another image or a digest), and `Pipeline.initializeWithArtifact` is born with `artifact = repository:tag` — the same shape the build path produces (`registry/app:pipelineId`), so `StatefulSetProcessor`, rollback, notifications and namespace migration need no special case. The pipeline never enters `RUNNING`: `IMMEDIATE` goes `INITIALIZED → DEPLOYING → ROLLING_OUT` through `ArtifactDeployRunner` (the deploy phase shared with rollback and manual deploy), `MANUAL` parks in `BUILD_SUCCEEDED` (the one `INITIALIZED → BUILD_SUCCEEDED` transition the state machine allows) for the ordinary `deployPipeline()` call. Consequences worth knowing: `Application.updateBuildConfig` drops the Dockerfile, build image and build commands for an IMAGE application rather than keep dead configuration; the pipeline page and list show no step bar and no special mark (`Pipeline.hasBuild()` is false, as for a rollback, but an image publish is an ordinary release, not a rollback, so it gets no badge or icon); the IDE is refused, like ZIP, because there is no repository to clone; a wrong tag fails the rollout as `ImagePullBackOff` rather than at publish time, since nothing checks the registry; re-publishing the same tag leaves the pod template unchanged and so restarts nothing; and rollback is by tag, not digest, so a tag overwritten in the registry rolls back to whatever it now holds. First version supports only images pullable without credentials — `ImagePullSecretProcessor` still copies just the environment's own registry secret.

Pipeline build logs are served over SSE, one container at a time (see **Pipeline log streaming** below). A `@Scheduled(fixedRate=5000)` job (`PipelineInstanceScanJob`) polls K8s for build completion and rollout convergence. Pipeline state transitions use optimistic locking: `PipelineRepository.updateStatusIfMatch()` does a conditional UPDATE and returns row count (0 = lost the race). **Every** status write goes through it, user-triggered ones included — `stopPipeline` transitions from the status the caller read and fails with "state changed concurrently" when that read is stale, and a lost `DEPLOYING → ROLLING_OUT` claim (the pipeline was stopped while its artifact was applied) is logged and not announced as rolling out. Two ways a pipeline could otherwise stick forever and block its application are recovered by the scan job: a RUNNING pipeline whose build Job the cluster no longer has (deleted, namespace cleaned, TTL-reaped) is failed at once, and a pipeline that has stayed DEPLOYING for 10 minutes (its driver died mid-deploy) is failed by whichever server can claim its lock — measured from when that server first saw it deploying, since no column records status-change time and the conditional updates bypass entity timestamps.

Expand Down Expand Up @@ -202,8 +204,10 @@ Deployment triggering logic lives in `DeploymentService` (not `PipelineService`)

The K8s client is created per-task and closed via try-with-resources in `ArtifactDeployTask.call()`.

**One editor tab, one request**: every tab of the application editor saves through exactly one endpoint, in one transaction — the build tab's per-environment build commands ride inside `PUT .../build/config` as `environmentConfigs`, and the basic-info tab's environment bindings ride inside `PUT .../applications/{name}` as `environments`. In both, an omitted list means unchanged, so an OpenAPI caller that does not know about the list (a profile PUT built from an older client, or the CLI's `app build set` without `--build-command`) cannot wipe it; `PUT .../environments` still exists for changing the bindings alone. Never chain a second request to save part of a form: the first call can wipe what the second restores, and a failed second call leaves a half-saved record behind a generic error toast.

**Per-application config entities**:
- `ApplicationBuildConfig`: Stores source type (`GIT`/`ZIP`), repository/source key, build image/commands, and Dockerfile config (`BUILTIN` path or inline `USER` content). Frontend: `application-build-info.tsx`.
- `ApplicationBuildConfig`: Stores source type (`GIT`/`ZIP`/`IMAGE`), repository or image name (`SourceConfig` JSON blob: `GitSourceConfig`, `ZipSourceConfig`, `ImageSourceConfig`), build image/commands, and Dockerfile config (`BUILTIN` path or inline `USER` content). Frontend: `application-build-info.tsx`; the publish page renders the IMAGE tag as a prefixed input (`components/ui/input-group.tsx`) with the image name fixed in front of it.
- `ApplicationServiceConfig`: Stores container `port` and per-environment hostname/HTTPS overrides (`List<EnvironmentConfig>` as JSON blob). Frontend: `application-service-info.tsx`.

Each host can also carry **HTTP basic auth** (`basicAuthEnabled` / `basicAuthUsername` / `basicAuthPasswordHash`, set in the host editor dialog `apps/components/host-editor-dialog.tsx`, which is the single add/edit form for a host's name, HTTPS and basic auth). Only the BCrypt hash is stored — the plaintext is hashed in `ApplicationService.updateApplicationServiceConfig` and never returned, so the DTO exposes a write-only `basicAuthPassword` plus a read-only `basicAuthPasswordSet` marker, and a blank password on update carries the stored hash forward. At deploy time `IngressRouteProcessor` writes an htpasswd Secret and a Traefik `basicAuth` Middleware named `{app}-basic-auth-{host}` (labelled `oops.resource=basic-auth` so hosts that turn auth off get their pair deleted), attached to the route that actually serves traffic — for an HTTPS host that is the `websecure` route, since the `web` one only redirects. Note the endpoint rewrites the whole service config: a client that PUTs `environmentConfigs` without these fields (e.g. `oops app service set`) clears basic auth, exactly as it already clears unlisted hosts.
Expand Down Expand Up @@ -420,6 +424,12 @@ OOPS uses Flyway to apply schema and data migrations automatically during applic
`environmentName` JSON keys to `environment`. Its column renames are guarded on
`information_schema`, because a database that ran the 3.0 Go release already carries the new
names while its `flyway_schema_history` stopped at V21 — the migration must be a no-op there
- `V23__widen_enum_columns_to_varchar.sql` turns the `@Enumerated(STRING)` columns
(`source_type`, `publish_type`, `cert_mode`, `provider`, `role`) into `varchar(255)`. A database
that predates Flyway got them from Hibernate's DDL as native MySQL `ENUM(...)` columns frozen at
the constants of the day, so adding a constant (`IMAGE`) failed with "Data truncated for column"
and rolled the save back, even though `V1__baseline_schema.sql` documents them as varchar. Never
declare a new enum-backed column as `ENUM`, for the same reason

## Configuration Notes

Expand Down
33 changes: 33 additions & 0 deletions skills/oops/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,29 @@ python skills/oops/scripts/oops.py app build set -n <ns> <app> \

`builtin` reads the real Dockerfile from the repository.

Per-environment build commands are set with `--build-command <env>=<command>` (repeatable):

```bash
python skills/oops/scripts/oops.py app build set -n <ns> <app> \
--source git --repository "https://github.com/owner/repo.git" \
--build-command "dev=npm ci && npm run build" \
--build-command "prod=npm ci && npm run build -- --mode production"
```

Without any `--build-command` the existing commands are left as they are; to clear them all, pass `--build-command ""`.

#### Image source (prebuilt image, nothing is built)

```bash
python skills/oops/scripts/oops.py app build set -n <ns> <app> \
--source image \
--image "ghcr.io/owner/app"
```

`--image` is the image name **without a tag or digest** — the tag is chosen at each deploy
(step 7). Dockerfile, build image and build commands do not apply and are dropped. Only images
pullable without credentials are supported.

### Step 4 — Bind to environments

```bash
Expand Down Expand Up @@ -185,6 +208,16 @@ python skills/oops/scripts/oops.py deploy git -n <ns> <app> --env <env> --branch

Uses the repository configured in step 3; `--branch` is optional.

#### Image mode

```bash
python skills/oops/scripts/oops.py deploy image -n <ns> <app> --env <env> --tag 1.2.3 --wait
```

Deploys `<repository>:<tag>` with the image name from step 3; `--tag` defaults to `latest`. No
build runs, so `--wait` only follows the rollout. A tag that does not exist fails the rollout as
`ImagePullBackOff`, not at trigger time, and re-deploying the same tag changes nothing.

### Step 8 — Verify

```bash
Expand Down
51 changes: 49 additions & 2 deletions skills/oops/scripts/oops.py
Original file line number Diff line number Diff line change
Expand Up @@ -251,25 +251,52 @@ def cmd_app_build_get(client: Client, args: argparse.Namespace) -> None:
def human(c):
print(f"Source: {dash(c.get('sourceType'))}")
print(f"Repository: {dash(c.get('repository'))}")
print(f"Image: {dash(c.get('image'))}")
print(f"Build image: {dash(c.get('buildImage'))}")
df = c.get("dockerFileConfig") or {}
print(f"Dockerfile: type={dash(df.get('type'))}, path={dash(df.get('path'))}")
for item in c.get("environmentConfigs") or []:
print(f"Build cmd: {item.get('environment')}={dash(item.get('buildCommand'))}")
render(args.json, config, human)


def parse_build_commands(values: Optional[List[str]]) -> Optional[List[Dict[str, str]]]:
"""`ENV=COMMAND` pairs → environmentConfigs. None (flag absent) leaves the stored commands
alone; a lone empty string clears them all."""
if values is None:
return None
if values == [""]:
return []
configs = []
for value in values:
if "=" not in value:
die(f"--build-command expects ENV=COMMAND, got {value!r}")
env, command = value.split("=", 1)
if not env.strip():
die(f"--build-command has an empty environment name: {value!r}")
configs.append({"environment": env.strip(), "buildCommand": command})
return configs


def cmd_app_build_set(client: Client, args: argparse.Namespace) -> None:
body = {
"namespace": args.namespace,
"applicationName": args.name,
"sourceType": args.source.upper(),
# Both travel every time: the Git URL and the image name are separate fields, so an
# application that switches source keeps the one it is not using.
"repository": args.repository or "",
"image": args.image or "",
"dockerFileConfig": {
"type": args.dockerfile_type.upper(),
"path": args.dockerfile_path,
"content": args.dockerfile_content or "",
},
"buildImage": args.build_image or "",
}
build_commands = parse_build_commands(args.build_commands)
if build_commands is not None:
body["environmentConfigs"] = build_commands
client.put(f"/openapi/namespaces/{args.namespace}/applications/{args.name}/build/config", body)
render(args.json, {"updated": True}, lambda _: print(f"Build config updated for {args.namespace}/{args.name}"))

Expand Down Expand Up @@ -544,6 +571,12 @@ def cmd_deploy_zip(client: Client, args: argparse.Namespace) -> None:
_trigger_deploy(client, args.namespace, args.name, args.env, args.mode, strategy, args.wait, args.json)


def cmd_deploy_image(client: Client, args: argparse.Namespace) -> None:
# Nothing is built: the image name comes from the build config and only the tag is chosen here.
strategy = {"type": "IMAGE", "tag": args.tag}
_trigger_deploy(client, args.namespace, args.name, args.env, args.mode, strategy, args.wait, args.json)


# ---------------------------------------------------------------------------
# Argument parser
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -606,12 +639,17 @@ def build_parser() -> argparse.ArgumentParser:
build_set = build_sub.add_parser("set")
build_set.add_argument("-n", "--namespace", required=True)
build_set.add_argument("name")
build_set.add_argument("--source", default="git", choices=["git", "zip"])
build_set.add_argument("--repository", default="")
build_set.add_argument("--source", default="git", choices=["git", "zip", "image"])
build_set.add_argument("--repository", default="", help="Git URL, for --source git")
build_set.add_argument("--image", default="",
help="image name without a tag, for --source image")
build_set.add_argument("--dockerfile-type", default="user", choices=["builtin", "user"], dest="dockerfile_type")
build_set.add_argument("--dockerfile-path", default="Dockerfile", dest="dockerfile_path")
build_set.add_argument("--dockerfile-content", dest="dockerfile_content")
build_set.add_argument("--build-image", dest="build_image")
build_set.add_argument("--build-command", action="append", dest="build_commands", metavar="ENV=COMMAND",
help="per-environment build command (repeatable); omit to keep the existing ones, "
"pass an empty string to clear them all")

# app service
service_p = app_sub.add_parser("service")
Expand Down Expand Up @@ -736,6 +774,14 @@ def build_parser() -> argparse.ArgumentParser:
dz.add_argument("--mode", default="immediate", choices=["immediate", "manual"])
dz.add_argument("--wait", action="store_true", default=False)

di = deploy_sub.add_parser("image")
di.add_argument("-n", "--namespace", required=True)
di.add_argument("name")
di.add_argument("--env", required=True)
di.add_argument("--tag", default="latest", help="tag of the image configured in the build config")
di.add_argument("--mode", default="immediate", choices=["immediate", "manual"])
di.add_argument("--wait", action="store_true", default=False)

return root


Expand Down Expand Up @@ -793,6 +839,7 @@ def main() -> None:
("pipeline", "watch"): lambda: cmd_pipeline_watch(client, args),
("deploy", "git"): lambda: cmd_deploy_git(client, args),
("deploy", "zip"): lambda: cmd_deploy_zip(client, args),
("deploy", "image"): lambda: cmd_deploy_image(client, args),
}

if args.command == "app":
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import com.github.wellch4n.oops.domain.application.ApplicationRuntimeSpec;
import com.github.wellch4n.oops.domain.application.ApplicationServiceConfig;
import com.github.wellch4n.oops.domain.application.GitSourceConfig;
import com.github.wellch4n.oops.domain.application.ImageSourceConfig;
import com.github.wellch4n.oops.domain.application.ZipSourceConfig;
import com.github.wellch4n.oops.domain.shared.ApplicationSourceType;
import com.github.wellch4n.oops.domain.shared.DockerFileType;
Expand All @@ -19,6 +20,11 @@ public final class ApplicationConfigDto {
private ApplicationConfigDto() {
}

/**
* @param environments the environment bindings, saved together with the profile because the
* basic-info editor edits both on one form. {@code null} on update means
* "leave the bindings as they are"; ignored on create.
*/
public record Profile(
String id,
LocalDateTime createdTime,
Expand All @@ -27,7 +33,8 @@ public record Profile(
String icon,
String namespace,
String owner,
List<String> collaborators
List<String> collaborators,
List<EnvironmentBinding> environments
) {
public Application toDomain() {
Application application = new Application();
Expand All @@ -48,7 +55,13 @@ public record BuildConfig(
String namespace,
String applicationName,
ApplicationSourceType sourceType,
/** Git URL, used when sourceType is GIT. */
String repository,
/**
* Image name without a tag, used when sourceType is IMAGE. Separate from
* {@code repository} so an application that switches between the two sources keeps both.
*/
String image,
DockerFileConfig dockerFileConfig,
String buildImage,
List<BuildEnvironmentConfig> environmentConfigs
Expand All @@ -64,6 +77,7 @@ public static BuildConfig from(ApplicationBuildConfig config) {
config.getApplicationName(),
config.getSourceType(),
config.repository(),
config.image(),
DockerFileConfig.from(config.getDockerFileConfig()),
config.getBuildImage(),
map(config.getEnvironmentConfigs(), BuildEnvironmentConfig::from)
Expand All @@ -77,9 +91,11 @@ public ApplicationBuildConfig toDomain() {
config.setNamespace(namespace);
config.setApplicationName(applicationName);
config.setSourceType(sourceType);
config.setSourceConfig(sourceType == ApplicationSourceType.ZIP
? new ZipSourceConfig()
: new GitSourceConfig(repository));
config.setSourceConfig(switch (sourceType != null ? sourceType : ApplicationSourceType.GIT) {
case GIT -> new GitSourceConfig(repository);
case ZIP -> new ZipSourceConfig();
case IMAGE -> new ImageSourceConfig(image);
});
config.setDockerFileConfig(dockerFileConfig != null ? dockerFileConfig.toDomain() : null);
config.setBuildImage(buildImage);
config.setEnvironmentConfigs(map(environmentConfigs, BuildEnvironmentConfig::toDomain));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = "type")
@JsonSubTypes({
@JsonSubTypes.Type(value = GitDeployStrategyParam.class, name = "GIT"),
@JsonSubTypes.Type(value = ZipDeployStrategyParam.class, name = "ZIP")
@JsonSubTypes.Type(value = ZipDeployStrategyParam.class, name = "ZIP"),
@JsonSubTypes.Type(value = ImageDeployStrategyParam.class, name = "IMAGE")
})
public sealed interface DeployStrategyParam permits GitDeployStrategyParam, ZipDeployStrategyParam {
public sealed interface DeployStrategyParam permits GitDeployStrategyParam, ZipDeployStrategyParam, ImageDeployStrategyParam {

ApplicationSourceType getType();
}
Loading
Loading