|
| 1 | +# Plan: Attach observation-portal to Keycloak via `pyobs-auth` |
| 2 | + |
| 3 | +Our self-hosted `observation-portal` (the MONET fork of the OCS |
| 4 | +[observatorycontrolsystem/observation-portal](https://github.com/observatorycontrolsystem/observation-portal) |
| 5 | +project) currently authenticates entirely against its own local Django user database. This plan |
| 6 | +makes it a first-class `pyobs-auth` client like pyobs-archive / pyobs-portal / pyobs-web-admin: |
| 7 | +the portal's login page gains a Keycloak login button and its API accepts Keycloak bearer tokens — |
| 8 | +**additive and env-gated** on top of the existing local username/password auth, with |
| 9 | +`KEYCLOAK_SERVER_URL` unset disabling Keycloak entirely. |
| 10 | + |
| 11 | +Status: proposed, 2026-08-28. No GitHub issue filed yet — file one before implementation if the |
| 12 | +repo convention wants it. |
| 13 | + |
| 14 | +Repos: observation-portal (MONET fork), pyobs-auth. |
| 15 | + |
| 16 | +## Relationship to prior auth work (direction change) |
| 17 | + |
| 18 | +- **ADR 0011** (parallel Keycloak + "odin" backends) was superseded 2026-08-19 by the brokering |
| 19 | + design; not relevant here. |
| 20 | +- **`specs/design/shared-auth-keycloak.md` + `specs/plans/2026-08-12-shared-auth-keycloak.md` |
| 21 | + (Section 0)** — the original direction was to *broker* observation-portal **behind** Keycloak: |
| 22 | + enable OIDC on the portal's own `django-oauth-toolkit` provider, register the portal in Keycloak |
| 23 | + as an upstream Identity Provider (pure Keycloak admin config, tracked in #748, still open). |
| 24 | + **This plan changes that direction**: the portal becomes a *relying party* (client) of Keycloak |
| 25 | + instead, exactly like the other pyobs services. Users then log into the portal with Keycloak |
| 26 | + identities directly, which makes brokering the portal behind Keycloak unnecessary for the MONET |
| 27 | + deployment — Section 0 is superseded. The design doc's "observation-portal is brokered behind |
| 28 | + Keycloak" stance gets a status note (section 10) when this lands. |
| 29 | +- **`specs/plans/2026-08-21-keycloak-idp-hint-login.md`** — `pyobs-auth >= 2.0.0.dev9` ships |
| 30 | + `IDP_HINT`/`kc_idp_hint`; the portal adopts the same dual-button login-page pattern as |
| 31 | + archive/portal/web-admin. |
| 32 | +- **Single-issuer stance is unaffected**: `pyobs_auth.authentication.KeycloakAuthentication` |
| 33 | + validates Keycloak tokens only; the portal's own OAuth2 provider role (`oauth2_provider`, |
| 34 | + `OAUTH_CLIENT_APPS_BASE_URLS`, `OAUTH_SERVER_KEY`) and its tokens keep working unchanged |
| 35 | + (stacking-safety: KeycloakAuthentication defers on non-matching issuers, section 5). |
| 36 | + |
| 37 | +## Problem |
| 38 | + |
| 39 | +The portal today (`observation_portal/`): |
| 40 | + |
| 41 | +- **Login**: session-based `CustomLoginView` (username/email + password, |
| 42 | + `accounts/views.py:132`) + JSON `ApiLoginView` (`accounts/views.py:174`); self-registration via |
| 43 | + `django-registration-redux` (`accounts/urls.py:15`, `CustomRegistrationView`). |
| 44 | +- **Backends** (`settings.py:205`): `EmailOrUsernameModelBackend`, Django `ModelBackend`, |
| 45 | + `oauth2_provider.backends.OAuth2Backend` (portal-issued tokens only). |
| 46 | +- **DRF auth** (`settings.py:303`): Session, Token (`rest_framework.authtoken`), and |
| 47 | + `oauth2_provider.contrib.rest_framework.OAuth2Authentication`. |
| 48 | +- **Users**: standard `django.contrib.auth.User` + a required `Profile` (`accounts/models.py:18`) |
| 49 | + created by the registration flow — **not** by a signal, so any user minted outside registration |
| 50 | + (which is what `resolve_user` will do) must create its Profile explicitly. |
| 51 | +- **Dependencies**: Django 4.2.30 (`poetry.lock`), Python 3.10/3.11 (Dockerfile |
| 52 | + `python:3.10-slim`, CI matrix `.github/workflows/build.yaml`). |
| 53 | + |
| 54 | +No shared identity: a user authenticated against any pyobs web service has no way to be |
| 55 | +recognized by the portal, and vice versa — the gap this plan closes, consistent with the |
| 56 | +shared-auth design. |
| 57 | + |
| 58 | +## Approach |
| 59 | + |
| 60 | +Port the pyobs-portal integration (2026-08-12 plan section 3) into the portal's existing |
| 61 | +`observation_portal.accounts` app, using pyobs-archive's user-mapping shape |
| 62 | +(`Profile.keycloak_sub`, section 3). Everything is additive: local username/password login, |
| 63 | +self-registration, the portal's OAuth2 *provider* role for downstream LCO apps, and DRF token auth |
| 64 | +all stay. `KEYCLOAK_SERVER_URL` unset ⇒ portal behaves exactly as today. |
| 65 | + |
| 66 | +## 0. Prerequisite: Django 5.2 upgrade (gating) |
| 67 | + |
| 68 | +`pyobs-auth` requires `Django>=5.2,<7` and `Python>=3.11`; the portal is on Django 4.2.30 / |
| 69 | +Python 3.10+. This is the only hard external prerequisite and the biggest chunk of the plan — |
| 70 | +**recommend running it as its own track/plan first** if it balloons. |
| 71 | + |
| 72 | +- [ ] `pyproject.toml`: `django = "^4"` → `^5.2` (or 6.x, within pyobs-auth's `<7` pin); |
| 73 | + `python = ">=3.9,<3.12"` → `>=3.11`. |
| 74 | +- [ ] Dockerfile (`python:3.10-slim` → 3.11/3.12-slim) and CI matrix (drop 3.10). |
| 75 | +- [ ] Dependency audit for Django 5.2: |
| 76 | + - `django-oauth-toolkit ^2` → `^3` (DOT 3.0 supports Django 5.2; needed for the portal's |
| 77 | + OIDC provider role). |
| 78 | + - `django-registration-redux ^2.9` → `>=2.13`. |
| 79 | + - `drf-yasg ^1.20` → `>=1.21.10`. |
| 80 | + - Verify/`bump`: `django-filter`, `django-bootstrap4 <4.0` (effectively unmaintained — real |
| 81 | + risk for the Django-5 upgrade; fallback `django-bootstrap5` or drop if unused beyond the |
| 82 | + login/registration templates), `django-storages`, `django-extensions`, `django-cors-headers`, |
| 83 | + `django-dramatiq`, `django-health-check`, `django-object-actions`. |
| 84 | +- [ ] Settings sweep for removed/renamed Django 5 APIs (concrete hits already in |
| 85 | + `observation_portal/settings.py`): |
| 86 | + - `USE_L10N` (line 240) — removed in Django 5.0, delete. |
| 87 | + - `STATICFILES_STORAGE` / `DEFAULT_FILE_STORAGE` (lines 261, 265) — removed in Django 5.1, |
| 88 | + migrate to the `STORAGES` dict. |
| 89 | + - grep for deprecated `django.utils.timezone.utc` / other 4.x deprecations; run |
| 90 | + `manage.py check --deploy` and the test suite to surface the rest. |
| 91 | + |
| 92 | +## 1. Dependency + app wiring |
| 93 | + |
| 94 | +- [ ] `pyproject.toml`: add `pyobs-auth>=2.0.0.dev9` (pin whatever dev version is current at |
| 95 | + implementation, per `do-python-release` conventions). |
| 96 | +- [ ] `observation_portal/settings.py` `INSTALLED_APPS`: add `'pyobs_auth'` (no new local app — |
| 97 | + resolver/model/context-processor live in the existing `observation_portal.accounts`, |
| 98 | + mirroring pyobs-archive rather than pyobs-portal's separate `authentication` app). |
| 99 | + |
| 100 | +## 2. `PYOBS_AUTH` settings (env-driven) |
| 101 | + |
| 102 | +- [ ] `observation_portal/settings.py`: add the `PYOBS_AUTH` dict, matching the other services: |
| 103 | + |
| 104 | + ```python |
| 105 | + PYOBS_AUTH = { |
| 106 | + "SERVER_URL": os.getenv("KEYCLOAK_SERVER_URL", ""), |
| 107 | + "REALM": os.getenv("KEYCLOAK_REALM", "pyobs"), |
| 108 | + "CLIENT_ID": os.getenv("KEYCLOAK_CLIENT_ID", "observation-portal"), |
| 109 | + "CLIENT_SECRET": os.getenv("KEYCLOAK_CLIENT_SECRET", ""), |
| 110 | + "REDIRECT_URI": os.getenv("KEYCLOAK_REDIRECT_URI", ""), |
| 111 | + "POST_LOGOUT_REDIRECT_URI": os.getenv("KEYCLOAK_POST_LOGOUT_REDIRECT_URI", ""), |
| 112 | + "IDP_HINT": os.getenv("KEYCLOAK_IDP_HINT", ""), |
| 113 | + "IDP_LABEL": os.getenv("KEYCLOAK_IDP_LABEL", ""), |
| 114 | + "USER_RESOLVER": "observation_portal.accounts.keycloak.resolve_user", |
| 115 | + } |
| 116 | + ``` |
| 117 | + |
| 118 | + Unset `SERVER_URL` disables Keycloak entirely (pyobs-auth's `get_settings()` raises only on |
| 119 | + first use — harmless while nothing presents a Keycloak token or hits the login views). |
| 120 | +- [ ] Document the `KEYCLOAK_*` env vars in `README.md`'s environment-variable table and in |
| 121 | + `deploy/.env` / `k8s/envs/local/settings.env` (next to the existing auth-related vars). |
| 122 | + |
| 123 | +## 3. User mapping — `Profile.keycloak_sub` + `resolve_user` |
| 124 | + |
| 125 | +Mirror pyobs-archive's implemented pattern (`Profile.keycloak_sub` migration + resolver), not |
| 126 | +pyobs-portal's separate `KeycloakIdentity` model — the portal already has a `Profile` |
| 127 | +(`accounts/models.py:18`). |
| 128 | + |
| 129 | +- [ ] `observation_portal/accounts/models.py`: add `keycloak_sub = models.CharField(max_length=255, |
| 130 | + unique=True, blank=True, null=True)` to `Profile` + migration. Keycloak's `sub` claim is the |
| 131 | + join key (stable; email/username can change), per the design doc. |
| 132 | +- [ ] New `observation_portal/accounts/keycloak.py` with `resolve_user(claims) -> User`, shaped like |
| 133 | + `pyobs_portal.authentication.keycloak.resolve_user`: |
| 134 | + - `sub` → `Profile.keycloak_sub` lookup first; |
| 135 | + - else match existing `User` by email, falling back to username (use `.first()` — Django's |
| 136 | + `User.email` is not unique, duplicates exist in practice); |
| 137 | + - else mint `User(username=..., email=..., is_active=False)` **and** a `Profile` with |
| 138 | + `institution=''`, `title=''` (non-blank CharFields; no auto-creation signal exists — the |
| 139 | + registration flow creates Profiles today). Note: `Profile` post-save fires |
| 140 | + `update_or_create_client_applications_user` (`accounts/signals/handlers.py:22`) — harmless, |
| 141 | + it no-ops without `OAUTH_CLIENT_APPS_BASE_URLS`. |
| 142 | + - store/refresh `Profile.keycloak_sub`; refuse/inactive users are handled by pyobs-auth |
| 143 | + (new accounts minted inactive ⇒ per-portal activation gate, unchanged from the design). |
| 144 | +- [ ] Decision to confirm (open question below): auto-link existing portal users by email on first |
| 145 | + Keycloak login (design-doc default, what archive/portal do) vs. requiring an explicit |
| 146 | + admin-side link. Recommended: auto-link by email. |
| 147 | + |
| 148 | +## 4. URLs — mount `pyobs_auth.urls` |
| 149 | + |
| 150 | +- [ ] `observation_portal/urls.py`: add `path('accounts/keycloak/', include('pyobs_auth.urls'))` |
| 151 | + **before** `re_path(r'^accounts/', include(accounts_urls))` (line 119). Ordering matters: |
| 152 | + `accounts/urls.py` ends in the catch-all |
| 153 | + `re_path(r'', include('registration.backends.default.urls'))`, which would swallow |
| 154 | + `accounts/keycloak/...` and 404 it. |
| 155 | +- [ ] Resulting endpoints: `/accounts/keycloak/login/`, `/callback/`, `/logout/` |
| 156 | + (`pyobs_auth.urls`). No change to the portal's own `/accounts/login/` etc. |
| 157 | + |
| 158 | +## 5. DRF authentication — stack `KeycloakAuthentication` |
| 159 | + |
| 160 | +- [ ] `observation_portal/settings.py` `REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES']`: append |
| 161 | + `'pyobs_auth.authentication.KeycloakAuthentication'` after |
| 162 | + `'oauth2_provider.contrib.rest_framework.OAuth2Authentication'`. Safe to stack: |
| 163 | + KeycloakAuthentication returns `None` (defers) for any Bearer token whose issuer isn't the |
| 164 | + Keycloak realm, so portal-minted OAuth2 tokens still authenticate via the oauth2_provider |
| 165 | + class. |
| 166 | +- [ ] No `AUTHENTICATION_BACKENDS` change — the redirect-based OIDC flow is handled by |
| 167 | + `pyobs_auth.views`, not a backend class (see design doc). |
| 168 | + |
| 169 | +## 6. Login page + context processor |
| 170 | + |
| 171 | +- [ ] New `observation_portal/accounts/context_processors.py` with `keycloak(request)` exposing |
| 172 | + `keycloak_login_enabled`, `keycloak_idp_hint`, `keycloak_idp_label` (mirror |
| 173 | + `pyobs_portal/frontend/context_processors.py`); register it in |
| 174 | + `settings.py` `TEMPLATES['OPTIONS']['context_processors']`. |
| 175 | +- [ ] `observation_portal/accounts/templates/registration/login.html`: add the Keycloak buttons |
| 176 | + behind `{% if keycloak_login_enabled %}` — dual-button pattern when `IDP_HINT` is configured |
| 177 | + ("Log in with {{ keycloak_idp_label }}" via `{% url 'pyobs_auth:login' %}` and "Log in with |
| 178 | + local Keycloak account" via `?idp_hint=`), single button otherwise; **preserve `next` with |
| 179 | + `|urlencode`** on every link (and fix the existing unescaped `next` in the local form while |
| 180 | + the file is open). The local username/password form stays as-is. |
| 181 | + |
| 182 | +## 7. Logout |
| 183 | + |
| 184 | +- [ ] The portal has no user-facing logout URL today (`accounts/urls.py` only wires login/register/ |
| 185 | + password views; logout is admin-only). Wire the site's logout action to POST |
| 186 | + `{% url 'pyobs_auth:logout' %}` (pyobs-auth's RP-Initiated Logout ends the Keycloak SSO |
| 187 | + session for Keycloak-originated sessions, and does an ordinary local logout otherwise), or |
| 188 | + add a plain `django.contrib.auth.urls` logout include as the non-Keycloak fallback. |
| 189 | + |
| 190 | +## 8. Interplay with existing portal features |
| 191 | + |
| 192 | +- [ ] Self-registration (`CustomRegistrationView`, `registration.backends.default.urls`) untouched — |
| 193 | + local accounts remain a first-class path. |
| 194 | +- [ ] `password_expiration` (`CustomLoginView.get_success_url`, `ApiLoginView`) applies to |
| 195 | + local-password logins only; Keycloak sessions bypass those views entirely — no change. |
| 196 | +- [ ] Portal's OAuth2 provider role (`/o/` endpoints, `OAUTH2_PROVIDER` OIDC settings, |
| 197 | + `OAUTH_CLIENT_APPS_BASE_URLS`, `OAUTH_SERVER_KEY`) untouched — this plan does not enable OIDC |
| 198 | + on the portal's own provider, so Section 0's Keycloak-admin items are not needed either. |
| 199 | +- [ ] `LimitAnonymousAccessMiddleware` / throttles / `ApiLoginView` unchanged. |
| 200 | + |
| 201 | +## 9. Tests |
| 202 | + |
| 203 | +- [ ] `resolve_user`: first-login links existing user by email and by username; mints |
| 204 | + User+Profile (inactive) with empty-string profile fields; re-login resolves via `sub`; |
| 205 | + duplicate-email case resolves deterministically. |
| 206 | +- [ ] DRF: `KeycloakAuthentication` authenticates a valid Keycloak token; defers (returns `None`) |
| 207 | + on a foreign-issuer token; refuses an inactive user. |
| 208 | +- [ ] Stacking: a portal-minted oauth2 access token still authenticates with both |
| 209 | + `OAuth2Authentication` and `KeycloakAuthentication` registered. |
| 210 | +- [ ] Context processor / template: `keycloak_login_enabled` false ⇒ login page identical to today. |
| 211 | + |
| 212 | +## 10. Docs (pyobs-core) |
| 213 | + |
| 214 | +- [ ] `specs/design/shared-auth-keycloak.md`: status note — observation-portal is attached as a |
| 215 | + `pyobs-auth` client (relying party) rather than brokered behind Keycloak; Section 0 of the |
| 216 | + 2026-08-12 plan is superseded. |
| 217 | +- [ ] `specs/plans/index.md`: this plan's entry. |
| 218 | + |
| 219 | +## Verification |
| 220 | + |
| 221 | +- [ ] Full portal test suite green (existing + new). |
| 222 | +- [ ] Regression: `KEYCLOAK_SERVER_URL` unset ⇒ login page renders exactly today's form; API |
| 223 | + behavior unchanged. (Verifiable from a checkout.) |
| 224 | +- [ ] Live browser E2E against the running Keycloak (operational — not verifiable from a repo |
| 225 | + checkout): Keycloak login button → GWDG SSO (with `IDP_HINT`) / local Keycloak account → |
| 226 | + callback → `resolve_user` → portal session; logout ends both the portal and the Keycloak SSO |
| 227 | + session; a Keycloak bearer token authenticates a portal API call. |
| 228 | + |
| 229 | +## Not in this plan |
| 230 | + |
| 231 | +- **Brokering the portal behind Keycloak** (Section 0 of the 2026-08-12 plan) — superseded by this |
| 232 | + direction. A deployment that wants Keycloak to federate the portal's own users can still do it as |
| 233 | + optional Keycloak admin config (enable OIDC on the portal's provider, register it as upstream |
| 234 | + IdP), independent of this plan. |
| 235 | +- **Service-to-service auth** — the portal's OAuth2 provider for downstream LCO apps, and |
| 236 | + pyobs-core's static-token calls to portal (`Authorization: Token`), both stay as-is; Keycloak |
| 237 | + client-credentials remains optional (see design doc). |
| 238 | +- **Upstreaming the changes to `observatorycontrolsystem/observation-portal`** — the changes are |
| 239 | + additive and env-gated, so upstreaming is feasible, but it's a separate decision (the OCS project |
| 240 | + is the upstream; the MONET fork is the vehicle here). |
| 241 | +- **Keycloak admin/deployment config** for the MONET realm (registering an |
| 242 | + `observation-portal` client, redirect URIs) — operational, outside repo code. |
| 243 | + |
| 244 | +## Open questions |
| 245 | + |
| 246 | +- **Django target**: 5.2 LTS vs 6.x for the prerequisite upgrade (pyobs-auth allows `<7`). |
| 247 | +- **`keycloak_sub` on `Profile` vs a separate `KeycloakIdentity` model** — recommended: |
| 248 | + `Profile.keycloak_sub` (mirrors pyobs-archive, no new app). |
| 249 | +- **Auto-linking existing portal users by email on first Keycloak login** vs explicit admin-side |
| 250 | + linking — recommended: auto-link by email (design-doc default), acknowledging the |
| 251 | + non-unique-email caveat (`.first()`). |
| 252 | +- **Post-login redirect for Keycloak sessions**: pyobs-auth defaults `next` to `/`; the portal's |
| 253 | + local login redirects to `/accounts/loggedinstate/` — decide whether the Keycloak button should |
| 254 | + pass `next=/accounts/loggedinstate/` to keep the post-login landing consistent. |
0 commit comments