SENS Cloud and SENS Ops rollout plan
Status: Approved on 2026-09-02. Phase 1 is prepared for review. Source, GitOps, and server implementation has not started.
This plan is the execution order for the target architecture in ADR-0012. Follow one phase at a time. Finish its gate before starting the next phase.
This document and its related ADR, glossary, and Roadmap changes prepare Phase 1 while Samuel finishes the independent Phase 0 release baseline. Phase 2 must not start until both Phase 0 and Phase 1 gates have passed.
Outcome
Section titled “Outcome”The work ends with two physical servers and two independent single-node k3s clusters:
sens-ops server and cluster Argo CD OpenBao SENS Ops web and controller Valkey VictoriaMetrics, VictoriaLogs, vmauth, vmalert, and Alertmanager public status and external probes | | deploys and observes vsens-cloud server and cluster sens-cloud-test -> SENS Platform pre-release sens-cloud-production -> approved stable SENS Platform release environment-local PostgreSQL/TimescaleDB and NATS environment-local Secrets and persistent volumes cluster-local ESO, Reloader, metrics, and log agentsSENS Platform remains usable when SENS Ops is unavailable. The initial single-node clusters provide separation, not high availability.
Fixed decisions
Section titled “Fixed decisions”sens-platformis the portable product, its image family, and its OCI Helm chart.sens-cloudis the SENS-hosted installation of SENS Platform.sens-cloud-testandsens-cloud-productionare the only SENS Platform environments in the first installation.sens-opsis the management system on the second server. It has one live environment.sens-platformis the source monorepo for product source, Ops source, both first-party Helm charts, tests, CI, and documentation.sens-platform-infraremains the separate private GitOps desired-state repository.- The current runtime data and the Git history of
sens-platform-opsdo not need migration. - The current pre-customer runtime uses rebuild-only recovery. Backup and restore become mandatory before real customer data is accepted.
- SENS Ops application releases deploy automatically after a successful merge
to
main. - SENS Cloud Test deploys automatically. SENS Cloud Production retains the reviewed promotion and manual Argo CD synchronization.
- CI calculates impact down to a workspace package or deployable. Unknown and repository-wide changes fall back to full validation.
- SENS Platform keeps one atomic product version even when CI checks only the affected services.
Change impact
Section titled “Change impact”- Affected modules: every SENS Platform workspace, SENS Ops web and
controller, both first-party Helm charts, root CI and release tooling,
documentation, and both target-cluster definitions in
sens-platform-infra. - Affected contracts: product and Ops release identities, OCI chart locations, GitOps release-selection files, GitHub workflow-dispatch inputs, Argo CD destinations, Helm values, Secret references, DNS, and cross-cluster monitoring and Secret synchronization.
- Migrations: no application database, runtime-data, Secret-data, or Git history migration. Database migrations still run when each empty SENS Platform environment starts.
- Configuration: new server addresses, firewall rules, DNS targets, k3s settings, GitHub permissions, GHCR packages, cluster credentials, Helm values, and value-free Secret mappings.
- Tests: impact-resolver unit tests, affected package and integration tests, full scheduled CI, Helm lint and rendering, disposable cluster installs, platform verifiers, live route checks, and approved failure exercises.
- Documentation: glossary, ADR-0012, Roadmap, service inventory, system overview, release and Secret contracts, local development, bootstrap, operations, recovery, and customer-owned installation instructions.
- Security and tenant isolation: separate cluster credentials, least privilege Argo Projects, environment-local state and identities, no Secret values in Git, private administration, narrow firewall paths, and explicit Test-to-Production isolation checks.
- Deployment and operations: a hard cut to two single-node clusters. The topology separates failures but is not highly available. The current rebuild uses forward fixes or clean reinitialization instead of an old-runtime rollback.
- Expected SENS Platform release impact:
nonefor repository, CI, and topology-only work. Publishing the first officially supported OCI chart and customer-owned installation contract is expected to requireminor. This plan does not authorize a version bump. - Assumptions: both physical servers are available, current development data may be erased, all working trees are clean before Phase 2, GitHub-hosted Actions and GHCR remain available, and Samuel supplies the external inputs in the execution record.
Repository end state
Section titled “Repository end state”The source monorepo has one pnpm workspace and one lockfile:
sens-platform/├── apps/├── services/├── packages/├── operations/│ └── sens-ops/├── deploy/│ └── charts/│ ├── sens-platform/│ └── sens-ops/├── docs/├── scripts/├── .github/workflows/├── package.json├── pnpm-workspace.yaml└── pnpm-lock.yamlThe GitOps repository contains desired state and no application source or Secret values:
sens-platform-infra/├── clusters/│ ├── sens-ops/│ └── sens-cloud/├── installations/│ └── sens-cloud/│ ├── test/│ └── production/├── config/├── docs/└── scripts/The source monorepo publishes OCI artifacts. sens-platform-infra selects an
exact released version. Argo CD reads only sens-platform-infra when it
reconciles live clusters.
Responsibilities
Section titled “Responsibilities”Human responsibilities
Section titled “Human responsibilities”Samuel owns actions that require external authority or secret material:
- Finish the current CI and release alignment.
- Start execution only when all three current working trees are clean.
- Confirm that both servers and all current development data may be erased.
- Provide the server addresses, administrative network ranges, and DNS access.
- Approve every destructive server action, the initial activation of automatic synchronization, and every SENS Cloud Production synchronization.
- Enter Secret values through the approved Secret handoff. Do not send Secret values through chat or commit them to Git.
- Merge reviewed pull requests and approve the first SENS Cloud Production release.
AI responsibilities
Section titled “AI responsibilities”The implementing AI owns repository work and verification:
- Inspect the current state before each phase and preserve unrelated changes.
- Make product, Ops, and GitOps changes as separate reviewable diffs.
- Update contracts, documentation, tests, and verification scripts with each behavior change.
- Produce exact Secret and configuration handoffs without values.
- Run local and disposable verification before asking for a live action.
- Stop at every live or destructive step until Samuel gives explicit approval.
- Record the evidence required by each phase gate.
Automated responsibilities
Section titled “Automated responsibilities”- GitHub Actions validates affected code and publishes immutable OCI artifacts.
- A workflow in
sens-platform-infrais the only automated writer of release selection files. - Argo CD automatically synchronizes SENS Ops and SENS Cloud Test.
- Argo CD does not synchronize SENS Cloud Production until a human reviews the diff and manually triggers synchronization.
What Samuel should prepare now
Section titled “What Samuel should prepare now”- Finish the current CI and release work from Phase 0.
- Reserve one physical server for SENS Ops and one for SENS Cloud.
- Collect the non-secret facts in the execution record below.
- Confirm access to the DNS provider, both servers, both GitHub repositories, repository settings, Actions, and GHCR.
- Decide which administrative network or Tailscale identities may reach SSH, Argo CD, OpenBao, and the Kubernetes APIs.
- Confirm in writing that the old development runtime and its data may be erased after the new topology passes Phase 10.
Do not switch DNS, erase a server, create broad credentials, or copy current runtime data yet. The AI first prepares reviewable repository changes, idempotent bootstrap scripts, and a value-free Secret handoff.
Execution record
Section titled “Execution record”Fill this table before Phase 6. Store addresses and non-secret facts in the appropriate infrastructure document. Store credentials only in the approved Secret store.
| Input | Required value |
|---|---|
| SENS Ops server hostname | <required> |
| SENS Ops public IPv4 and optional IPv6 | <required> |
| SENS Ops private or Tailscale address | <required> |
| SENS Ops CPU, RAM, root disk, and data disks | <required> |
| SENS Cloud server hostname | <required> |
| SENS Cloud public IPv4 and optional IPv6 | <required> |
| SENS Cloud private or Tailscale address | <required> |
| SENS Cloud CPU, RAM, root disk, and data disks | <required> |
| Administrative source CIDRs | <required> |
| Administrative operating-system user | <required> |
| Operating system and version on both servers | <required> |
| Approved pinned k3s version | <required> |
| DNS provider and responsible person | <required> |
| ACME account e-mail address | <required> |
| GHCR owner and package visibility | <required> |
| GitHub App installation and repository access | <required> |
| Netmore MQTT endpoint and required egress port | <required> |
| SMTP endpoint and required egress port | <required or disabled> |
Do not start server provisioning while any required value is missing.
Phase 0: finish the current release baseline
Section titled “Phase 0: finish the current release baseline”Owner: Samuel.
-
Finish the in-progress CI changes.
-
Bring the current Test and Production release logic to the intended common version level.
-
Commit or deliberately remove every current working-tree change.
-
Run these read-only checks from the workspace:
Terminal window git status --shortgit -C ../sens-platform-infra status --shortgit -C ../sens-platform-ops status --shortpnpm version:status -
Confirm that all three
git status --shortcommands return no output.
Gate: the current release baseline is green, the three working trees are clean, and the selected SENS Platform base version is known.
Phase 1: align decisions and documentation
Section titled “Phase 1: align decisions and documentation”Owner: AI. Human approval is required before merge.
- Update ADR-0012 to describe the Greenfield hard cut, automatic Ops releases, selective CI, and rebuild-only recovery before customer go-live.
- Record the canonical terms in
CONTEXT.md. - Link this plan from the Internal index and the Roadmap.
- Mark the plan as the authoritative execution order while ADR-0012 remains the source of architectural reasons and trade-offs.
- Run the documentation build and link checks.
Gate: the glossary, ADR, Roadmap, English plan, and German plan describe the same target and contain no open architecture decision.
Phase 2: create the source monorepo
Section titled “Phase 2: create the source monorepo”Owner: AI. Human approval is required before merge.
- Take the clean current snapshot of
sens-platform-ops. - Add that snapshot under
operations/sens-ops/. Do not import its Git history and do not add a nested.gitdirectory. - Preserve the current product root so existing SENS Platform commit and release references remain valid.
- Add
operations/sens-opstopnpm-workspace.yaml. - Merge the Ops dependencies into the root
pnpm-lock.yamlwith the pinned workspace pnpm version. - Keep the root
package.jsonversion authoritative for SENS Platform. - Keep
operations/sens-ops/package.jsonauthoritative for the Ops base version. - Move Ops GitHub workflows to
.github/workflows/and give every workflow an explicitplatform,ops, orrepositoryname. - Update repository adapters, documentation links, Docker contexts, test configuration, Renovate configuration, Dependabot configuration, and CODEOWNERS for the new path.
- Move Ops documentation into the Internal and Technical documentation areas. Preserve customer-facing boundaries.
- Keep the old
sens-platform-opsrepository unchanged until Phase 11.
Run at least:
pnpm install --frozen-lockfilepnpm --filter sens-platform-ops formatpnpm --filter sens-platform-ops lintpnpm --filter sens-platform-ops typecheckpnpm --filter sens-platform-ops testpnpm --filter sens-platform-ops buildThe final package filter may change during implementation. The package name in
operations/sens-ops/package.json remains the authoritative value.
Gate: one clean clone and one pnpm install --frozen-lockfile can build and
test both SENS Platform and SENS Ops. The old Ops repository is no longer a
build or release dependency.
Phase 3: implement selective CI
Section titled “Phase 3: implement selective CI”Owner: AI. Human approval is required before branch-protection changes.
Add the impact resolver
Section titled “Add the impact resolver”- Add
scripts/ci/resolve-impact.mjsand its tests. - Read changed paths from the merge base to the tested commit.
- Derive workspace dependencies from each package’s
package.json. - Calculate every transitive dependent of a changed package.
- Maintain a small explicit map only for files outside workspace packages, such as Dockerfiles, charts, root configuration, and workflows.
- Emit JSON for affected products, packages, deployables, integration suites, charts, documentation, and release workflows.
- Select full validation for an unknown path, a root lockfile change, a shared CI change, or an impact-resolver change.
The resolver must implement at least these rules:
| Changed area | Required impact |
|---|---|
operations/sens-ops/** | Ops checks, Ops image, and Ops chart as applicable |
| One product service | That service and its direct test and image paths |
| One shared package | The package and every transitive product dependent |
| Database migrations | Database integration and every database consumer |
| Product contracts | Every contract consumer and relevant integration suite |
| Product Helm chart | Product chart lint and both environment renders |
| Ops Helm chart | Ops chart lint and disposable cluster smoke test |
| Documentation | Documentation checks only |
| Root lockfile, workspace, CI, or unknown path | Full repository validation |
Split the workflow
Section titled “Split the workflow”- Keep one always-present required result job.
- Run cheap repository checks for every pull request: Secret scan, workspace validation, impact calculation, and changed-file formatting.
- Generate package, integration, container, and chart matrices from the impact resolver output.
- Skip unaffected jobs while reporting an explicit successful result.
- Build only affected container images in pull requests.
- Run all Ops checks whenever Ops is affected because Ops has one live environment.
- Run full SENS Platform validation for the frozen commit before stable Production promotion.
- Add a scheduled full CI run to detect missing impact rules.
- Test the resolver against every top-level directory and every shared package.
Gate: representative pull requests prove Ops-only, docs-only, service-only, shared-package, chart-only, and fallback-to-all behavior. Branch protection requires only the stable overall result job.
Phase 4: publish independent OCI releases
Section titled “Phase 4: publish independent OCI releases”Owner: AI. Human approval is required for GitHub permissions and the first publication.
SENS Platform release
Section titled “SENS Platform release”- Move the product chart to
deploy/charts/sens-platform. - Package the chart with the same release version and
appVersionas the product image set. - Keep Test versions in the existing
MAJOR.MINOR.PATCH-alpha.RUN.ATTEMPTformat. - Keep Production versions as stable
MAJOR.MINOR.PATCH. - Publish one complete and immutable product artifact set for every product release. Selective pull-request CI does not create component-specific Production versions.
- Verify image labels, chart metadata, runtime configuration, and
/versionequality.
SENS Ops release
Section titled “SENS Ops release”-
Add
deploy/charts/sens-opsfor first-party Ops web, controller, Services, Jobs, ConfigMaps, and policies. -
Keep Argo CD, OpenBao, VictoriaMetrics, VictoriaLogs, ESO, and other upstream software outside the first-party Ops chart.
-
Derive an immutable Ops version from the base in
operations/sens-ops/package.json:MAJOR.MINOR.PATCH-main.RUN.ATTEMPT -
Change the stable Ops base version only through a reviewed Ops source change. Merging an Ops-affecting change remains the release trigger; there is no separate manual release action.
-
Set the Ops image tag, OCI chart version, chart
appVersion, andSENS_OPS_VERSIONto the same value. -
After an Ops-affecting merge to
main, run all Ops checks and the disposable Kubernetes smoke test. -
Publish the image and chart only after all checks pass.
-
Dispatch the Ops release selection workflow in
sens-platform-infra. -
Make one exact artifact version retry safe. An existing tag for the same commit completes successfully. An existing tag for another commit fails closed.
-
Treat a GitHub workflow rerun as a new immutable release attempt because
ATTEMPTchanges. Never overwrite artifacts from an earlier attempt.
GitHub preparation
Section titled “GitHub preparation”The AI produces a value-free handoff that lists every required GitHub App permission, repository variable, Actions Secret name, and GHCR package. Samuel configures the values after reviewing that handoff.
Gate: a disposable test pulls both OCI charts from GHCR, renders them, installs them, and observes the exact expected versions. No live cluster references a source-tree chart.
Phase 5: rebuild sens-platform-infra
Section titled “Phase 5: rebuild sens-platform-infra”Owner: AI. Human approval is required before merge because Argo CD reads this repository.
-
Replace
clusters/sens-k3swith explicitclusters/sens-opsandclusters/sens-cloudtrees. -
Move product environment values to
installations/sens-cloud/testandinstallations/sens-cloud/production. -
Name the product Applications and Helm releases
sens-cloud-testandsens-cloud-production. -
Keep product artifacts named
sens-platform. -
Add a local automatic SENS Ops Application in the
sens-opscluster. -
Add explicit remote SENS Cloud Applications to the
sens-opsArgo CD. -
Keep SENS Cloud Test automatic and SENS Cloud Production manual.
-
Select OCI chart versions instead of Git chart paths.
-
Store exact release versions and source revisions in separate owned files:
clusters/sens-ops/release.yamlinstallations/sens-cloud/test/release.yamlinstallations/sens-cloud/production/release.yaml -
Make one
sens-platform-infraworkflow the serialized writer of release files. Product and Ops publishers dispatch it with typed inputs. -
On a non-fast-forward push, refetch
main, regenerate only the owned release file, rerun validation, and retry the push. -
Keep Secret names, SecretStore contracts, and ExternalSecret resources in Git. Keep Secret values out of Git and CI artifacts.
-
Add validators that reject product workloads on
sens-ops, central Ops workloads onsens-cloud, cross-environment Secret references, invalid destinations, and version drift. -
Update the infrastructure verification skill and feature map for both clusters.
Gate: all charts and manifests render from a clean clone. The infrastructure verifier proves exactly one local Ops deployment, two remote product deployments, separated namespaces, and valid release identities.
Phase 6: prepare servers, network, and DNS
Section titled “Phase 6: prepare servers, network, and DNS”Owner: Samuel for access and external changes. AI prepares scripts and performs read-only checks after explicit authorization.
Record server facts
Section titled “Record server facts”- Complete the execution record in this plan.
- Confirm that each server has enough CPU, RAM, and storage for its assigned workloads plus operating reserve.
- Confirm that both servers use a supported Linux distribution and synchronized time.
- Confirm that the target disks and current development data may be erased.
- Record public and private connectivity without recording credentials.
Prepare firewall rules
Section titled “Prepare firewall rules”Implement the narrow target paths:
| Destination | Port | Allowed source |
|---|---|---|
| Both servers SSH | TCP 22 | Approved administrative network only |
| Public ingress | TCP 80 and 443 | Internet where the listed public endpoint requires it |
| SENS Cloud Kubernetes API | TCP 6443 | SENS Ops and approved administration only |
| SENS Ops Kubernetes API | TCP 6443 | Approved administration only |
| SENS Ops HTTPS services | TCP 443 | SENS Cloud, approved administration, and public routes as configured |
| Outbound DNS and time | Required DNS and NTP ports | Approved resolvers and time source |
| Outbound Git, registry, and ACME | TCP 443 | GitHub, GHCR, and certificate endpoints |
| Outbound MQTT and SMTP | Contract ports | Exact Netmore and mail endpoints only |
Do not expose NATS, PostgreSQL, Valkey, VictoriaMetrics, VictoriaLogs, Alertmanager, or internal controller ports publicly.
Prepare DNS
Section titled “Prepare DNS”Use these target assignments unless Samuel approves a hostname change:
| Hostname | Target server |
|---|---|
ops.iot-sens.schlossers.at | SENS Ops |
status.iot-sens.schlossers.at | SENS Ops |
argo.iot-sens.schlossers.at | SENS Ops |
secrets.iot-sens.schlossers.at | SENS Ops |
api.dev.iot-sens.schlossers.at | SENS Cloud |
test-frontend.dev.iot-sens.schlossers.at | SENS Cloud |
docs.dev.iot-sens.schlossers.at | SENS Cloud |
api.iot-sens.schlossers.at | SENS Cloud |
app.iot-sens.schlossers.at | SENS Cloud |
docs.iot-sens.schlossers.at | SENS Cloud |
Set a short TTL before the hard cut. Create A and AAAA records only for addresses that are actually routed to the server. Confirm forward resolution from an external resolver before requesting certificates.
Prepare reproducible bootstrap scripts
Section titled “Prepare reproducible bootstrap scripts”The AI adds reviewed, idempotent scripts under sens-platform-infra/scripts
for the two server roles. Each script must:
- require an explicit cluster role and pinned k3s version;
- print the selected disks, addresses, and role before changing the host;
- stop before destructive storage work unless the human passes a dedicated confirmation flag;
- enable k3s Secret encryption;
- disable unneeded bundled components explicitly;
- converge when it runs twice;
- wait for the local Kubernetes API and Node readiness;
- print only safe next steps.
Gate: firewall and DNS plans are reviewed, bootstrap scripts pass shell and disposable-host checks, and Samuel explicitly approves the two exact servers.
Phase 7: initialize the SENS Ops server
Section titled “Phase 7: initialize the SENS Ops server”Owner: Samuel runs approved server commands. AI guides and verifies after each step.
-
Install the supported operating-system updates.
-
Set the approved hostname and verify time synchronization.
-
Clone
sens-platform-infraat the reviewed commit. -
Run the reviewed SENS Ops bootstrap script with the pinned k3s version.
-
Verify the initial node:
Terminal window sudo kubectl get nodes -o widesudo kubectl get pods -A -
Install the pinned Argo CD bootstrap and its root Application.
-
Install cert-manager and the approved issuers.
-
Install OpenBao and initialize it as new state. Record recovery and bootstrap material only in the approved human-controlled location.
-
Configure OpenBao policies and authentication for Ops, SENS Cloud Test, and SENS Cloud Production.
-
Create the bootstrap Kubernetes Secrets listed by the generated Secret handoff.
-
Let Argo CD install ESO, Valkey, monitoring, logging, alerting, public status, and the first SENS Ops release.
-
Verify every namespace and Application:
Terminal window sudo kubectl get applications -n argocdsudo kubectl get pods -Asudo kubectl get pvc -Asudo kubectl get ingress -A -
Verify that the public status route contains no authenticated Ops route and that administrative routes use the approved private boundary.
Gate: SENS Ops is Healthy and Synced, the exact Ops version is visible, public status works, alerts can be delivered, and no product workload runs on this cluster.
Phase 8: initialize the SENS Cloud server
Section titled “Phase 8: initialize the SENS Cloud server”Owner: Samuel runs approved server commands. AI guides and verifies after each step.
- Install the supported operating-system updates.
- Set the approved hostname and verify time synchronization.
- Clone
sens-platform-infraat the reviewed commit. - Run the reviewed SENS Cloud bootstrap script with the same approved k3s version unless the compatibility contract records a deliberate difference.
- Verify the initial node with
sudo kubectl get nodes -o wide. - Create only the cluster-local bootstrap resources needed before remote Argo CD takes ownership.
- Restrict the Kubernetes API to SENS Ops and the approved administrative network.
- Register SENS Cloud in the Argo CD instance on SENS Ops with the smallest permissions required by the two product AppProjects.
- Create separate namespaces, identities, Secrets, PVCs, TimescaleDB instances, and NATS instances for Test and Production.
- Install namespace-scoped ESO and Reloader instances.
- Install only the cluster-local metrics and log agents that send bounded operational signals to SENS Ops.
- Keep central Argo CD, OpenBao, Valkey, VictoriaMetrics, VictoriaLogs, Alertmanager, and the Ops application off this cluster.
Gate: SENS Ops can read and reconcile the approved SENS Cloud namespaces. Test and Production cannot read each other’s Secrets or state. SENS Cloud has no central management workload.
Phase 9: deploy SENS Cloud and switch DNS
Section titled “Phase 9: deploy SENS Cloud and switch DNS”Owner: AI prepares GitOps changes. Samuel approves DNS and the first live synchronizations.
- Select the verified SENS Platform pre-release in the Test release file.
- Let Argo CD synchronize
sens-cloud-testautomatically. - Wait for migrations, NATS bootstrap, workloads, readiness, and version checks.
- Run the full synthetic telemetry verifier against isolated Test state.
- Select the approved stable artifact set in the Production release file.
- Inspect the Production diff in Argo CD.
- After explicit approval, synchronize
sens-cloud-productionmanually. - Verify its migrations, NATS bootstrap, workloads, readiness, and exact version.
- Point the approved DNS records to their target servers.
- Wait for public resolution and cert-manager certificates.
- Verify every public route, CORS rule, TLS chain, readiness endpoint, and version endpoint.
Gate: both product environments are Healthy and Synced at their selected versions. Every DNS name resolves to its assigned server and presents the expected certificate.
Phase 10: prove automation and failure boundaries
Section titled “Phase 10: prove automation and failure boundaries”Owner: AI prepares tests. Samuel approves live failure exercises.
Prove selective CI
Section titled “Prove selective CI”Create or use real reviewed changes that cover:
- documentation only;
- Ops only;
- one leaf product service;
- one shared package with several dependents;
- one product chart;
- one Ops chart;
- one root or unknown path that must select full validation.
For every case, retain the impact JSON and GitHub Actions job list as evidence.
Prove automatic Ops delivery
Section titled “Prove automatic Ops delivery”- Merge a real Ops change to
main. - Observe all Ops checks.
- Verify the immutable image and chart version.
- Verify the serialized
sens-platform-infrarelease commit. - Observe the automatic Argo CD rollout.
- Verify readiness and the live
SENS_OPS_VERSION. - Dispatch the same exact release selection again and confirm that it creates no second infrastructure commit.
- Rerun the release workflow and confirm that the new
ATTEMPTversion is published immutably, selected once, deployed, and reported live.
Prove failure separation
Section titled “Prove failure separation”After explicit approval:
- Stop SENS Ops application access and confirm that existing SENS Platform API, ingestion, NATS, TimescaleDB, and Kubernetes workloads continue.
- Restore SENS Ops and confirm that observation resumes.
- Stop the approved SENS Cloud test workload and confirm that SENS Ops reports the failure without losing its own availability.
- Restore Test and confirm that alerts resolve.
Gate: CI selects the correct work, automatic Ops delivery works from merge to live version, and the two servers exhibit the required failure behavior.
Phase 11: remove obsolete runtime and repository paths
Section titled “Phase 11: remove obsolete runtime and repository paths”Owner: AI prepares removals. Samuel approves destructive and external actions.
- Confirm again that no current data requires retention.
- Remove the old Coolify Ops deployment and its DNS targets.
- Remove the old in-cluster central Argo CD, OpenBao, and monitoring data plane from SENS Cloud if any remain.
- Remove obsolete
clusters/sens-k3s,environments/operations, Git-source chart, Compose Production, and old repository adapter paths. - Remove
sens-platform-opsfrom GitHub App repository lists and monitoring targets. - Archive the old
sens-platform-opsrepository after the monorepo build, release, and live Ops deployment no longer reference it. - Update the current service inventory, system overview, local development, release contracts, Secret handoff, operational runbooks, and Roadmap.
- Run full source, Ops, documentation, GitOps, and live verification.
Gate: only sens-platform and sens-platform-infra are required for source,
release, deployment, and recovery. Coolify and the old Ops repository are not
runtime or automation dependencies.
Phase 12: rehearse a clean customer-owned installation
Section titled “Phase 12: rehearse a clean customer-owned installation”Owner: AI uses disposable local infrastructure. No customer environment is contacted.
- Start an empty supported Kubernetes cluster.
- Pull the released
sens-platformOCI chart. - Supply existing Kubernetes Secrets instead of requiring SENS OpenBao.
- Install one isolated SENS Platform environment with customer-style values.
- Keep central SENS Ops connectivity disabled.
- Run migrations, NATS bootstrap, login, API, and synthetic telemetry checks.
- Destroy the disposable cluster and repeat the installation from the written instructions.
Gate: the same released chart installs twice from empty state without source changes, SENS-only infrastructure, or central runtime connectivity.
Customer go-live gate
Section titled “Customer go-live gate”The Greenfield rebuild deliberately carries no current development data. Before the first real customer data enters SENS Cloud or a customer-owned installation, approve and prove a separate persistence plan that covers:
- PostgreSQL full backup and WAL recovery;
- OpenBao snapshots and recovery material;
- k3s datastore recovery;
- required NATS recovery or documented replay;
- retention, RPO, RTO, alerting, and responsible roles;
- isolated restore exercises.
This later gate does not block the current two-server rebuild. It blocks real customer production.
Completion
Section titled “Completion”The plan is complete only when every phase gate has recorded evidence, the Roadmap describes the implemented state, and ADR-0012 no longer says that the target migration is pending.