gamertan
bf56dbce0f
Export the reviewed allowlisted snapshot from private source commit 07c1655921f21ee5e4fc4d85639d199e8867b17d. This records the Docker Compose activation, schema-compatible rollback, and stateful migration resource findings from Observatory Preview 19 dogfooding. AI-Assisted: OpenAI Codex Signed-off-by: Cole Speelman <crspeelman@gmail.com>
99 lines
4.8 KiB
Markdown
99 lines
4.8 KiB
Markdown
# Two-service build, approval, and deployment
|
|
|
|
This walkthrough assumes one small Linux host running systemd and Caddy, two Go
|
|
services, and a trusted Gitea build runner. The host has the same root-owned Tend
|
|
binary at `/usr/local/bin/tend`; it does not need Git or Go.
|
|
|
|
## Prepare the host once
|
|
|
|
1. Create `/etc/tend/services`, `/etc/tend/environment`, and
|
|
`/var/lib/tend/incoming`. The incoming and environment directories are
|
|
root-owned mode `0700`.
|
|
2. Install one schema-2 file per service, one per-service deployment-event log,
|
|
and one root-owned mode-`0600`
|
|
environment file per service. A singleton's shared environment file must
|
|
not define its configured listen key; its installed unit owns the live
|
|
address and Tend overrides only the transient candidate. Give each
|
|
singleton an imported Caddy handler and a root-owned handler template with
|
|
exactly one `{{UPSTREAM}}` marker; this is the bounded traffic handoff while
|
|
its fixed-address unit restarts.
|
|
3. Install the receive policy, forced `authorized_keys` entry, and exact sudoers
|
|
rule from `examples/server/` after replacing every placeholder. The sudoers
|
|
fragment preserves only `SSH_ORIGINAL_COMMAND`; the root receiver requires
|
|
its exact protocol value and refuses every other requested command.
|
|
4. Pin the server host key in a dedicated client file. Do not accept a new key
|
|
interactively during deployment.
|
|
5. Run `sudo tend check-server` and inspect the allowlisted service names.
|
|
6. Run `sudo tend reconcile --config /etc/tend/services/example-site.json
|
|
--json` before the first maintenance release. A settled report is expected;
|
|
any retained candidate or unknown handler must be understood before traffic
|
|
changes. The command is read-only.
|
|
|
|
## Build and approve
|
|
|
|
Trusted CI checks a clean pushed commit, uses a pinned Go toolchain, packages
|
|
twice, and requires byte-identical archives. It publishes the archive,
|
|
`RELEASE.json`, `BUILDINFO.json`, SPDX SBOM, and SHA-256 evidence.
|
|
|
|
The maintainer reads the candidate report and copies the exact approved digest
|
|
into the deployment command. Tend refuses a digest that is merely inferred from
|
|
the local file or differs from the produced value.
|
|
|
|
Inspect the exact downloaded bytes before transfer. This is a complete,
|
|
read-only artifact validation and does not stage a candidate:
|
|
|
|
```text
|
|
tend inspect --config /review/example-site.json \
|
|
--artifact /approved/example-site.tar.gz \
|
|
--sha256 <produced> --approve-sha256 <reviewed>
|
|
```
|
|
|
|
On the host, compare Tend's journal with the conventional service state before
|
|
and after activation. Reconciliation reports facts and never stops a unit,
|
|
rewrites a pointer, or changes Caddy:
|
|
|
|
```text
|
|
sudo tend reconcile --config /etc/tend/services/example-site.json
|
|
```
|
|
|
|
```text
|
|
tend push --target tend-deploy@server.example \
|
|
--known-hosts /secure/tend_known_hosts \
|
|
--identity /secure/tend_deploy_ed25519 \
|
|
--service example-site \
|
|
--artifact /approved/example-site.tar.gz \
|
|
--sha256 <produced> --approve-sha256 <reviewed> --activate
|
|
```
|
|
|
|
Repeat independently with `--service docs-site`. Both builds and transfers can
|
|
run concurrently. The host-wide lock serializes only activation because both
|
|
services share Caddy. Tend does not stop the other application.
|
|
|
|
## Failure and recovery exercises
|
|
|
|
- Change a candidate marker: activation must fail, preserve the active release,
|
|
and record the failed attempt without calling the candidate active.
|
|
- Make a Caddy template invalid: validation must fail and restore prior bytes.
|
|
- Make the public marker fail after an initially successful request: the
|
|
activation window must catch the transient routed failure and restore the
|
|
former slot/pointer.
|
|
- For a singleton, verify repeated canonical-origin requests remain successful
|
|
while Tend routes to the candidate, restarts the fixed-address unit, and
|
|
returns traffic to it. Inject failure at both Caddy reloads and require the
|
|
prior handler and pointer to be restored.
|
|
- Stop the previous blue/green slot during the activation window: Tend must
|
|
restore the old Caddy handler instead of accepting reduced continuity.
|
|
- Run `tend rollback --activate` for one service and verify the other service's
|
|
units, pointers, and public origin did not change.
|
|
- Interrupt a transfer: no release becomes active and the incomplete incoming
|
|
file is removed when the receiver exits.
|
|
- Interrupt singleton recovery after the candidate is proven. `tend reconcile
|
|
--json` must name the operation-scoped candidate unit, report whether it is
|
|
active and whether the handler file targets it, and perform no stop, restart,
|
|
reload, pointer, or state mutation.
|
|
|
|
After the soak, prune per service. Active and previous releases remain protected.
|
|
If activation fails, run `reconcile` before manual recovery so the retained
|
|
candidate, route, release pointers, and journal disagreement are preserved in
|
|
one bounded report.
|