tend/docs/WALKTHROUGH.md

Two-service build, approval, and deployment

This walkthrough assumes one small Linux host running systemd and Caddy, two Go services, and a trusted Gitea build runner. The host has the same root-owned Tend binary at /usr/local/bin/tend; it does not need Git or Go.

Prepare the host once

1. Create /etc/tend/services, /etc/tend/environment, and /var/lib/tend/incoming. The incoming and environment directories are root-owned mode 0700.
2. Install one schema-2 file per service, one per-service deployment-event log, and one root-owned mode-0600 environment file per service. A singleton's shared environment file must not define its configured listen key; its installed unit owns the live address and Tend overrides only the transient candidate. Give each singleton an imported Caddy handler and a root-owned handler template with exactly one {{UPSTREAM}} marker; this is the bounded traffic handoff while its fixed-address unit restarts.
3. Install the receive policy, forced authorized_keys entry, and exact sudoers rule from examples/server/ after replacing every placeholder. The sudoers fragment preserves only SSH_ORIGINAL_COMMAND; the root receiver requires its exact protocol value and refuses every other requested command.
4. Pin the server host key in a dedicated client file. Do not accept a new key interactively during deployment.
5. Run sudo tend check-server and inspect the allowlisted service names.
6. Run sudo tend reconcile --config /etc/tend/services/example-site.json --json before the first maintenance release. A settled report is expected; any retained candidate or unknown handler must be understood before traffic changes. The command is read-only.

Build and approve

Trusted CI checks a clean pushed commit, uses a pinned Go toolchain, packages twice, and requires byte-identical archives. It publishes the archive, RELEASE.json, BUILDINFO.json, SPDX SBOM, and SHA-256 evidence.

The maintainer reads the candidate report and copies the exact approved digest into the deployment command. Tend refuses a digest that is merely inferred from the local file or differs from the produced value.

Inspect the exact downloaded bytes before transfer. This is a complete, read-only artifact validation and does not stage a candidate:

tend inspect --config /review/example-site.json \
  --artifact /approved/example-site.tar.gz \
  --sha256 <produced> --approve-sha256 <reviewed>

On the host, compare Tend's journal with the conventional service state before and after activation. Reconciliation reports facts and never stops a unit, rewrites a pointer, or changes Caddy:

sudo tend reconcile --config /etc/tend/services/example-site.json

tend push --target tend-deploy@server.example \
  --known-hosts /secure/tend_known_hosts \
  --identity /secure/tend_deploy_ed25519 \
  --service example-site \
  --artifact /approved/example-site.tar.gz \
  --sha256 <produced> --approve-sha256 <reviewed> --activate

Repeat independently with --service docs-site. Both builds and transfers can run concurrently. The host-wide lock serializes only activation because both services share Caddy. Tend does not stop the other application.

Failure and recovery exercises

• Change a candidate marker: activation must fail, preserve the active release, and record the failed attempt without calling the candidate active.
• Make a Caddy template invalid: validation must fail and restore prior bytes.
• Make the public marker fail after an initially successful request: the activation window must catch the transient routed failure and restore the former slot/pointer.
• For a singleton, verify repeated canonical-origin requests remain successful while Tend routes to the candidate, restarts the fixed-address unit, and returns traffic to it. Inject failure at both Caddy reloads and require the prior handler and pointer to be restored.
• Stop the previous blue/green slot during the activation window: Tend must restore the old Caddy handler instead of accepting reduced continuity.
• Run tend rollback --activate for one service and verify the other service's units, pointers, and public origin did not change.
• Interrupt a transfer: no release becomes active and the incomplete incoming file is removed when the receiver exits.
• Interrupt singleton recovery after the candidate is proven. tend reconcile --json must name the operation-scoped candidate unit, report whether it is active and whether the handler file targets it, and perform no stop, restart, reload, pointer, or state mutation.

After the soak, prune per service. Active and previous releases remain protected. If activation fails, run reconcile before manual recovery so the retained candidate, route, release pointers, and journal disagreement are preserved in one bounded report.