Record current production operations evidence

This commit is contained in:
SimpleTest 2026-08-21 00:27:58 +03:00
parent bb7eb58c8f
commit cd9a21af85
2 changed files with 52 additions and 1 deletions

View File

@ -705,7 +705,10 @@ Restic snapshot identifier, verification time, and `restore_verified: true`;
it contains no database rows, account identifiers, recipient addresses, or
credentials. A failed backup or failed restore never advances the heartbeat.
Install the daily user-systemd timer only after one manual `run` succeeds:
Install the daily user-systemd timer on the operator workstation only after one
manual `run` succeeds. The production host does not run this timer. The
workstation connects to production for the source dump and publishes the
restore-verified heartbeat to the independent repository host:
```bash
./scripts/install-production-backup-timer.sh

View File

@ -3217,3 +3217,51 @@ promoted.
Mailpit containers remained healthy and public readiness remained `ready`.
The remote repository, Caddy, production release, and frozen-test deployment
were unchanged.
# 2026-08-21 backup schedule and current production read-only load
- The operator workstation's user-systemd timer
`who-need-help-production-backup.timer` was loaded, enabled, active, and
waiting. It last triggered at 00:00 EEST on 21 August and is scheduled to
trigger again at 00:00 EEST on 22 August. The production host has no backup
timer or service. This matches the reviewed design: the workstation obtains
the source dump over SSH, sends the encrypted Restic snapshot to the separate
BuyVM host, and publishes the restore-verified heartbeat there.
- The scheduled service exited `0/SUCCESS` after 4 minutes 48.415 seconds. It
measured 2.294 seconds of local CPU time and a 50.8 MiB local memory peak.
PostgreSQL dump, checksum, catalogue, full repository data check, and the
isolated restore drill passed. The completed snapshot identifier is
`afdb5c5caa1df84203fc261817a153ade57d133a9102c501db10befc48fc4b77`.
- The independent monitor uses the operator-selected stale-heartbeat threshold
of 129600 seconds, or 36 hours. The current monitor state was `up`, with
readiness, metrics, and backup freshness all passing. This threshold does not
define retention, RPO, RTO, capacity, ownership, or encryption-key custody.
- A current read-only production load probe targeted exact revision
`bb7eb58c8f14d8936cae0e968b50ae721516d213` on the 2-CPU, 4,004,224-KiB host.
For 60 seconds it ran 20 HTTP virtual users against the fixed public GET
allow-list and 20 Phoenix WebSocket virtual users sending only heartbeat
frames. It performed 11,464 HTTP requests with zero failed checks and opened
20 of 20 sockets with 40 heartbeat replies and zero socket errors. HTTP
duration averaged 54.11 ms, p95 was 73.85 ms, and the maximum was 607.87 ms.
- During that probe, application-container CPU averaged 51.53% and peaked at
58.29%. Container memory peaked at 207,827,763 bytes. Host available memory
remained at or above 1,352,896 KiB. Readiness returned the exact ready payload
before and after the run. Counts for users, user tokens, and rate-limit
buckets remained `[16, 7, 20]`; PostgreSQL reported no new rollback, temporary
file, temporary byte, or deadlock count.
- This is a public read-only regression measurement, not a representative pilot
traffic model. It does not cover request creation, assignments, chat writes,
location updates, authentication delivery, support submission, push
delivery, or SMTP. No capacity limit or scaling threshold is inferred from
it. The non-secret evidence is retained under
`output/performance/production-current-bb7eb58-20260821/`.
- A headed production browser check reused the existing authenticated Chrome
session. The account menu exposed the expected Account settings route. The
route redirected to the reauthentication screen and displayed the explicit
message that reauthentication is required. No account setting was changed and
no new authentication email was requested.
- Current production metrics exposed bounded purpose labels rather than email
addresses. The observed process reported two successful
`support_confirmation` deliveries, two successful `mail` queue jobs, and no
delivery exception in the selected metric set. These counters describe only
the current application process, not historical delivery volume.