fix: make kind rollout verification race-free

docs/local-hardening-plan.md

@@ -46,11 +46,12 @@ The goal remains open while any row lacks reproducible local evidence.
 - A forced LiveSocket transport loss while Chromium is offline exposes the
   reconnect alert, and restoring the network clears it and reconnects the
   authenticated LiveView.
-- The isolated API 30/34/37 Android matrix passes six device tests per API:
+- The isolated API 30/34/37 Android matrix passes seven device tests per API:
   missing and granted location permission, same-origin deep-link routing across
   Activity recreation, native foreground location upload, notification Stop
   with remote cleanup, a disconnected Stop request with visible retry state,
-  and continued uploads after Home and Activity destruction. A separate probe
+  continued uploads after Home and Activity destruction, and a forced
+  main-frame disconnect followed by native Retry and a rendered DOM. A separate probe
   externally kills the instrumentation process and verifies that the
   `START_NOT_STICKY` service and notification do not return. The suite uses an
   in-process loopback fixture, runs each emulator without external networking,
@@ -117,9 +118,9 @@ The goal remains open while any row lacks reproducible local evidence.
   retained no credentials, published no host ports, and removed its exact
   project, volume, network, and one-run images. FCM/APNs device registration is
   not claimed.
-- The final regression repeated the 167-test quality/security gate, isolated
+- The final regression repeated the 174-test quality/security gate, isolated
   browser 1/1 bootstrap plus 8/8 Chromium specs, Android debug/staging builds
-  and 5/5 API 37 device tests, the 50,000-row database benchmark, authenticated
+  and 7/7 device tests on API 30, 34, and 37, the 50,000-row database benchmark, authenticated
   HTTP/LiveView/WebSocket/tracking load, Compose crash/replacement and kind
   rolling drills, alert firing/resolution, encrypted backup/restore failure
   paths, and the external protocol boundary. Scoped fixtures and one-run

docs/performance.md

@@ -361,3 +361,30 @@ BEAM; the leaderboard itself uses composite keyset pagination.
 Ignored evidence for the recorded run:
 
 - `output/db-scale/20260719054559-2339191/`
+
+## Current final replay
+
+The same threshold-free profiles were replayed on 2026-07-20 against
+application commit `b96d443`.
+
+- The 30-second 3-web/2-worker load run used 40 public HTTP, 40 heartbeat
+  WebSocket, and 8 authenticated VUs. It completed 13,765 iterations and 38,826
+  HTTP requests; all 35,280 checks passed and HTTP failures were 0. The
+  authenticated scenario completed 1,765 page/LiveView/chat/tracking
+  start-update-stop chains, and the heartbeat scenario opened and answered 240
+  sockets. PostgreSQL recorded no rollback, deadlock, conflict, or temporary
+  file. The run-specific fixtures were removed and the cleanup count diff was
+  empty. Evidence is `output/performance/final-load-20260720`.
+- The database benchmark again used 50,000 configured rows per large table on
+  PostgreSQL 18.4. After migration, the asserted list queries selected their
+  cursor indexes; for example, observed execution time changed from 8.139 ms to
+  0.127 ms for urgent-help discovery and from 8.035 ms to 0.050 ms for
+  Activity discovery. These are single local observations, not thresholds or
+  portable performance claims. Evidence is
+  `output/db-scale/20260719233526-97476`.
+- The corrected kind rolling run replaced all four application pod UIDs,
+  retained four Ready zero-restart replacements, passed the four-node BEAM and
+  PubSub probes, and produced an empty database diff. All 395 readiness samples
+  ultimately returned HTTP 200; two required one retry during local NodePort
+  endpoint replacement. Evidence is
+  `output/resilience/final-kind-rollout-fixed-20260720`.

docs/verification.md

@@ -1,6 +1,6 @@
 # Who Need Help — implementation verification
 
-Observed through 2026-07-19 in the local workspace. This report separates observed
+Observed through 2026-07-20 in the local workspace. This report separates observed
 results from product limits and unknown production properties.
 
 ## Verified MVP capabilities
@@ -700,6 +700,74 @@ unrelated application records:
   Evidence is
   `output/regression/full-public-web-quality-20260719.log`.
 
+## Final Android and operational replay
+
+Application commit `b96d443` and its exact source state were subjected to the
+following additional local and temporary-origin checks:
+
+- `./scripts/quality.sh` passed all configured source, Compose, Helm,
+  observability, dependency, release-image, and security gates with 174/174
+  ExUnit tests. Evidence is
+  `output/regression/android-cross-quality-20260720.log`.
+- Android debug and staging unit/lint/APK builds passed. The isolated API
+  30/34/37 matrix passed 7/7 device tests per API. Public API 37 DOM smoke
+  passed 1/1. The Android/browser staging run passed Android magic-link login,
+  private chat in both directions, foreground tracking, browser marker
+  appearance/removal, fixture verification, and an empty 19-table cleanup
+  diff. Evidence is
+  `output/android-browser-staging-e2e/20260719232413-3987628`.
+- The external-boundary drill passed OAuth, SMTP, provider-neutral HTTP push,
+  product job integration, retry, rejection, timeout, replay, and
+  deduplication paths with two isolated workers. Its one-run project, volume,
+  images, network, and generated secrets were removed. Evidence is
+  `output/external-boundaries/final-boundaries-20260720`.
+- The isolated 3-web/2-worker Compose failure drill crashed one web and one
+  worker BEAM process, sequentially replaced every replica, rejoined all five
+  nodes, passed PubSub, and completed the injected Oban job on attempt 2 after
+  one recorded failure. All 725 readiness samples succeeded and the exact job
+  was deleted. Evidence is
+  `output/resilience/final-resilience-20260720`.
+- The 30-second, 88-VU load replay completed 13,765 iterations and 38,826 HTTP
+  requests. All 35,280 checks passed; HTTP failures were 0. It completed 1,765
+  authenticated LiveView page/chat/tracking start-update-stop chains and 240
+  heartbeat sockets without a functional error. PostgreSQL observed zero
+  rollbacks, deadlocks, conflicts, or temporary files. Exact fixture cleanup
+  produced an empty application-table diff. Evidence is
+  `output/performance/final-load-20260720`.
+- Prometheus exactly matched and scraped all three web targets before and after
+  the induced replica stop. Grafana's provisioned Prometheus datasource
+  returned `OK`; Alertmanager delivered firing and resolved webhooks for the
+  scoped instance. The database count diff was empty. Evidence is
+  `output/observability/final-observability-20260720`.
+- Restic created and checked an encrypted local S3 snapshot, restored it into a
+  fresh database, rejected a corrupted repository, and published no snapshot
+  for an interrupted upload. The interrupted repository passed checking after
+  pruning unreferenced data; the source database diff was empty. Evidence is
+  `output/backups-s3/final-backup-20260720`.
+- The 50,000-row database replay selected every asserted cursor index and
+  retained the same generated table counts before and after the index
+  migration. Its isolated project, volume, and image were removed. Evidence is
+  `output/db-scale/20260719233526-97476`.
+- A clean deployment from `git archive b96d443` generated independent secrets,
+  applied all 10 migrations, repeated migration without changing its 10
+  migration or 14 category counts, and passed HTTP, Mailpit, four-node cluster,
+  and PubSub checks on 2 web/2 worker replicas. Its workspace, image, project,
+  volumes, and networks were absent after cleanup. Evidence is
+  `output/portability/20260719233712-142771-ac1883`.
+- The project-owned kind cluster was rebuilt with the current image. The final
+  rolling drill replaced all four application pod UIDs with four Ready,
+  zero-restart pods, rejoined all four BEAM nodes, passed PubSub, and left an
+  empty database diff. All 395 readiness samples ultimately succeeded; two
+  used one transport retry during the local single-node NodePort endpoint
+  replacement. Evidence is
+  `output/resilience/final-kind-rollout-fixed-20260720`.
+
+The first 2026-07-20 kind attempt exposed a race in the verification harness:
+one label-based `kubectl wait` invocation retained a terminating pod in its
+initial resource set. The harness now waits for old UIDs to disappear, captures
+the exact four replacement pod names, and waits for those resources. The fixed
+full replay above passed.
+
 ## Known work before a public production launch
 
 - Replace the temporary staging origin with the production-owned domain and

scripts/kind-rolling-verify.sh

@@ -273,13 +273,6 @@ probe_pid=$!
 "${kube[@]}" rollout status "deployment/$WORKER_DEPLOYMENT" \
   "--timeout=$ROLLOUT_TIMEOUT" >"$output_dir/worker-rollout-status.txt"
 
-"${kube[@]}" wait \
-  --for=condition=Ready \
-  --timeout="$ROLLOUT_TIMEOUT" \
-  pod \
-  -l app.kubernetes.io/instance=who-need-help \
-  >"$output_dir/pods-ready.txt"
-
 old_pod_deadline=$((SECONDS + CLUSTER_JOIN_TIMEOUT))
 
 while ((SECONDS < old_pod_deadline)); do
@@ -310,6 +303,31 @@ if [[ "$retained_count" != "0" ]]; then
   exit 1
 fi
 
+mapfile -t current_application_pods < <(
+  "${kube[@]}" get pods \
+    -l app.kubernetes.io/instance=who-need-help \
+    -o json |
+    jq -r '
+      .items[]
+      | select(
+          .metadata.labels["app.kubernetes.io/component"] == "web" or
+          .metadata.labels["app.kubernetes.io/component"] == "worker"
+        )
+      | "pod/" + .metadata.name
+    '
+)
+
+if [[ "${#current_application_pods[@]}" -ne 4 ]]; then
+  echo "Expected four replacement application pods; observed ${#current_application_pods[@]}." >&2
+  exit 1
+fi
+
+"${kube[@]}" wait \
+  --for=condition=Ready \
+  --timeout="$ROLLOUT_TIMEOUT" \
+  "${current_application_pods[@]}" \
+  >"$output_dir/pods-ready.txt"
+
 deployment_snapshot "$output_dir/deployments-after.json"
 pod_snapshot "$output_dir/pods-after.json"
 database_snapshot "$output_dir/database-after.txt"