# Who Need Help — implementation verification

Observed through 2026-07-21 in the local workspace. This report separates observed
results from product limits and unknown production properties.

## Verified MVP capabilities

| Requirement | Status | Observed evidence | Limit |
| --- | --- | --- | --- |
| Urgent medicine-help flow | Implemented and tested | Request creation, discovery, matching, start, handover, two-party completion, and review rules are covered by the Phoenix test suite and exercised in the local UI. | The product coordinates pickup of an already purchased or reserved legal item; it is not a pharmacy, medical, or emergency service. |
| Urgent roadside help | Implemented and tested | Fuel, car wheel, bicycle, motorcycle, vehicle-breakdown, and secured-incident categories are seeded as a translated hierarchy. Server and LiveView tests exercise category paths, required fields, allowed values, boolean normalization, unknown-field rejection, and request creation. | Roadside requests require no immediate danger; this is not emergency response or professional recovery. |
| Extensible categories | Implemented and tested | Categories and validated text/select/boolean fields are stored in PostgreSQL. Proposal, vote, approve, reject, and merge paths have automated tests. | Coffee, cinema, hiking, and other social activities remain separate from urgent-help safety and ranking rules. |
| Separate Activity mode | Implemented and tested | Coffee, cinema, walk, and hiking categories use a separate activity lifecycle. Domain and two-client LiveView tests cover creation, join request, organizer approval, capacity enforcement, public/pending/chat privacy, exact-location disclosure to approved users, group chat, blocking, completion, and zero impact on helper reputation. Activity and message reports expose only the linked group conversation to an audited moderator; moderators can hide and restore reported activities. | This does not guarantee participant identity or physical safety. |
| Map and discovery | Implemented and browser-verified | The committed isolated Chromium suite rendered request and Activity maps, waited for MapLibre `idle`, and completed with no console, page, or request failures against a local PNG raster fixture. An earlier headed session rendered the configured OpenStreetMap tiles. | A production operator must configure a tile provider appropriate for its policy and traffic. |
| Private matched chat | Implemented and cross-client verified | A message sent from the helper browser appeared in the requester's browser without reload. An earlier Android emulator run also sent a message that appeared in the requester browser in real time. | There is no unsolicited general-purpose inbox. |
| Consent-driven live tracking | Implemented and cross-client verified | On API 37, Android started `TrackingService` as a location foreground service with a persistent Stop notification. After Home minimized the Activity, an emulator coordinate change reached PostGIS. Notification Stop removed the service, notification, active session, and raw position. | Browsers stop with the page. Android has no `ACCESS_BACKGROUND_LOCATION`, unattended start, or route history. |
| Privacy settings | Implemented and browser-verified | The profile exposed hidden, approximate public, exact for active match, and explicit exact-public options. Blocking and current-position cleanup have automated tests. | Exact public location remains a user opt-in; legal privacy and retention text still requires jurisdiction-specific review before launch. |
| Reputation and anti-abuse | Implemented at MVP level | Handover codes, two-party completion, double-blind reviews, unique-counterpart ranking, optional movement/proximity evidence, reports, blocks, abuse signals, and moderator audit paths have automated tests. | The system is not bot-proof and does not claim identity verification. No punitive numeric policy is enabled without measured and approved thresholds. |
| Account registration and sign-in | Implemented and browser-verified | Email registration sends a confirmation magic link and does not require a password. Confirmed users can keep using magic links or add a password in settings. Google OpenID Connect registration, sign-in, link, unlink, replay prevention, verified-email enforcement, and account-ownership rules are covered by the 260-test suite. A headed Chrome run completed registration, confirmation through Mailpit, password setup, logout, password login, and localized settings with zero console warnings or errors. | The local Compose environment captures email in Mailpit. A production SMTP relay and a real Google Web client are not configured or externally verified. |
| Social profiles | Manual links implemented; optional GitHub verification implemented and automated-tested | Manual links cannot set verification fields. The optional GitHub flow uses state, PKCE, a user-bound one-time session, unique provider ownership, and an audit record. The local protocol drill also performs real HTTP token/user exchanges without returning an access token to the application. | GitHub OAuth credentials are intentionally absent and are not required for registration or the help flow. The real external provider redirect/callback remains disabled and unverified; other providers remain manual/unverified. |
| Support and content removal | Implemented and browser-verified | Public support, account deletion, general removal, and TAKE IT DOWN forms create separate audited queues; private email status links verify public contacts; authenticated submissions use the account email; moderator-only operations can update status and notify verified contacts. TAKE IT DOWN accepts URLs/text only and records a 48-hour review due time. | The current product hosts no user media and does not claim TAKE IT DOWN coverage. Staffing, jurisdiction-specific legal classification, final retention rules, actual account erasure/export, and identical-media-copy handling remain operational/legal work. |
| Voluntary thanks | Implemented as an external optional link | A helper can expose an optional link after completion; the UI states that the platform does not process the payment. | The platform does not provide payments, escrow, refunds, tax reporting, or payment guarantees. |
| Android client | Local and public-staging clients implemented and emulator-verified | The native packages `org.whoneedhelp.mobile.debug` and `org.whoneedhelp.mobile.staging` launch the same authenticated LiveView app. Seven lifecycle, permission, deep-link, foreground tracking, recoverable main-page failure, notification-Stop, and Activity-destruction tests passed on each of API 30, 34, and 37. The API 37 staging smoke asserted the public home and Safety DOM over HTTPS. A run-scoped Android/browser staging test passed login, private chat in both directions, foreground tracking, live marker appearance/removal, and exact cleanup. | Production signing, Play Store publication, verified Android App Links, unattended/background-permission tracking, and iOS are not implemented. |
| Multiple web/worker instances | Implemented and locally failure/rollout-verified | The final isolated Compose drill passed BEAM crashes and sequential replacement with 3 web/2 worker replicas: all five nodes joined, PubSub passed, and 744/744 readiness requests succeeded. The project-owned kind cluster replaced all 2 web/2 worker pod UIDs under `maxUnavailable=0`; all four replacement pods joined, PubSub passed, and 363/363 samples ultimately succeeded. | Local PostGIS is a single instance. Production database HA, backups, and recovery are operator work and are not claimed complete. |
| Local observability | Implemented and protocol-verified | Pinned Prometheus scraped the exact 3 web and 2 worker targets with a file Bearer credential; Grafana provisioned a healthy datasource and ten-panel web/worker/BEAM/Ecto/Oban dashboard; Alertmanager delivered firing and resolved webhooks for an induced scoped replica stop. | Local delivery does not establish production retention, notification-provider reliability, on-call policy, or measured alert thresholds. |
| Encrypted local backup | Implemented and failure-verified | Pinned Restic streamed PostgreSQL custom format into pinned local MinIO with no host plaintext dump, passed full-data checking and a fresh-database restore, rejected a corrupted repository, and published no snapshot for an interrupted upload. The one-run MinIO project and volume were removed after retaining the non-secret evidence. | The drill proves the local mechanism, not off-site durability, database HA, or a production RPO/RTO/retention policy. |
| External protocol boundaries | Implemented and locally failure-verified | The production release used its configured Assent/Req and Swoosh/gen_smtp clients against internal-only mocks. GitHub OAuth, Google OIDC discovery/authorization/token/JWKS with nonce and PKCE, and SMTP success/rejection/retry/replay/timeout paths passed. The HTTP push boundary passed disabled, retry, rejection, timeout, and idempotency paths. Request acceptance and new-chat transactions created durable jobs processed by two Oban worker replicas; the chat event completed on Oban attempt 2 after an injected temporary failure. | This does not verify real external provider availability or device delivery. Google/GitHub production clients, production SMTP, FCM/APNs token registration, and provider selection remain external work; SMTP exactly-once delivery is not claimed. |

## Reproducible checks

- On 2026-07-21, `./scripts/test.sh` and the Dockerized `mix precommit` each
  passed 273 tests after the support/content-removal implementation. The full
  isolated `./scripts/quality.sh` gate passed compiler, xref, Credo, Sobelow,
  Dialyzer, Hex/npm audits, Compose/Helm validation, and configured source/image
  scans. Headed Chrome over `https://whoneedhelp.imalto.site` verified public
  support acknowledgement and contact verification, general removal intake,
  the separate urgent TAKE IT DOWN form and 48-hour due time, passwordless
  account confirmation, admin-only queues, audited operator response email,
  account-deletion navigation, Russian localization, selected-status
  preservation, and zero browser-console errors or warnings.
- On 2026-07-20, after adding Google OpenID Connect authentication,
  `./scripts/test.sh` and `mix precommit` each passed 260 tests with zero
  failures. The full isolated `./scripts/quality.sh` gate also passed format,
  compilation with warnings as errors, xref, Credo, Sobelow, Dialyzer, Hex and
  npm audits, Compose/Helm rendering, environment validation, and configured
  HIGH/CRITICAL source/image scans.
- The Google boundary drill used the production Assent adapter against a local
  OIDC server with discovery, authorization, token, signed ID token, JWKS,
  state, nonce, PKCE, unverified-email rejection, and one-time callback replay
  checks. It passed together with GitHub OAuth, SMTP, push, and two-worker Oban
  paths. Evidence is retained at
  `output/external-boundaries/google-auth-fixed-20260720`.
- Headed Chrome verified the public temporary HTTPS origin through the
  email-only registration form, Mailpit confirmation link, one-time login,
  password creation, logout, password login, Russian locale selection, and
  Google connection settings. The configured Google credential pair is empty,
  so the UI correctly left Google actions disabled. Browser console inspection
  reported zero errors and zero warnings. The run-owned account and its one
  cascading login token were removed after read-only relationship checks; no
  help, activity, message, tracking, review, or OAuth rows belonged to it. The
  browser was left open.
- The isolated Phoenix suite completed on 2026-07-19 with 172
  tests and 0 failures after cursor pagination, database aggregation, and the
  full localization changes
  on Elixir 1.20.2 and Erlang/OTP 29.0.3.
- `mix compile --force --warnings-as-errors` and
  `mix format --check-formatted`: passed against the same final source.
- `./scripts/quality.sh` passed ShellCheck 0.11.0, Hadolint 2.14.0 at warning
  threshold, actionlint 1.7.12, every configured Compose profile render, Helm
  lint, Trivy source/rendered-manifest scanning, xref, Credo high-priority
  checks, Sobelow strict/private checks, Hex audit, 186 Phoenix tests, both npm
  audits, and the
  backup/MinIO/mc/external-mock/release-image vulnerability scans. The rendered
  Helm manifest reported zero HIGH/CRITICAL misconfigurations; the Alpine
  backup, MinIO, mc, and external-mock images and the Debian 13.6 release image
  each reported zero HIGH/CRITICAL vulnerabilities under the configured gates.
  The backup binary reported
  Restic 0.19.1 compiled with Go 1.26.5; MinIO and mc reported their pinned
  commits and Go 1.26.5.
- Dialyzer passed with three path- and warning-specific documented filters and
  zero unused filters. Two findings are opaque-type warnings at Ecto
  `Multi`/`MapSet` call sites; the third is the generated Gettext backend's
  three-form plural-rule opaque-value warning;
  the filters are visible in `.dialyzer_ignore.exs` rather than hidden by a
  broad pattern. This is not a claim that third-party dependency compilation is
  warning-free.
- The generated Activity migration was rolled back by exactly one step and
  migrated forward again against `who_need_help_test`; both directions passed.
- The Activity-report migration was also rolled back and migrated forward. The
  observed database constraint changed from exactly one of 3 urgent-help
  targets to exactly one of 5 urgent-help/Activity targets.
- `mix format --check-formatted`: passed in the final run.
- Android local Docker build targets `testDebugUnitTest`, `lintDebug`,
  `assembleDebug`, and `assembleDebugAndroidTest` passed. The isolated runners
  then passed 7/7 instrumentation tests on API 24, API 30, API 34, and API 37.
  Each API also passed the separate external process-death probe. The
  sixth test verifies that the foreground service survives Home plus Activity
  destruction and remains user-stoppable. The seventh forces the first
  main-frame request to disconnect, checks the native recovery dialog, presses
  Retry, and asserts the subsequently rendered WebView DOM. The
  public-staging target passed unit tests, `lintStaging`, `assembleStaging`,
  and `assembleStagingAndroidTest`; the API 37 staging smoke asserted the
  rendered `/` and `/safety` DOM from the temporary HTTPS origin with no
  observed load/TLS errors. Evidence is retained at
  `output/android-instrumentation/api24/20260720134554-2360359`,
  `output/android-instrumentation/api30/20260720135239-2511867`,
  `output/android-instrumentation/api34/20260720133058-2058554`,
  `output/android-instrumentation/api37-0/20260720133350-2117132`, and
  `output/android-staging-smoke/20260720052802-802174`.
  The AVD configuration requests a 1 GiB userdata partition. Emulator 36.6.11
  nevertheless checked for approximately 7.37 GiB while initializing API 34;
  two attempts exited before ADB because only 6.1-6.3 GiB remained in the
  container writable layer. The runner now mounts the AVD directory on an
  isolated run-scoped Docker volume, verifies that the emulator remains
  running before waiting for ADB, and removes both the container and volume on
  exit. The complete four-API matrix passed with that layout, and a post-run
  check found no retained `who-need-help-android-avd-*` volumes.
- Release Android guard: a staging/release build with a non-HTTPS
  `WNH_BASE_URL` failed at its dedicated preflight; the successful staging
  build used the explicit temporary HTTPS origin.
- Helm lint, template rendering, server-side dry-run, rollout waits, readiness
  checks, and cross-node PubSub verification passed in the local kind cluster.
- The hardened kind rollout then ran PostGIS as UID/GID 70 and Mailpit as
  UID/GID 10001. Both used read-only root filesystems, dropped all Linux
  capabilities, disabled privilege escalation, and used RuntimeDefault seccomp.
  Both web and both worker replicas were Ready; their migration-wait init
  containers also used read-only root filesystems and dropped capabilities.
  The cross-node probe observed all four BEAM nodes, and live/readiness returned
  `ok`/`ready`. Database counts remained `0 users / 0 help requests`.
- The final isolated Compose resilience drill observed a restart count increase
  for one crashed web and worker BEAM process, replaced every replica
  sequentially, observed all 5 cluster nodes, passed PubSub, and completed a
  real Oban retry on attempt 2 after one recorded error. Its exact job row and
  all fixture domain rows were absent afterward; all 744 readiness samples
  succeeded.
- The final reproducible kind rolling drill replaced all four pod UIDs,
  observed all 4 BEAM nodes, passed PubSub, and left the database-count diff
  empty. All 363 readiness samples ultimately returned 200; two samples needed
  one reconnect attempt each during local single-node NodePort endpoint
  replacement.
- The final local observability drill matched all 3 web and 2 worker
  Prometheus instance targets, checked the provisioned Grafana datasource and
  ten-panel dashboard, received firing and resolved Alertmanager webhooks for
  the exact stopped/recovered web replica, restored every target to `up`, and
  left the application-table count diff empty. Its retained evidence contains
  neither the metrics token nor the random Grafana password. Final evidence is
  retained at
  `output/observability/queue-worker-metrics-fixed-20260720`.
- The observability stop command changed only the four scoped monitoring
  container states. All app/database/worker container IDs stayed unchanged and
  running, and the checked user/request/message counts were identical before
  and after. A subsequent full drill returned all monitoring services to
  healthy.
- The final encrypted S3 drill restored 23 public application tables, all
  10 current migrations, 14 categories, and PostGIS 3.6.4 into a fresh
  temporary database. `restic check --read-data` passed before cleanup.
  Flipping the first byte of an isolated cloned repository made
  both `check` and `dump` exit nonzero. Stopping the exact backup container
  after uploaded encrypted objects produced exit 130 and zero snapshots;
  Restic identified and pruned 32.809 MiB of unreferenced packs, then passed a
  second full-data check. The source database count diff was empty and no
  configured secret appeared in retained evidence. The scoped MinIO project,
  repository volume, and runtime directories were subsequently removed; the
  evidence remains at
  `output/backups-s3/final-backup-current-20260719`.
- The latest external-boundary drill passed real client-side OAuth token and
  user HTTP exchanges, PKCE/state checks, provider denial, one-time-code replay,
  a fresh flow after a temporary token error, and timeout failure. It also
  passed SMTP acceptance, permanent rejection without retry, one retry after a
  temporary greeting, timeout, and repeated submission. The push checks passed
  the disabled adapter, rejection, HTTP retry, replay and ambiguous-timeout
  deduplication, then created real acceptance/chat events against an ephemeral
  PostGIS database. Two worker replicas processed them; acceptance completed on
  attempt 1, chat completed on Oban attempt 2 after a temporary provider
  failure, replay never reached HTTP, and the chat text was absent from the
  payload. Every retained file is mode `0600`, no generated credential was
  retained, no host port was published, and the exact project, volume, network,
  and one-run images were absent after cleanup. Evidence is retained at
  `output/external-boundaries/final-boundaries-20260719`.
- The committed browser suite passed its 1/1 bootstrap and all 30/30 scenarios:
  10 each in Chromium, Firefox, and WebKit, against a fresh PostGIS volume with
  two web and two worker replicas on 2026-07-20. The retained successful-run
  artifact directory is `output/e2e/20260720042117-3279852`.
  It covered axe checks across four pages and two themes, keyboard navigation,
  three responsive widths, Activity/moderation/privacy, the two-user medicine
  handover and blind reviews, public boundaries, local raster maps, and an
  offline LiveSocket transport failure/reconnect. Each browser also recovered
  active chat and tracking after its serving BEAM node stopped. It verified
  persistent Russian/Ukrainian public language selection and the selected
  Ukrainian locale inside an authenticated LiveView. The uniquely named
  Compose volume and networks were removed.
- The default localization domain contains 508 current messages and the errors
  domain contains 40. Russian and Ukrainian catalogs have no empty or fuzzy
  current entries. Category names, descriptions, structured-field labels and
  select/boolean values use locale maps with an English fallback; the
  compatibility `description` column remains as a fallback for pre-existing
  categories.
- The isolated database-scale harness migrated an empty PostgreSQL 18.4
  database to the exact pre-index version, seeded the configured canonical
  sample, captured JSON `EXPLAIN (ANALYZE, BUFFERS)` plans before and after the
  generated cursor-index migration, and removed its project and volume. The
  final 50,000-row-per-large-table run selected every asserted cursor index;
  two consecutive request pages matched the first 50 ordered rows with no gap
  or duplicate. Exact observations and their non-SLO limitations are recorded
  in `docs/performance.md`.
- Browser verification used the required headed Chrome wrapper. The final
  public homepage rendered its account/language controls, urgent-medicine hero,
  help steps, Safety link, and three language choices. The browser reported
  zero console errors or warnings; the document, fingerprinted CSS/JS, and logo
  requests all returned HTTP 200. The browser was left open.

Local generated evidence (ignored by Git):

- `output/playwright/final-request-map-chat.png`
- `output/playwright/final-privacy-profile.png`
- `output/android/final-image-smoke.png`
- `output/android/foreground-notification.png`
- `output/android/staging-deep-link-safety.png`
- `output/android-instrumentation/20260719053610-2122819/results.txt`
- `output/e2e/20260719053436-2083768/`

Android artifact:

- `android/dist/who-need-help-debug.apk`
- SHA-256:
  `4520aa0b50eaf53bb7052e7f4c096a73d456f45781380187586c79d7267c21c5`
- `android/dist-staging/who-need-help-staging.apk`
- SHA-256:
  `08d990a9268a382052be5f2d3ba5afba2aa65a9ad5f9ae1ad4bff720f8999a6e`
- Observed staging manifest values: package
  `org.whoneedhelp.mobile.staging`, version `0.1.0-staging`, minimum SDK 24,
  target and compile SDK 37, cleartext traffic disabled, `singleTop`
  `MainActivity`, and an exact HTTPS host of `whoneedhelp.imalto.site`.
- `apksigner verify` accepted the staging artifact's v2 generic debug
  signature. That signing identity is only for installable staging verification
  and is not a production release identity.

## Configuration finding

`WNH_DEBUG_BASE_URL` is a required Android build input, because the trusted
same-origin value is compiled into the debug APK. Its concrete local value is
read from the repository's ignored `.env` by `scripts/android-build.sh` and is
passed to Docker with `--build-arg`. `android/Dockerfile` only declares
`ARG WNH_DEBUG_BASE_URL`; it contains no URL default.

The example `http://10.0.2.2:4010` remains only as unit-test data for origin
matching. It is not a runtime or build default. The ignored `.env` is also
excluded from the root Docker build context.

## Database observations

The post-roadside local Compose observation found 2 users, 1 help request, 7
messages, and 1 assignment before and after deployment. Category seeding
changed only the category count from 1 to 9: 7 selectable categories and 2
inactive grouping nodes. These are local scenario data; the database is not
assumed empty.

The Activity migration then preserved those same user/help/message/assignment
counts, added empty `activities`, `activity_participants`, and
`activity_messages` tables, and expanded the category tree to 14 rows: 9 help
rows and 5 Activity rows. A validated custom-format dump was created immediately
before that migration at
`output/backups/compose-before-activity-20260718-203930.dump`; this ignored local
artifact is not a substitute for the still-required isolated restore drill.

The final kind observation found 0 users, 0 help requests, and 0 messages. That
state was queried before changing the database workload. The migration from the
old `emptyDir` Deployment created a custom-format dump, validated it with
`pg_restore --list`, changed PostGIS to a persistent StatefulSet, and restored
the dump before the Helm upgrade. The final kind database reported PostgreSQL
18.4 and PostGIS 3.6.4.

The local kind Secret contains independently generated values. Only their
decoded lengths were inspected: the PostgreSQL password is 64 characters and
each application secret is 128 characters. Secret values were not printed or
written to tracked files.

Before the local `auth_identities` migration, a PostgreSQL custom-format dump
was created at `output/backups/compose-20260720-203913.dump`; its SHA-256 is
`ed8d663b114f3823479198005857af5501a90dacbec3f0855296327ee9cd6d20`.
An isolated restore drill read 23 public tables, 1,488 rows, 12 applied
migrations, and PostGIS 3.6.4 from that dump, then removed the temporary
database. The source Compose database subsequently reported 13 applied
migrations and an empty `auth_identities` table before browser verification.

## Dependency-upgrade observations

- The running Compose and kind releases reported Elixir 1.20.2 and Erlang/OTP
  29.
- Compose reported Traefik 3.7.8 and Mailpit 1.30.4; Compose and kind both
  reported PostgreSQL 18.4 and PostGIS 3.6.4.
- The production asset build reported Tailwind CSS 4.3.3 and daisyUI 5.6.18;
  esbuild is configured at 0.28.1.
- The isolated Node build stage reported Node.js 24.18.0, npm 12.0.1,
  `npm outdated --json` returned `{}`, and `npm ci` reported zero known
  vulnerabilities.
- The Android build used Android Gradle Plugin 9.3.0, Gradle 9.6.1, Command-line
  Tools 22.0, and its embedded Android CLI 1.0.15857036. Unit tests, lint, and
  debug assembly passed; the lint report says “No errors or warnings.” The
  rebuilt emulator image contains the API 37 ps16k AVD and APK, and its emulator
  36.6.11.0 binary returned its version successfully at runtime.
- The final complete `scripts/kind-up.sh` run was idempotent and finished with
  both 2-replica Deployments Ready and a successful cross-replica PubSub probe.

The migration `20260718114233` and reverse block lookup index
`blocks_blocked_id_blocker_id_index` were present. An actual `EXPLAIN ANALYZE`
for that reverse lookup selected the index. The table contained no block rows,
so this confirms query shape, not production performance.

Exact production capacity, minimum CPU/RAM, and scaling thresholds are unknown:
there is no representative production dataset, traffic model, or
target-environment measurement. The isolated measurements in
`docs/performance.md` include the earlier paced public/readiness/heartbeat
comparison and a later 3-web/2-worker run with real authentication, LiveView,
chat writes, tracking writes, and explicit position deletion. The latter
completed 1,888 authenticated chains and 240 heartbeat sockets without a
functional failure and restored every tracked application table count after
fixture cleanup. Neither short run found a saturation point or represents
production traffic, so the Helm chart does not invent resource limits or an HPA
policy.

The 2026-07-20 memory investigation did identify and remove a development
runtime artifact: kind's nested container runtime exposed a huge `nofile`
limit, causing each OTP VM to allocate a roughly 1.5 GiB port table. Explicit
`+Q 65536` configuration reduced the entire kind container from 9.748 GiB to
1.796 GiB. A later rolling replacement retained that value in all four pods,
left the application-table count diff empty, and passed readiness and
cross-node PubSub. This fixes that specific over-allocation; it is not a
production minimum-RAM measurement. Detailed evidence and per-container values
are recorded in `docs/performance.md`.

The optional kind and load verification environments were stopped after their
final checks without deleting their containers or volumes. Immediately before
the pause, both isolated databases contained zero users, requests, and
messages; every kind pod was Ready. The stopped kind control plane had been
using 1.706 GiB, while the running load application/database/monitoring
containers used roughly another 1.6 GiB in the same observation. A complete
resume test measured the Kubernetes API Ready after 6.83 seconds, then
`kind-up.sh` reconciled the chart, rolled both 2-replica Deployments, passed
four-node PubSub and the BEAM port-limit checks, and `kind-stop.sh` paused it
again. The ordinary public Compose route remained HTTP 200 throughout.

## Public staging observation

On 2026-07-18, `whoneedhelp.imalto.site` was published through the existing
Ubuntu Nginx gateway and its OpenVPN path to the local Compose proxy. HTTP
redirected to HTTPS; the homepage, fingerprinted assets, and both health
endpoints returned HTTP 200. A headed Chrome session rendered the public page
with zero console errors or warnings. A complete WebSocket Upgrade request to
`/live/websocket` returned `101 Switching Protocols`.

The observed Let's Encrypt certificate had the correct
`DNS:whoneedhelp.imalto.site` SAN and an expiry of 2026-10-16. Nginx and the
Certbot renewal timer were active. This verifies the current staging path; it
does not make the workstation or gateway a production availability
environment.

Earlier on 2026-07-19 a read-only check resolved the origin to `77.110.101.144`;
HTTP redirected to HTTPS, and the homepage, Safety page, and readiness endpoint
returned HTTP 200 with successful TLS verification. The certificate still had
the exact SAN and the same expiry. The backing ordinary Compose project was
healthy and its four-node PubSub probe passed. It contained 2 users, 1 help
request, 7 messages, and 8 applied migrations. At that checkpoint no mutation
was performed before a validated backup/restore and explicit authorization.
The later authorized rollout is recorded below; this paragraph is historical,
not the current deployment state.

A mode-`0600` custom-format backup of that exact database was then created at
`output/backups/compose-20260719-060810.dump` with SHA-256
`726cca11f69b1aab5e7141ba5f13ebe92c7f27c6b14172891cd2e462acc069f0`.
Its checksum and `pg_restore` catalog were validated. The restore drill read 23
public tables and 1,484 rows from a uniquely named temporary database, observed
PostGIS 3.6.4, applied all 10 current migrations there, passed migration
readiness, and removed that database. A final read-only source check still
reported 8 migrations and the same user/request/message counts; the four
running application container IDs were unchanged.

The same archive then passed
`./scripts/upgrade-rehearsal-compose.sh` against application commit `af9018f`.
The isolated restored copy advanced from 8 to all 10 current migrations and
reported all 11 cursor indexes valid. Two web and two worker replicas formed a
four-node BEAM cluster, the cross-node PubSub probe passed, public requests
produced the expected production HTTPS redirects, trusted-proxy pages and both
health endpoints returned HTTP 200, and the application-table count diff was
empty. The exact project, database volume, networks, and one-run image were
absent after cleanup. Evidence is retained at
`output/upgrade-rehearsal/20260719062507-3274364`. The ordinary Compose
container IDs, 8-migration state, and user/request/message counts remained
unchanged.

After explicit approval to update the development Compose project, application
commit `143e159` was deployed on 2026-07-19. Immediately before the rollout, a
new mode-`0600` custom-format backup was created at
`output/backups/compose-20260719-093528.dump`; its SHA-256 is
`fd5f293c63972b19af9bc967ea229cc2617c17ad32d5fcd695f2dafafef4d790`.
The archive passed the isolated restore drill before any source-database
migration.

The ordinary Compose migration job then advanced the source database from 8 to
all 10 current migrations and started two web and two worker replicas from
image
`sha256:6685247b8e872ade5e36f2b9e88bfca09b1606823ffb3e4f1469fbcfdc15d48c`.
The complete before/after application-table snapshot had an empty diff,
including the existing 2 users, 1 request, 7 messages, 1 assignment, and 4
ended tracking sessions. Post-rollout catalog checks found the localized
category-description column and all 11 valid cursor indexes. Local live and
readiness checks, the four-node cluster/PubSub probe, public home, Safety,
live/readiness, TLS verification, and a WebSocket Upgrade all passed. The
checked web/worker log window contained no application error, warning,
exception, deadlock, or HTTP 5xx match.

Headed Chrome then rendered the updated Safety page in English and Russian,
redirected the protected Requests route to the Russian login form with the
expected authorization notice, and reported zero console errors or warnings.
The visible browser was deliberately left open. Mode-`0600` rollout evidence
is retained under
`output/staging-rollout/20260719093528-3223196/`; the browser screenshot is
retained under ignored `.playwright-cli/`.

After the Activity rollout, headed Chrome rendered the updated public
navigation, followed the Activity link to the authenticated route, and received
the expected login redirect and flash with zero console errors or warnings.

After commit `307794c`, staging was rebuilt with the optional GitHub OAuth
boundary. Both web and both worker replicas started, the cross-node PubSub probe
passed, public home/readiness returned HTTP 200, and the before/after database
counts remained `2 users / 1 request / 7 messages / 14 categories / 1
assignment / 0 activities / 0 reports / 0 social identities`.

A headed Chrome E2E run then registered a uniquely named `example.invalid`
account, consumed its confirmation link from local Mailpit, authenticated,
opened `/profile`, and observed:

- the GitHub network option and the explicit provider-disabled explanation;
- no Verify button while both OAuth environment variables were absent;
- a manually added Telegram link labelled `unverified`;
- removal of that owner-controlled manual link;
- a direct `/auth/social/github` request returning to the profile with the
  controlled `provider is not configured` flash;
- zero browser console errors or warnings.

The exact E2E user was checked for every foreign-key relationship to `users`.
It had only its one authentication token and no domain records after the manual
link was removed. That exact user was then deleted through Ecto, its browser
session became invalid, the generated Mailpit message was deleted individually,
and the original database counts were observed again. The browser windows were
left open.

Local ignored browser evidence:

- `.playwright-cli/page-2026-07-18T18-09-44-303Z.png`

## Public Android and cross-client observation

The public-staging APK was built with the explicit ignored `.env`
`WNH_BASE_URL`, installed on the API 37 emulator, and connected to
`https://whoneedhelp.imalto.site`. Android package inspection found the exact
configured HTTPS authority and no filter for `attacker.example`. Because the
manifest intentionally does not claim App Link verification, a normal same-host
implicit `ACTION_VIEW` may show Android's resolver. Selecting the application
opened its native `MainActivity`; both `/` and `/safety` loaded from the public
origin with no observed WebView load or TLS errors. The external origin
resolved to the browser rather than the app.

The manifest intentionally sets `android:autoVerify="false"`: this is a
same-origin HTTPS deep link, not a claimed verified Android App Link. Enabling
verification requires the final production application ID and signing
certificate fingerprint to be published in the deployment's
`/.well-known/assetlinks.json`.

A cross-client scenario then used headed Chrome as the requester and the
public-staging Android app as the helper. The observed behavior was:

- the Android app authenticated against the public HTTPS origin and rendered
  the MapLibre request;
- chat messages travelled in both directions, including an Android-bound
  browser message appearing without reload;
- Android granted foreground location permissions, started
  `TrackingService`, and showed its persistent Stop notification;
- after Home minimized the Activity, an emulator location change from
  `50.46009833, 30.5334` to `50.4611, 30.5344` reached PostGIS and produced
  `121.97` metres of observed movement;
- the headed browser received the helper marker in real time and removed it
  when tracking stopped;
- notification Stop removed the Android service and notification, ended the
  exact tracking session, left zero active sessions, and deleted its raw current
  position.

The final reproducible cross-client runner performed the same boundary through
the staging instrumentation APK and containerized Chromium. Android consumed a
run-scoped one-time login token, chat messages crossed in both directions
without reload, Android foreground tracking produced a live marker in the
browser, and stopping it removed the marker. The fixture verifier found exactly
the two expected messages, one ended tracking session, and no raw current
position. Cleanup removed only the UUIDs recorded in the run manifest; the
before/after count diff across 19 application tables was empty. Browser,
instrumentation, fixture verification, and cleanup all passed. Evidence is
`output/android-browser-staging-e2e/20260720053109-872921`.

The browser scenario exposed one UI defect: movement evidence was persisted and
the marker updated, but the evidence badge remained stale until reload. The
tracking PubSub event now carries the already-derived movement/proximity
timestamps, and the LiveView updates its in-memory assignment from that same
event without an extra query. Domain and two-client LiveView regression tests
assert the event and badge change.

That fix was deployed with image digest
`sha256:1b991c1e07d98babb5151c010250cd0b9458195cf3de5a91972456ee72d75673`.
Both web and both worker containers used the digest, the cross-node PubSub probe
passed, local and public readiness returned HTTP 200, and no application
error/warning was found in the checked post-rollout logs. The before/after
database counts remained `2 users / 1 matched request / 7 messages / 14
categories / 1 assignment / 4 ended tracking sessions / 0 active tracking
sessions / 0 activities / 0 reports / 0 social identities`.

The exact temporary E2E users, tokens, request, assignment, messages, tracking
session, and audit records were removed in one scoped transaction. Queries by
their exact identifiers found no remaining rows and the original database
counts were restored. The headed browser windows and emulator were left open.

## Operations and metrics verification

On 2026-07-18, the Compose database was archived with PostgreSQL 18
custom-format `pg_dump`. The published archive and SHA-256 sidecar were
validated before an isolated restore. The restore drill created a database from
`template0`, restored with `pg_restore --exit-on-error`, read 23 public
application tables and 1106 restored rows, reported PostGIS 3.6.4, observed all
8 current migrations, and removed the exact temporary database. A follow-up
catalog query returned zero remaining restore-drill databases. Source counts
before and after remained `2 users / 1 help request / 7 messages / 14 categories
/ 1 assignment / 0 activities / 0 reports / 0 social identities`.

The same rollout started 2 healthy web replicas and 2 worker replicas. Local
live/readiness, public HTTPS readiness, and the public root returned HTTP 200.
The cross-node PubSub probe passed across all four connected BEAM nodes. The
protected metrics route returned HTTP 401 both locally and through public HTTPS
without credentials, returned valid Prometheus text with the generated ignored
local token, and was scraped directly from each web container. Helm 4.2.3 lint
and template rendering passed.

Before the runtime image change, every BEAM container logged that
`libsctp.so.1` was unavailable. The rebuilt Debian trixie release contains the
`libsctp1` package and the exact shared library; no SCTP, application error, or
application warning appeared in the post-rollout web/worker logs checked during
this verification window.

## Final full-stack regression

The broad pre-production regression below tested application commit `c6aa3d1`.
The later responsive-header follow-up at `768d63a` was separately subjected to
the complete quality and browser suites, deployed, and verified as recorded
after this section.

- `./scripts/quality.sh` passed the complete static, formatting, compiler,
  xref, Credo, Sobelow, Dialyzer, dependency-audit, manifest, secret, and image
  gates. ExUnit reported 172 passed. The final log is
  `output/regression/final-20260719/quality-current.log`.
- The isolated browser project passed its bootstrap 1/1 and 27/27 scenarios:
  9 in each of Chromium, Firefox, and WebKit, with no skipped, flaky, or
  unexpected result. The suite includes accessibility, responsive navigation,
  medicine handover, blind reviews, Activity privacy/moderation, localization,
  reconnect, and active chat/tracking recovery after the serving BEAM node
  stops. Evidence is `output/e2e/20260719162432-2790071`.
- Android debug and staging unit/lint/build gates passed. Seven instrumentation
  tests passed independently on API 30, 34, and 37, including foreground
  tracking surviving Home plus Activity destruction and native Retry recovery
  after a forced main-frame disconnect. The separate API 37 public-staging
  smoke asserted the home and Safety DOM over the exact temporary HTTPS origin,
  matched no external-origin app filter, and observed zero load/TLS errors.
- A real public Chromium two-user scenario registered isolated requester/helper
  fixtures, completed medicine discovery, acceptance, realtime chat, consent
  tracking, handover, both confirmations, and blind reviews. It passed 1/1 in
  12.8 seconds. Exact fixture cleanup removed its users and domain rows; the
  before/after public database diff was empty. Evidence is
  `output/staging-e2e/20260719170530-3787963`.
- The 600.5-second authenticated soak completed 271,072 iterations and 767,225
  HTTP requests with zero failed HTTP requests. All 703,995 public checks
  passed; expected-response latency was 2.337 ms average, 5.705 ms p95, and
  175.955 ms maximum in this local run. Eight authenticated sessions completed
  31,607 request-page/LiveView/chat/tracking start-update-stop chains and 4,800
  heartbeat WebSockets without an authenticated or WebSocket error. The
  database observed zero rollbacks, deadlocks, conflicts, or temporary files
  during the measured interval. Exact fixture cleanup restored every tracked
  table count. These are observations, not production SLOs or capacity limits;
  evidence is `output/performance/authenticated-soak-10m-20260719`.
- The final isolated Compose failure/replacement drill used 3 web and 2 worker
  replicas. All five nodes rejoined, PubSub passed, 744/744 readiness samples
  succeeded, and the injected Oban job completed on attempt 2 after exactly one
  recorded error. Evidence is
  `output/resilience/final-resilience-current-20260719`.
- The final project-owned kind rolling drill replaced all four web/worker pod
  UIDs, left all four replacements Ready with zero restarts, passed PubSub and
  an empty database diff, and completed 363/363 readiness samples; two needed
  one retry during the local single-node endpoint change. The current retained
  kind release is Helm revision 13 with 2/2 web, 2/2 worker, and PostGIS 1/1
  Ready. Evidence is `output/resilience/final-kind-current-20260719`.
- Prometheus matched all three direct web targets before and after the scoped
  failure; Grafana provisioning passed; Alertmanager delivered both firing and
  resolved webhooks for `who_need_help_load-web-19`; the database diff was
  empty. Evidence is
  `output/observability/final-observability-current-20260719`.
- Restic/MinIO passed encrypted streaming backup, full-data repository check,
  fresh-database restore, corrupted-repository fail-closed checks, and
  interruption after uploaded objects with zero published snapshots. The
  source database diff was empty. Evidence is
  `output/backups-s3/final-backup-current-20260719`; the one-run MinIO project
  and volume were removed after the drill.
- The database-scale harness used 50,000 configured rows per large table,
  captured 16 before/after plans, selected every asserted cursor index, and
  verified consecutive request pages without a gap or duplicate. The exact
  observations and limits are in
  `output/db-scale/20260719164620-3314314` and `docs/performance.md`.
- OAuth, SMTP, and HTTP-push boundaries passed success, rejection, timeout,
  retry, replay, and deduplication checks against local internal-only
  counterparts. Product acceptance and chat events produced durable jobs
  consumed by two worker replicas; message content was excluded from push.
  GitHub OAuth remains optional and disabled without credentials. Evidence is
  `output/external-boundaries/final-boundaries-20260719`.
- A clean tracked-archive deployment at `c6aa3d1` applied all 10 migrations,
  repeated migration idempotently, seeded 14 categories without duplication,
  started 2 web and 2 worker replicas, passed HTTP/Mailpit/cluster/PubSub, and
  left its project, image, volume, and extracted workspace absent after
  cleanup. Evidence is
  `output/portability/20260719171321-3973031-61769f`.
- The ordinary public Compose database was backed up to mode-`0600`
  `output/backups/compose-20260719-165722.dump` with SHA-256
  `5b4d8a78665c5d47863af8e1e885684b110d94c278474e37c8465e0814f7000f`.
  The archive passed catalog validation and an isolated restore before rollout.
  The current public image ID is
  `sha256:009f6754fb57a410f16252205bbfa1467a08b82a733de59c3c80e431afa8fc42`
  on 2 healthy web and 2 running worker replicas. The deployment diff was
  empty; the final read-only observation remains 2 users, 1 request, 7
  messages, 4 ended tracking sessions, and 10 migrations.
- After the workstation restart, the public route initially timed out because
  its existing OpenVPN connection was down. Inspection confirmed Nginx still
  targeted the assigned client address `10.8.0.14`; reactivating the existing
  `client1` NetworkManager profile restored that address and the public path.
  The final observation returned HTTPS 200, HTTP 301 to HTTPS, and the correct
  `77.110.101.144` endpoint. Headed Chrome then rendered the homepage with zero
  console errors or warnings; its document, CSS, JS, and logo requests returned
  200. The visible browser was left open.
- Scoped cleanup left zero containers, volumes, and networks for the
  `who_need_help_load` project and removed its ignored observability/backup-S3
  runtime directories. The reusable ordinary public Compose project and the
  project-owned kind cluster intentionally remain running.

## Responsive-header follow-up

Commit `768d63a` replaced the duplicated public navigation bars with one
responsive application header. At widths below 1280 px it exposes the primary
links, language selection, theme controls, and authentication actions through a
single native `details` menu; wider layouts show the same actions inline.

- `./scripts/quality.sh` passed after the change, including 172 ExUnit tests,
  formatting, compiler, xref, Credo, Sobelow, Dialyzer, dependency audits,
  Compose/Helm validation, and gated image scans. Evidence is
  `output/regression/header-unification-quality-20260719.log`.
- The isolated browser project passed bootstrap 1/1 and 27/27 scenarios across
  Chromium, Firefox, and WebKit. Its responsive accessibility case exercised
  widths 360, 768, 1030, and 1440 px, asserted a single header and primary
  navigation, and detected no horizontal overflow. Evidence is
  `output/e2e/20260719174824-537439`.
- Before the public Compose rollout, the database was backed up to mode-`0600`
  `output/backups/compose-before-header-20260719-175353.dump` with SHA-256
  `fea893340d2428a61ebed8abb83816a6273b1afeb5d794763b00c81a37d15e2d`.
  The archive passed an isolated restore with 23 readable public tables, 1,488
  rows, all 10 migrations, and PostGIS 3.6.4 before the temporary restore
  database was removed.
- The deployed image ID is
  `sha256:8c3790e6b5fdd3f1feb08165d33d5b2ddaa9089b6606da44a804c5facd4765f6`
  on 2 healthy web and 2 running worker replicas. The before/after database
  snapshots were byte-identical: 2 users, 1 request, 7 messages, 4 tracking
  sessions, and 10 migrations.
- Public HTTPS and `/healthz/ready` returned HTTP 200. The four BEAM replicas
  were connected and the deployment's cross-node PubSub check passed. Headed
  Chrome then verified the public page at 1030 and 360 px: one visible header,
  accessible menu contents, no horizontal overflow, and zero console errors or
  warnings. The visible browser was left open at 1030 px with the menu closed.

## Brand icon follow-up

Commit `c28c9a3` replaced the placeholder Phoenix mark with the selected
Who Need Help symbol. The symbol remains image-only while the header name stays
as accessible HTML text.

- One source SVG reproducibly generates 48 px browser fallback, PWA 192/512,
  maskable 512/1024, Apple touch 180, Apple-ready 1024, Google Play 512, and
  legacy Android density assets through `scripts/generate-brand-assets.sh`.
  The Google Play file is 512×512, 8-bit RGBA PNG, and 27,107 bytes.
- The PWA manifest separates `any`, `maskable`, and `monochrome` purposes.
  Android API 26+ uses foreground/background adaptive layers, API 33+ adds a
  monochrome layer, and API 24/25 retain generated density resources. The
  notification small icon is now a dedicated monochrome vector rather than the
  launcher artwork.
- The pinned Android SDK 37 Docker build passed JVM unit tests, `lintDebug`,
  `assembleDebug`, and `assembleDebugAndroidTest`. The exported debug APK and
  lint report are under `android/dist/`.
- `./scripts/quality.sh` passed with 173 ExUnit tests and every static,
  dependency, security, manifest, and image gate. Evidence is
  `output/regression/brand-assets-quality-20260719.log`.
- The isolated responsive/accessibility browser suite passed 6/6 across
  Chromium, Firefox, and WebKit. Evidence is
  `output/e2e/20260719193958-2786366`.
- Before rollout, mode-`0600`
  `output/backups/compose-before-brand-20260719-194155.dump` was created with
  SHA-256
  `9be1197a02783b959e2eee783c6d74b4bf2a65a581c6a4566f0e815c7465ea61`.
  Its isolated restore retained 24 public tables, the observed domain counts,
  all 10 migrations, and PostGIS 3.6.4; the temporary database was removed.
- Public image ID
  `sha256:8db2281f2cf51b787fc22d99f2911ee870c9ee66a0ae6e9b801796ab1aff605f`
  runs on 2 healthy web and 2 running worker replicas. The before/after
  database snapshots are byte-identical and cross-node PubSub passed.
- Every public brand URL, the manifest, service worker, homepage, and readiness
  endpoint returned HTTP 200. Public and repository SVG SHA-256 values matched.
  Headed Chrome showed the new 36×36 header mark at 1030 px with no overflow or
  console messages. Service worker `v2` activated and removed the old `v1`
  cache. Evidence is `output/brand/public-header-1030-20260719.png`.

## Current public functional replay

The committed two-user medicine scenario now includes browser geolocation in
the primary path rather than relying only on the separate failure-recovery
scenario.

- Against `https://whoneedhelp.imalto.site`, the current public release passed
  request creation, helper acceptance, two-way realtime chat, explicit
  geolocation consent, appearance of the helper's live marker in the
  requester's browser, explicit tracking stop and marker deletion, the
  one-time handover code, both completion confirmations, and double-blind
  review reveal. The Chromium scenario passed 1/1 in 14.7 seconds. Exact
  cleanup removed its two users, request, assignment, messages, reviews,
  tracking session, abuse signals, and audit events; the before/after
  application-table count diff was empty. Evidence is
  `output/staging-e2e/20260719202129-3641894`.
- The complete browser suite with that expanded primary path passed 27/27:
  9 scenarios each in Chromium, Firefox, and WebKit against a fresh isolated
  2-web/2-worker Compose cluster. The independent active-node-failure path also
  verified tracking and chat recovery after the serving BEAM node restarted.
  Evidence is `output/e2e/20260719202632-3770163`; its containers, networks,
  and database volume were removed by the run-scoped cleanup.

The broader public-staging replay was then expanded without reusing or deleting
unrelated application records:

- One run against `https://whoneedhelp.imalto.site` passed 3/3 Chromium
  scenarios covering registration, email confirmation, password setup, email
  change and re-login; medicine request, helper acceptance, private realtime
  chat, browser geolocation sharing and deletion, handover confirmation and
  double-blind reviews; motorcycle broken-chain roadside help with helper
  withdrawal and requester cancellation; Activity creation, join approval,
  approved group chat, reporting, evidence, blocking/unblocking, privacy
  defaults, social-link add/remove, category proposal and moderator actions.
- The fixture tool first verified the exact database name and an unused
  run-specific email prefix. Cleanup validated ownership of every related
  request, assignment, activity, participant, message, report, proposal,
  review, tracking, audit and push-job record before deleting it. It removed
  only the six observed run users and their validated relationships. Exact
  Mailpit message IDs for the run were deleted separately and their absence was
  rechecked. The before/after counts across 19 application tables produced an
  empty diff. Evidence is
  `output/staging-full-e2e/20260719210733-461538`.
- The current isolated browser matrix passed bootstrap 1/1 and application
  scenarios 30/30: 10 each in Chromium, Firefox and WebKit. It includes active
  BEAM-node failure while chat and browser tracking are in use, followed by
  reconnect on an available replica and continued chat/tracking operation.
  Evidence is `output/e2e/20260719211815-708067`; the isolated containers,
  networks and PostgreSQL volume were removed automatically.
- `./scripts/quality.sh` passed on the same source state: formatting, strict
  compilation, xref, Credo, Sobelow, Dialyzer, 173 ExUnit tests, dependency
  audits, Compose/Helm/observability checks and the configured image scans.
  Evidence is
  `output/regression/full-public-web-quality-20260719.log`.

## Final Android and operational replay

Application commit `b96d443` and its exact source state were subjected to the
following additional local and temporary-origin checks:

- `./scripts/quality.sh` passed all configured source, Compose, Helm,
  observability, dependency, release-image, and security gates with 174/174
  ExUnit tests. Evidence is
  `output/regression/android-cross-quality-20260720.log`.
- Android debug and staging unit/lint/APK builds passed. The isolated API
  30/34/37 matrix passed 7/7 device tests per API. Public API 37 DOM smoke
  passed 1/1. The Android/browser staging run passed Android magic-link login,
  private chat in both directions, foreground tracking, browser marker
  appearance/removal, fixture verification, and an empty 19-table cleanup
  diff. Evidence is
  `output/android-browser-staging-e2e/20260720053109-872921`.
- The external-boundary drill passed OAuth, SMTP, provider-neutral HTTP push,
  product job integration, retry, rejection, timeout, replay, and
  deduplication paths with two isolated workers. Its one-run project, volume,
  images, network, and generated secrets were removed. Evidence is
  `output/external-boundaries/final-boundaries-20260720`.
- The isolated 3-web/2-worker Compose failure drill crashed one web and one
  worker BEAM process, sequentially replaced every replica, rejoined all five
  nodes, passed PubSub, and completed the injected Oban job on attempt 2 after
  one recorded failure. All 725 readiness samples succeeded and the exact job
  was deleted. Evidence is
  `output/resilience/final-resilience-20260720`.
- The 30-second, 88-VU load replay completed 13,765 iterations and 38,826 HTTP
  requests. All 35,280 checks passed; HTTP failures were 0. It completed 1,765
  authenticated LiveView page/chat/tracking start-update-stop chains and 240
  heartbeat sockets without a functional error. PostgreSQL observed zero
  rollbacks, deadlocks, conflicts, or temporary files. Exact fixture cleanup
  produced an empty application-table diff. Evidence is
  `output/performance/final-after-queue-20260720`.
- Prometheus exactly matched and scraped all three web targets before and after
  the induced replica stop. Grafana's provisioned Prometheus datasource
  returned `OK`; Alertmanager delivered firing and resolved webhooks for the
  scoped instance. The database count diff was empty. Evidence is
  `output/observability/final-observability-20260720`.
- Restic created and checked an encrypted local S3 snapshot, restored it into a
  fresh database, rejected a corrupted repository, and published no snapshot
  for an interrupted upload. The interrupted repository passed checking after
  pruning unreferenced data; the source database diff was empty. Evidence is
  `output/backups-s3/final-backup-20260720`.
- The 50,000-row database replay selected every asserted cursor index and
  retained the same generated table counts before and after the index
  migration. Its isolated project, volume, and image were removed. Evidence is
  `output/db-scale/20260719233526-97476`.
- A clean deployment from `git archive b96d443` generated independent secrets,
  applied all 10 migrations, repeated migration without changing its 10
  migration or 14 category counts, and passed HTTP, Mailpit, four-node cluster,
  and PubSub checks on 2 web/2 worker replicas. Its workspace, image, project,
  volumes, and networks were absent after cleanup. Evidence is
  `output/portability/20260719233712-142771-ac1883`.
- The project-owned kind cluster was rebuilt with the current image. The final
  rolling drill replaced all four application pod UIDs with four Ready,
  zero-restart pods, rejoined all four BEAM nodes, passed PubSub, and left an
  empty database diff. All 395 readiness samples ultimately succeeded; two
  used one transport retry during the local single-node NodePort endpoint
  replacement. Evidence is
  `output/resilience/final-kind-rollout-fixed-20260720`.

The first 2026-07-20 kind attempt exposed a race in the verification harness:
one label-based `kubectl wait` invocation retained a terminating pod in its
initial resource set. The harness now waits for old UIDs to disappear, captures
the exact four replacement pod names, and waits for those resources. The fixed
full replay above passed.

## Final public and browser replay after one-time-link hardening

The final public replay exposed a transport edge case specific to one-time
confirmation links: the server had committed the email change and consumed the
token before Chromium reported `ERR_NETWORK_CHANGED`, while the generic
navigation helper then replayed the already-consumed URL. The browser harness
now opens one-time links at most once. If that single navigation loses its
response to a transient transport reset, it verifies the committed account
state through the idempotent settings page instead of replaying the token.

- The complete public Chromium replay passed 3/3 against
  `https://whoneedhelp.imalto.site`: registration and email/password changes;
  medicine request, helper matching, two-way chat, browser location tracking,
  marker deletion, handover and blind reviews; Activities, privacy, social
  links, blocks, reports, category voting and moderation. Its cleanup restored
  all 19 recorded application-table counts byte-for-byte and removed the
  run-specific Mailpit messages. Evidence is
  `output/staging-full-e2e/20260720042807-3462613`.
- The fresh isolated browser matrix passed bootstrap 1/1 and 30/30 application
  scenarios, 10 each in Chromium, Firefox and WebKit, including realtime
  recovery after the serving BEAM node was restarted. The run-scoped
  containers, networks and PostgreSQL volume were removed. Evidence is
  `output/e2e/20260720042117-3279852`.
- The public HTTPS PWA replay passed 1/1, covering manifest and install assets,
  service-worker cache update, offline public fallback, and exclusion of
  private application pages from the offline cache. Evidence is
  `output/staging-pwa-e2e/20260719235520-665802`.
- `./scripts/quality.sh` passed again on the same source state with 174/174
  ExUnit tests and all configured source, dependency, Compose, Helm,
  observability and container-image security gates. Evidence is
  `output/regression/final-single-use-quality-20260720.log`.

## Queue and ingress-policy hardening replay

The 2026-07-20 hardening pass removed the unused Oban `default` consumer and
made the two real queue limits explicit in Compose, generated local
environments, and Helm. After rebuilding the isolated load release, both
worker nodes reported exactly `maintenance: [limit: 2]` and
`push: [limit: 1]`; the five-node cluster and cross-node PubSub probe passed.
The explicit 1,000-job burst completed 1,000/1,000 jobs in the observed 4,516
ms, produced no matched application/database error, removed exactly its 1,000
run-scoped rows, left zero probe jobs, and retained identical domain-table
counts. Evidence is
`output/performance/oban-burst-1000-final-20260720`.

The Helm chart now renders one ingress NetworkPolicy for the chart instance.
Helm lint and the rendered-manifest Trivy scan pass. The manifest permits
chart-instance pod-to-pod Erlang distribution and the configured HTTP
listener while isolating other inbound pod ports. Actual packet enforcement
has not been claimed because it depends on the target cluster's CNI; that must
be verified on the eventual deployment environment.

The same source state passed `scripts/quality.sh`: 186/186 ExUnit tests,
format and warnings-as-errors compilation, xref, Credo, Sobelow, Dialyzer,
Hex/npm audits, ShellCheck, Hadolint, actionlint, every Compose render, Helm
lint, observability configuration, rendered-manifest scanning, and all
configured runtime image scans. Every reported HIGH/CRITICAL vulnerability
count was zero.

## Final pre-production audit replay on 2026-07-20

The final manual code pass fixed LiveView form metadata leaking into dynamic
category data, stale review state across replicas, forms retaining submitted
report/proposal/chat content, an invalid leave action on completed activities,
and blocked-organizer metadata remaining visible in a participant's activity
history. Regression coverage now includes all of those cases.

- `./scripts/quality.sh` passed on the final source state with 243/243 ExUnit
  tests, format and warnings-as-errors compilation, xref, Credo, Sobelow,
  Dialyzer, Hex/npm audits, Compose and Helm rendering, observability
  validation, source secret/misconfiguration scanning, and every configured
  production and infrastructure image scan. The reported configured-threshold
  vulnerability count was zero.
- The final isolated 50,000-row database replay measured public help discovery
  at 0.144 ms and the blocked participant-activity page at 22.837 ms for 25
  returned rows on this workstation. The aggregate leaderboard and helper
  reputation queries measured 45.006 ms and 38.443 ms respectively. These are
  observations, not production resource requirements. Evidence is
  `output/db-scale/20260720154617-630777`.
- The 30-second isolated load run used 3 web and 2 worker replicas. It completed
  38,936 HTTP requests and 2,060 WebSocket sessions with zero HTTP failures;
  overall HTTP p95 was 5.321 ms. It also completed 1,820 authenticated
  LiveView/chat/tracking cycles. Peak sampled container memory was 296-324 MB
  per web replica, 205-219 MB per worker, 121 MB for PostgreSQL, and about
  1.54 GiB when summing the seven measured runtime-container maxima. Database
  sampling observed no deadlocks, lock waits, temporary files, or rollbacks
  during the run. Evidence is
  `output/performance/final-current-20260720`.
- The subsequent replacement drill recorded 650/650 successful readiness
  samples, a five-node BEAM cluster, cross-node PubSub, healthy web/worker
  recovery, and an Oban probe completing on attempt 2 after its intentional
  first failure. Evidence is
  `output/resilience/final-current-resilience-20260720`.
- A visible-browser replay on `https://whoneedhelp.imalto.site` covered
  registration and magic login, account/profile/privacy controls, social-link
  semantics, medicine request creation and matching, private realtime chat,
  optional live tracking UI and cleanup, handover, double confirmation,
  double-blind reviews, activity creation/join approval/exact-location privacy,
  group chat and completion, reports/moderation/audit evidence, category
  proposal/voting/moderation, and blocking in both directions. The final fresh
  post-deployment tab reported zero console errors and zero warnings.
- The first public automation replay exposed two verification defects after the
  UI fixes: duplicate valid review status text made a broad locator ambiguous,
  and a committed password change lost only its redirect GET to
  `ERR_NETWORK_CHANGED`. Commit `9463009` scopes the review assertion to the
  persistent status and submits the password mutation only once before
  recovering through an idempotent settings GET. The clean replay passed 3/3.
  Evidence is `output/staging-full-e2e/20260720160445-1004343`.
- The final HTTPS PWA replay passed 1/1, Android/browser chat and tracking
  passed twice, and the packaged native staging APK passed its Android 37
  public-origin smoke. Evidence is
  `output/staging-pwa-e2e/20260720160533-1019120`,
  `output/android-browser-staging-e2e/20260720160544-1022997`, and
  `output/android-staging-smoke/20260720160743-1061722`.

The manually created browser-audit user, request, assignment, activity,
messages, tracking session, reviews, reports, category proposals, vote, abuse
signals, audit events, and Mailpit messages were removed by exact identifiers.
The existing helper account was restored to role `user`, no password, and no
blocks. Post-cleanup counts retained two users plus one unrelated request and
assignment; those unrelated rows were not changed.

## Production-handoff replay on 2026-07-20

Commit `14987bb` adds a first-deployment Compose path without inventing the
unknown server topology. `scripts/init-production-env.sh` generated independent
database, Phoenix, handover, BEAM-cookie and metrics values without printing
them, wrote mode `0600`, and refused to overwrite its destination.
`scripts/validate-production-env.sh` accepted the fully supplied fixture and
rejected the unresolved proxy/SMTP fixture. The configurable
`HTTP_BIND_ADDRESS` rendered as both `0.0.0.0` and `127.0.0.1`.
`compose.production.yaml` kept Mailpit inactive unless its explicit local
profile was enabled.

- The complete isolated quality/security gate passed after these changes:
  244/244 ExUnit tests, format and warnings-as-errors compilation, xref, Credo,
  Sobelow, Dialyzer, Hex/npm audits, ShellCheck, actionlint, all Compose/Helm
  renders, observability validation, tracked-source secret/misconfiguration
  scanning, and all configured production/infrastructure image scans. The
  configured-threshold vulnerability count was zero.
- The deployed Compose stack applied all 12 migrations and reported two
  healthy web plus two healthy worker replicas. A repeated cluster probe
  observed all four BEAM nodes and passed cross-node PubSub. Fresh cgroup
  observations were approximately 210-221 MB per application replica; this is
  not a future server minimum or capacity promise.
- Public HTTPS returned ready/live success, HSTS, secure HttpOnly SameSite
  cookies and the expected security headers. The scheme-specific CSP contained
  `wss://whoneedhelp.imalto.site` and no plaintext WebSocket origin.
- A headed Chrome registration used a unique run-scoped address, received its
  confirmation message in Mailpit, required explicit confirmation, and created
  an authenticated session. The profile exposed location-visibility choices,
  manually added unverified social-link semantics, reputation, blocks and an
  optional external thank-you link. Sensitive settings correctly required
  email reauthentication after the sudo window. The password form's browser
  accessibility warning was fixed by associating its username; the fresh
  deployed settings tab reported zero errors and zero warnings.
- A full public staging replay passed 3/3 registration/settings, mutual-aid and
  activity/moderation scenarios. Its database cleanup diff was empty. Evidence
  is `output/staging-full-e2e/20260720184356-3858682`.
- The public PWA install/cache-update/offline-fallback replay passed 1/1.
  Evidence is `output/staging-pwa-e2e/20260720184450-3875584`.
- The current staging APK/test APK were rebuilt for the configured HTTPS
  origin. The Android 37/browser replay passed magic login, bidirectional
  realtime chat, foreground location sharing, a PostGIS-backed position and
  notification Stop cleanup; it retained zero raw positions and its database
  cleanup diff was empty. Evidence is
  `output/android-browser-staging-e2e/20260720184520-3885007`.
- Backup `output/backups/compose-20260720-182338.dump` has SHA-256
  `ac19d5dbb3c1b797269003d6b302a2ae3d8d9634be6ad31ea2468d7f5a837a3b`.
  Its fresh-database restore drill found 23 public tables, 1,489 rows, all 12
  migrations and PostGIS 3.6.4. The upgrade rehearsal passed with an empty
  application-table diff, four cluster nodes, PubSub and HTTP; evidence is
  `output/upgrade-rehearsal/20260720182353-3474703`.

The final manual registration account had no request, assignment, message,
activity, review, report, block, social-identity or tracking references. It and
its two session tokens were removed in one exact transaction; its two exact
Mailpit message IDs were also removed. Final domain counts returned to two
users, one unrelated request, one unrelated assignment and seven unrelated
messages. One PostgreSQL error in the audit window came from an incorrect
column name in a read-only operator query; PostgreSQL aborted that transaction.
The subsequent log window contained no matched error/warning entry.

## Private-Git and final boundary replay on 2026-07-20

The newly configured private `origin` and its `main` branch both resolved to
commit `378768b` before this replay. The observed server API identified Gitea
1.22.0. Because that release reads repository workflows from
`.gitea/workflows/`, a separate Gitea workflow now runs the same four local
gates sequentially on the dedicated `who-need-help-ci` label. Both the GitHub
and Gitea workflows passed pinned actionlint locally. Remote Actions enablement,
runner registration and an actual Gitea run remain unverified until a trusted
runner is provisioned.

This replay found and corrected two integration regressions that the ordinary
Phoenix suite did not cover: the external-boundary workers were missing the
new required metrics credential, and the drill's product request was missing
the new required safety acknowledgement. The repaired drill then passed OAuth,
SMTP, provider-neutral push, two Oban worker replicas, retry, replay
deduplication and product-event delivery. Its non-secret mode-`0600` evidence
is `output/external-boundaries/preprod-gitea-pass`.

- The final isolated quality/security gate passed with 244/244 ExUnit tests and
  zero configured-threshold source/image findings. The Gitea runner state,
  binary and local data directory are now excluded from Git and Docker build
  contexts and covered by the quality script.
- The isolated Playwright suite passed all 30 scenarios across Chromium,
  Firefox and WebKit, including medicine matching, chat, tracking, handover,
  blind reviews, privacy, moderation, account changes and replica restart
  recovery. Evidence is `output/e2e/20260720190130-1182`.
- The debug Android build passed unit tests, lint, debug APK and test APK
  assembly using `.env.example`; no generated APK is tracked.
- The temporary HTTPS origin returned HTTP 200 from both `/healthz/live` and
  `/healthz/ready`. The observed local Compose project had two healthy web and
  two healthy worker replicas with zero restarts.
- One idle `docker stats --no-stream` snapshot measured approximately
  1,040 MiB combined across the database, Docker API proxy, Mailpit, edge
  proxy, two web replicas and two worker replicas. This workstation snapshot is
  not a production minimum, limit or capacity claim.

## Compose topology and database-mode verification on 2026-07-21

- The current ExUnit suite passed 260/260 after adding the combined runtime
  role and deployment-mode selection.
- The complete isolated quality/security gate passed: ShellCheck, Hadolint,
  actionlint, all Compose mode renders and production-environment rejection
  cases, Helm lint/render, tracked-source scanning, format, warnings-as-errors
  compilation, xref, Credo, Sobelow, Dialyzer, Hex/npm audits, 260/260 ExUnit
  tests, and every configured runtime/infrastructure image scan. The
  configured HIGH/CRITICAL finding count was zero.
- A fresh isolated `compact + container` production render contained only
  `db`, one-shot `migrate`, and `app`. Readiness and a local PubSub probe passed;
  the running combined container reported the expected 65,536 Erlang port
  limit. Its exact containers, networks, and database volume were removed.
- A separate PostgreSQL 18.4/PostGIS 3.6 fixture was published only on the
  Docker bridge gateway. The `split + external` render contained migrate,
  Docker API proxy, Traefik, two web replicas, and two worker replicas, and did
  not contain a `db` service or database volume. The preflight reported
  PostgreSQL/PostGIS versions and `TLS=false` for that local fixture, all 13
  migrations applied, readiness passed, all four BEAM nodes joined, and a
  cross-node PubSub broadcast passed. Its logs contained no error marker and
  all run-scoped containers, networks, fixture container, and fixture volume
  were removed.
- The first compact drill exposed an isolation defect: the generated
  production environment lacked `COMPOSE_PROJECT_NAME`, so Compose selected
  the ordinary project name and recreated its database container with the
  drill's environment. Authentication failed before any migration ran and the
  compact app remained in `Created`. Read-only checks still found 13
  migrations, 2 users, and 1 request. Backup
  `output/backups/compose-20260720-223237.dump` was created, the ordinary split
  stack was recreated with its original ignored `.env`, the same counts and
  HTTP readiness were rechecked, and the never-started compact container was
  removed. The initializer now writes and validates an explicit project name,
  and both subsequent isolated drills used distinct verified project names.
- The final ordinary development stack again reported two healthy web and two
  healthy worker replicas, cross-node PubSub, local and temporary-public-origin
  readiness, unchanged counts (13 migrations, 2 users, 1 request), and no
  error marker in the post-rollout application/proxy log window.

These checks establish the mode wiring on this workstation. They do not prove
the future provider's TLS/CA policy, network reachability, backup service, high
availability, or target-server capacity.

## UniSender Go HTTPS delivery boundary on 2026-07-21

- The production server returned HTTP 200 and a successful authenticated result
  from UniSender Go's `system/ping` Web API method. The API key was loaded from
  the server's mode-`0600` credential fragment and was not printed.
- Direct TCP connection attempts from the same server to SMTP ports 25, 465,
  and 587 timed out for UniSender Go; control attempts to other public SMTP
  providers also timed out. This observation does not establish where the
  filtering occurs.
- `WhoNeedHelp.Email.UnisenderGoAdapter` now maps the application's existing
  Swoosh messages to the provider's HTTPS `email/send.json` contract. Six
  focused tests passed for the exact request shape (including explicit
  `track_read=0` and `track_links=0`) and API-key header, success, redacted
  recipient rejection, structured API errors, invalid responses, and rejection
  of unsupported or provider-invalid messages before network I/O.
  Runtime configuration and production
  environment validation can select either `smtp` or `unisender_go` without
  requiring SMTP settings in API mode.
- Authoritative DNS and the provider UI both showed the sending domain as
  verified with DKIM active, while the delegated link domain showed configured.
  A second real Web API message was accepted for one recipient with no rejected
  recipients and reached Gmail's Inbox after one second. Gmail's original
  message view reported SPF PASS, DKIM PASS with `d=whoneedhelp.com` and
  selector `gokey`, and DMARC PASS.
- The delivered MIME still contained the provider's one-pixel
  `go2_read_tracker` even though the request explicitly supplied
  `track_read=0` and `track_links=0`. UniSender Go documents that honoring those
  zero values, and using `skip_unsubscribe=1`, requires provider approval. A
  support request covering all three permissions was sent to the official
  support address from the account mailbox. Privacy-disable behavior and
  unsubscribe removal remain unverified until the provider enables them and a
  new delivered MIME is inspected.

## Known work before a public production launch

- Replace the temporary staging origin with the production-owned domain and
  production infrastructure. The temporary origin is available only while the
  workstation, Compose stack, VPN path, gateway, and their networks are up.
- Confirm the final Android application ID before creating its Play Console
  listing, publish `/.well-known/assetlinks.json` for that ID and the final
  signing fingerprint if verified App Links are wanted, and complete store
  policy/release work. A dedicated upload key and signed APK/AAB have been
  created and verified locally, but no Play application has been registered.
- Operate PostgreSQL/PostGIS with off-site backups, recovery testing, monitoring,
  and the availability model selected for real usage.
- After provider approval, verify that delivered MIME contains neither open nor
  link tracking and omits the unsubscribe block, then exercise registration and
  magic-link delivery through the deployed application to a real mailbox.
- Deploy the already configured production and staging Google OAuth clients,
  then exercise registration, sign-in, and settings linking against Google on
  their exact HTTPS callback origins.
- Configure and verify a real mobile push provider and device-token lifecycle
  if native push is required. The provider-neutral HTTP boundary and product
  jobs are tested; FCM/APNs device delivery is not.
- Load-test representative data and traffic, then set measured pool, resource,
  autoscaling, and action-limit policies.
- Publish jurisdiction-specific emergency contacts, privacy, retention,
  prohibited-items, and voluntary-payment guidance after legal review.
- The UI now has authenticated and external account-deletion/data-request
  intake, contact verification, case status, and an audited operator queue.
  Actual erasure/anonymization and export remain manual until a legally reviewed
  retention policy defines the treatment of linked safety and dispute records.
- Staff and monitor the implemented moderation/support queues and establish an
  incident-response/on-call process for real users.
- If verified GitHub identity is desired, create and configure its OAuth App and
  exercise the external redirect/callback. This is an optional enhancement, not
  a registration or launch blocker.
- Production capacity, minimum CPU/RAM, database HA topology, and autoscaling
  thresholds remain unknown until representative target-environment
  measurements exist.
