who_need_help/docs/verification.md

Who Need Help — implementation verification

Observed through 2026-07-19 in the local workspace. This report separates observed results from product limits and unknown production properties.

Verified MVP capabilities

Requirement	Status	Observed evidence	Limit
Urgent medicine-help flow	Implemented and tested	Request creation, discovery, matching, start, handover, two-party completion, and review rules are covered by the Phoenix test suite and exercised in the local UI.	The product coordinates pickup of an already purchased or reserved legal item; it is not a pharmacy, medical, or emergency service.
Urgent roadside help	Implemented and tested	Fuel, car wheel, bicycle, motorcycle, vehicle-breakdown, and secured-incident categories are seeded as a translated hierarchy. Server and LiveView tests exercise category paths, required fields, allowed values, boolean normalization, unknown-field rejection, and request creation.	Roadside requests require no immediate danger; this is not emergency response or professional recovery.
Extensible categories	Implemented and tested	Categories and validated text/select/boolean fields are stored in PostgreSQL. Proposal, vote, approve, reject, and merge paths have automated tests.	Coffee, cinema, hiking, and other social activities remain separate from urgent-help safety and ranking rules.
Separate Activity mode	Implemented and tested	Coffee, cinema, walk, and hiking categories use a separate activity lifecycle. Domain and two-client LiveView tests cover creation, join request, organizer approval, capacity enforcement, public/pending/chat privacy, exact-location disclosure to approved users, group chat, blocking, completion, and zero impact on helper reputation. Activity and message reports expose only the linked group conversation to an audited moderator; moderators can hide and restore reported activities.	This does not guarantee participant identity or physical safety.
Map and discovery	Implemented and browser-verified	The committed isolated Chromium suite rendered request and Activity maps, waited for MapLibre idle, and completed with no console, page, or request failures against a local PNG raster fixture. An earlier headed session rendered the configured OpenStreetMap tiles.	A production operator must configure a tile provider appropriate for its policy and traffic.
Private matched chat	Implemented and cross-client verified	A message sent from the helper browser appeared in the requester's browser without reload. An earlier Android emulator run also sent a message that appeared in the requester browser in real time.	There is no unsolicited general-purpose inbox.
Consent-driven live tracking	Implemented and cross-client verified	On API 37, Android started TrackingService as a location foreground service with a persistent Stop notification. After Home minimized the Activity, an emulator coordinate change reached PostGIS. Notification Stop removed the service, notification, active session, and raw position.	Browsers stop with the page. Android has no ACCESS_BACKGROUND_LOCATION, unattended start, or route history.
Privacy settings	Implemented and browser-verified	The profile exposed hidden, approximate public, exact for active match, and explicit exact-public options. Blocking and current-position cleanup have automated tests.	Exact public location remains a user opt-in; legal privacy and retention text still requires jurisdiction-specific review before launch.
Reputation and anti-abuse	Implemented at MVP level	Handover codes, two-party completion, double-blind reviews, unique-counterpart ranking, optional movement/proximity evidence, reports, blocks, abuse signals, and moderator audit paths have automated tests.	The system is not bot-proof and does not claim identity verification. No punitive numeric policy is enabled without measured and approved thresholds.
Social profiles	Manual links implemented; GitHub verification implemented and automated-tested	Manual links cannot set verification fields. The optional GitHub flow uses state, PKCE, a user-bound one-time session, unique provider ownership, and an audit record; 167 tests pass, including callback replay/state checks. The local protocol drill also performs real HTTP token/user exchanges without returning an access token to the application.	The staging operator has not supplied GitHub OAuth credentials, so the real external provider redirect/callback remains disabled and has not been browser-verified. Other providers remain manual/unverified.
Voluntary thanks	Implemented as an external optional link	A helper can expose an optional link after completion; the UI states that the platform does not process the payment.	The platform does not provide payments, escrow, refunds, tax reporting, or payment guarantees.
Android client	Local and public-staging clients implemented and emulator-verified	The native packages org.whoneedhelp.mobile.debug and org.whoneedhelp.mobile.staging launch the same authenticated LiveView app. Public HTTPS login, map, two-way chat, permission prompts, minimized foreground-service location updates, notification Stop, deep-link routing, and server cleanup were exercised on API 37.	Production signing, Play Store publication, verified Android App Links, unattended/background-permission tracking, and iOS are not implemented.
Multiple web/worker instances	Implemented and locally failure/rollout-verified	The isolated Compose profile passed BEAM crashes and sequential replacement with 3 web/2 worker replicas, all five nodes joined, PubSub passed, and 758/758 readiness requests succeeded. The project-owned kind cluster replaced all 2 web/2 worker pod UIDs under maxUnavailable=0; all four replacement pods joined and PubSub passed.	Local PostGIS is a single instance. Production database HA, backups, and recovery are operator work and are not claimed complete.
Local observability	Implemented and protocol-verified	Pinned Prometheus scraped all 3 direct load web targets with a file Bearer credential; Grafana provisioned a healthy datasource and four-panel dashboard; Alertmanager delivered firing and resolved webhooks for an induced scoped replica stop.	Local delivery does not establish production retention, notification-provider reliability, on-call policy, or measured alert thresholds.
Encrypted local backup	Implemented and failure-verified	Pinned Restic streamed PostgreSQL custom format into pinned local MinIO with no host plaintext dump, passed full-data checking and a fresh-database restore, rejected a corrupted repository, and published no snapshot for an interrupted upload.	The retained MinIO volume is on the same workstation; this is not off-site storage, database HA, or a production RPO/RTO/retention claim.
External protocol boundaries	Implemented and locally failure-verified	The production release used its configured Assent/Req and Swoosh/gen_smtp clients against an internal-only mock; OAuth and SMTP success/rejection/retry/replay/timeout paths passed. The future HTTP push boundary passed disabled, retry, rejection and idempotency paths.	This does not verify external provider availability. Remote FCM/APNs delivery and product-workflow integration are not implemented; SMTP exactly-once delivery is not claimed.

Reproducible checks

• The isolated Phoenix suite completed on 2026-07-19 with 167 tests and 0 failures after cursor pagination, database aggregation, and the full localization changes on Elixir 1.20.2 and Erlang/OTP 29.0.3.
• mix compile --force --warnings-as-errors and mix format --check-formatted: passed against the same final source.
• ./scripts/quality.sh passed ShellCheck 0.11.0, Hadolint 2.14.0 at warning threshold, actionlint 1.7.12, all eight Compose renders, Helm lint, Trivy source/rendered-manifest scanning, xref, Credo high-priority checks, Sobelow strict/private checks, Hex audit, 167 Phoenix tests, both npm audits, and the backup/MinIO/mc/external-mock/release-image vulnerability scans. The rendered Helm manifest reported zero HIGH/CRITICAL misconfigurations; the Alpine backup, MinIO, mc, and external-mock images and the Debian 13.6 release image each reported zero HIGH/CRITICAL vulnerabilities under the configured gates. The backup binary reported Restic 0.19.1 compiled with Go 1.26.5; MinIO and mc reported their pinned commits and Go 1.26.5.
• Dialyzer passed with three path- and warning-specific documented filters and zero unused filters. Two findings are opaque-type warnings at Ecto Multi/MapSet call sites; the third is the generated Gettext backend's three-form plural-rule opaque-value warning; the filters are visible in .dialyzer_ignore.exs rather than hidden by a broad pattern. This is not a claim that third-party dependency compilation is warning-free.
• The generated Activity migration was rolled back by exactly one step and migrated forward again against who_need_help_test; both directions passed.
• The Activity-report migration was also rolled back and migrated forward. The observed database constraint changed from exactly one of 3 urgent-help targets to exactly one of 5 urgent-help/Activity targets.
• mix format --check-formatted: passed in the final run.
• Android local Docker build target: testDebugUnitTest, lintDebug, assembleDebug, and assembleDebugAndroidTest passed. The isolated API 37 runner then passed 5/5 instrumentation tests for denied/granted location permission, deep-link lifecycle, foreground location upload, notification Stop, and disconnected-Stop retry state. The public-staging target ran the unit tests plus lintStaging and assembleStaging; both final artifacts retain the documented hashes. Evidence for the final device run is output/android-instrumentation/20260719053610-2122819.
• Release Android guard: a staging/release build with a non-HTTPS WNH_BASE_URL failed at its dedicated preflight; the successful staging build used the explicit temporary HTTPS origin.
• Helm lint, template rendering, server-side dry-run, rollout waits, readiness checks, and cross-node PubSub verification passed in the local kind cluster.
• The hardened kind rollout then ran PostGIS as UID/GID 70 and Mailpit as UID/GID 10001. Both used read-only root filesystems, dropped all Linux capabilities, disabled privilege escalation, and used RuntimeDefault seccomp. Both web and both worker replicas were Ready; their migration-wait init containers also used read-only root filesystems and dropped capabilities. The cross-node probe observed all four BEAM nodes, and live/readiness returned ok/ready. Database counts remained 0 users / 0 help requests.
• The isolated Compose resilience drill observed a restart count increase for one crashed web and worker BEAM process, replaced every replica sequentially, observed all 5 cluster nodes, passed PubSub, and completed a real Oban retry on attempt 2 after one recorded error. Its exact job row and all fixture domain rows were absent afterward; 758 readiness samples had no final failure.
• The reproducible kind rolling drill advanced both Deployments from revision 19 to 20, replaced all four pod UIDs, observed all 4 BEAM nodes, passed PubSub, and left the database-count diff empty. All 362 readiness samples ultimately returned 200; two samples needed one reconnect attempt each during local single-node NodePort endpoint replacement.
• The canonical local observability drill matched all 3 generated/active Prometheus instance targets, checked the provisioned Grafana datasource and four-panel dashboard, received firing and resolved Alertmanager webhooks for the exact stopped/recovered web replica, restored every target to up, and left the application-table count diff empty. Its retained evidence contains neither the metrics token nor the random Grafana password. Final evidence is retained at output/observability/final-observability-20260719a.
• The observability stop command changed only the four scoped monitoring container states. All app/database/worker container IDs stayed unchanged and running, and the checked user/request/message counts were identical before and after. A subsequent full drill returned all monitoring services to healthy.
• The encrypted S3 canonical drill restored 23 public application tables, all 10 current migrations, 14 categories, and PostGIS 3.6.4 into a fresh temporary database. restic check --read-data passed for the retained repository. Flipping the first byte of an isolated cloned repository made both check and dump exit nonzero. Stopping the exact backup container after uploaded encrypted objects produced exit 130 and zero snapshots; Restic identified and pruned 32.809 MiB of unreferenced packs, then passed a second full-data check. Only the retained canonical bucket remained, the restore database and interruption container were absent, no configured secret appeared in retained evidence, and the source database count diff was empty. The obsolete successful-run bucket was then removed; only the final canonical bucket remains. Evidence is retained at output/backups-s3/final-backup-20260719a; the retained MinIO bucket is wnh-backup-final-backup-20260719a.
• The canonical external-boundary drill passed real client-side OAuth token and user HTTP exchanges, PKCE/state checks, provider denial, one-time-code replay, a fresh flow after a temporary token error, and timeout failure. It also passed SMTP acceptance, permanent rejection without retry, one retry after a temporary greeting, timeout, and repeated submission, plus the future push adapter's disabled default, rejection, one retry, replay deduplication and ambiguous-timeout deduplication. Every retained file is mode 0600, the evidence contains no generated credential, no host port was published, and the exact Compose project, network, and one-run images were absent after cleanup. Evidence is retained at output/external-boundaries/boundary-canonical-20260719d.
• The committed browser suite passed its 1/1 bootstrap and all 8/8 Chromium specs against a fresh PostGIS volume with two web and two worker replicas on 2026-07-19. The retained successful-run artifact directory is output/e2e/20260719053436-2083768. It covered axe checks across four pages and two themes, keyboard navigation, three responsive widths, Activity/moderation/privacy, the two-user medicine handover and blind reviews, public boundaries, local raster maps, and an offline LiveSocket transport failure/reconnect. It also verified persistent Russian/Ukrainian public language selection and the selected Ukrainian locale inside an authenticated LiveView. The uniquely named Compose volume and networks were removed.
• The default localization domain contains 508 current messages and the errors domain contains 40. Russian and Ukrainian catalogs have no empty or fuzzy current entries. Category names, descriptions, structured-field labels and select/boolean values use locale maps with an English fallback; the compatibility description column remains as a fallback for pre-existing categories.
• The isolated database-scale harness migrated an empty PostgreSQL 18.4 database to the exact pre-index version, seeded the configured canonical sample, captured JSON EXPLAIN (ANALYZE, BUFFERS) plans before and after the generated cursor-index migration, and removed its project and volume. The final 50,000-row-per-large-table run selected every asserted cursor index; two consecutive request pages matched the first 50 ordered rows with no gap or duplicate. Exact observations and their non-SLO limitations are recorded in docs/performance.md.
• Browser verification used headed Chrome. The authenticated matched-request page rendered its chat, MapLibre marker, and live-location controls with no console errors or warnings after the foreground-service rollout.

Local generated evidence (ignored by Git):

• output/playwright/final-request-map-chat.png
• output/playwright/final-privacy-profile.png
• output/android/final-image-smoke.png
• output/android/foreground-notification.png
• output/android/staging-deep-link-safety.png
• output/android-instrumentation/20260719053610-2122819/results.txt
• output/e2e/20260719053436-2083768/

Android artifact:

• android/dist/who-need-help-debug.apk
• SHA-256: 4520aa0b50eaf53bb7052e7f4c096a73d456f45781380187586c79d7267c21c5
• android/dist-staging/who-need-help-staging.apk
• SHA-256: 08d990a9268a382052be5f2d3ba5afba2aa65a9ad5f9ae1ad4bff720f8999a6e
• Observed staging manifest values: package org.whoneedhelp.mobile.staging, version 0.1.0-staging, minimum SDK 24, target and compile SDK 37, cleartext traffic disabled, singleTop MainActivity, and an exact HTTPS host of whoneedhelp.imalto.site.
• apksigner verify accepted the staging artifact's v2 generic debug signature. That signing identity is only for installable staging verification and is not a production release identity.

Configuration finding

WNH_DEBUG_BASE_URL is a required Android build input, because the trusted same-origin value is compiled into the debug APK. Its concrete local value is read from the repository's ignored .env by scripts/android-build.sh and is passed to Docker with --build-arg. android/Dockerfile only declares ARG WNH_DEBUG_BASE_URL; it contains no URL default.

The example http://10.0.2.2:4010 remains only as unit-test data for origin matching. It is not a runtime or build default. The ignored .env is also excluded from the root Docker build context.

Database observations

The post-roadside local Compose observation found 2 users, 1 help request, 7 messages, and 1 assignment before and after deployment. Category seeding changed only the category count from 1 to 9: 7 selectable categories and 2 inactive grouping nodes. These are local scenario data; the database is not assumed empty.

The Activity migration then preserved those same user/help/message/assignment counts, added empty activities, activity_participants, and activity_messages tables, and expanded the category tree to 14 rows: 9 help rows and 5 Activity rows. A validated custom-format dump was created immediately before that migration at output/backups/compose-before-activity-20260718-203930.dump; this ignored local artifact is not a substitute for the still-required isolated restore drill.

The final kind observation found 0 users, 0 help requests, and 0 messages. That state was queried before changing the database workload. The migration from the old emptyDir Deployment created a custom-format dump, validated it with pg_restore --list, changed PostGIS to a persistent StatefulSet, and restored the dump before the Helm upgrade. The final kind database reported PostgreSQL 18.4 and PostGIS 3.6.4.

The local kind Secret contains independently generated values. Only their decoded lengths were inspected: the PostgreSQL password is 64 characters and each application secret is 128 characters. Secret values were not printed or written to tracked files.

Dependency-upgrade observations

• The running Compose and kind releases reported Elixir 1.20.2 and Erlang/OTP 29.
• Compose reported Traefik 3.7.8 and Mailpit 1.30.4; Compose and kind both reported PostgreSQL 18.4 and PostGIS 3.6.4.
• The production asset build reported Tailwind CSS 4.3.3 and daisyUI 5.6.18; esbuild is configured at 0.28.1.
• The isolated Node build stage reported Node.js 24.18.0, npm 12.0.1, npm outdated --json returned {}, and npm ci reported zero known vulnerabilities.
• The Android build used Android Gradle Plugin 9.3.0, Gradle 9.6.1, Command-line Tools 22.0, and its embedded Android CLI 1.0.15857036. Unit tests, lint, and debug assembly passed; the lint report says “No errors or warnings.” The rebuilt emulator image contains the API 37 ps16k AVD and APK, and its emulator 36.6.11.0 binary returned its version successfully at runtime.
• The final complete scripts/kind-up.sh run was idempotent and finished with both 2-replica Deployments Ready and a successful cross-replica PubSub probe.

The migration 20260718114233 and reverse block lookup index blocks_blocked_id_blocker_id_index were present. An actual EXPLAIN ANALYZE for that reverse lookup selected the index. The table contained no block rows, so this confirms query shape, not production performance.

Exact production capacity, minimum CPU/RAM, and scaling thresholds are unknown: there is no representative production dataset, traffic model, or target-environment measurement. The isolated measurements in docs/performance.md include the earlier paced public/readiness/heartbeat comparison and a later 3-web/2-worker run with real authentication, LiveView, chat writes, tracking writes, and explicit position deletion. The latter completed 1,888 authenticated chains and 240 heartbeat sockets without a functional failure and restored every tracked application table count after fixture cleanup. Neither short run found a saturation point or represents production traffic, so the Helm chart does not invent resource limits or an HPA policy.

Public staging observation

On 2026-07-18, whoneedhelp.imalto.site was published through the existing Ubuntu Nginx gateway and its OpenVPN path to the local Compose proxy. HTTP redirected to HTTPS; the homepage, fingerprinted assets, and both health endpoints returned HTTP 200. A headed Chrome session rendered the public page with zero console errors or warnings. A complete WebSocket Upgrade request to /live/websocket returned 101 Switching Protocols.

The observed Let's Encrypt certificate had the correct DNS:whoneedhelp.imalto.site SAN and an expiry of 2026-10-16. Nginx and the Certbot renewal timer were active. This verifies the current staging path; it does not make the workstation or gateway a production availability environment.

On 2026-07-19 a new read-only check resolved the origin to 77.110.101.144; HTTP redirected to HTTPS, and the homepage, Safety page, and readiness endpoint returned HTTP 200 with successful TLS verification. The certificate still had the exact SAN and the same expiry. The backing ordinary Compose project was healthy and its four-node PubSub probe passed. It contained 2 users, 1 help request, 7 messages, and 8 applied migrations. Because that public-facing data cannot be assumed disposable, the final local regression did not migrate or replace that project without separate operator approval; current source was instead rebuilt and exercised in the isolated Compose profiles and the project-owned kind cluster.

A mode-0600 custom-format backup of that exact database was then created at output/backups/compose-20260719-060810.dump with SHA-256 726cca11f69b1aab5e7141ba5f13ebe92c7f27c6b14172891cd2e462acc069f0. Its checksum and pg_restore catalog were validated. The restore drill read 23 public tables and 1,484 rows from a uniquely named temporary database, observed PostGIS 3.6.4, applied all 10 current migrations there, passed migration readiness, and removed that database. A final read-only source check still reported 8 migrations and the same user/request/message counts; the four running application container IDs were unchanged.

The same archive then passed ./scripts/upgrade-rehearsal-compose.sh against application commit af9018f. The isolated restored copy advanced from 8 to all 10 current migrations and reported all 11 cursor indexes valid. Two web and two worker replicas formed a four-node BEAM cluster, the cross-node PubSub probe passed, public requests produced the expected production HTTPS redirects, trusted-proxy pages and both health endpoints returned HTTP 200, and the application-table count diff was empty. The exact project, database volume, networks, and one-run image were absent after cleanup. Evidence is retained at output/upgrade-rehearsal/20260719062507-3274364. The ordinary Compose container IDs, 8-migration state, and user/request/message counts remained unchanged.

After explicit approval to update the development Compose project, application commit 143e159 was deployed on 2026-07-19. Immediately before the rollout, a new mode-0600 custom-format backup was created at output/backups/compose-20260719-093528.dump; its SHA-256 is fd5f293c63972b19af9bc967ea229cc2617c17ad32d5fcd695f2dafafef4d790. The archive passed the isolated restore drill before any source-database migration.

The ordinary Compose migration job then advanced the source database from 8 to all 10 current migrations and started two web and two worker replicas from image sha256:6685247b8e872ade5e36f2b9e88bfca09b1606823ffb3e4f1469fbcfdc15d48c. The complete before/after application-table snapshot had an empty diff, including the existing 2 users, 1 request, 7 messages, 1 assignment, and 4 ended tracking sessions. Post-rollout catalog checks found the localized category-description column and all 11 valid cursor indexes. Local live and readiness checks, the four-node cluster/PubSub probe, public home, Safety, live/readiness, TLS verification, and a WebSocket Upgrade all passed. The checked web/worker log window contained no application error, warning, exception, deadlock, or HTTP 5xx match.

Headed Chrome then rendered the updated Safety page in English and Russian, redirected the protected Requests route to the Russian login form with the expected authorization notice, and reported zero console errors or warnings. The visible browser was deliberately left open. Mode-0600 rollout evidence is retained under output/staging-rollout/20260719093528-3223196/; the browser screenshot is retained under ignored .playwright-cli/.

After the Activity rollout, headed Chrome rendered the updated public navigation, followed the Activity link to the authenticated route, and received the expected login redirect and flash with zero console errors or warnings.

After commit 307794c, staging was rebuilt with the optional GitHub OAuth boundary. Both web and both worker replicas started, the cross-node PubSub probe passed, public home/readiness returned HTTP 200, and the before/after database counts remained 2 users / 1 request / 7 messages / 14 categories / 1 assignment / 0 activities / 0 reports / 0 social identities.

A headed Chrome E2E run then registered a uniquely named example.invalid account, consumed its confirmation link from local Mailpit, authenticated, opened /profile, and observed:

• the GitHub network option and the explicit provider-disabled explanation;
• no Verify button while both OAuth environment variables were absent;
• a manually added Telegram link labelled unverified;
• removal of that owner-controlled manual link;
• a direct /auth/social/github request returning to the profile with the controlled provider is not configured flash;
• zero browser console errors or warnings.

The exact E2E user was checked for every foreign-key relationship to users. It had only its one authentication token and no domain records after the manual link was removed. That exact user was then deleted through Ecto, its browser session became invalid, the generated Mailpit message was deleted individually, and the original database counts were observed again. The browser windows were left open.

Local ignored browser evidence:

• .playwright-cli/page-2026-07-18T18-09-44-303Z.png

Public Android and cross-client observation

The public-staging APK was built with the explicit ignored .env WNH_BASE_URL, installed on the API 37 emulator, and connected to https://whoneedhelp.imalto.site. Android package inspection found exactly one matching Activity for the configured HTTPS host and no matching Activity for attacker.example. A standard implicit ACTION_VIEW launch of /safety opened the native MainActivity and displayed the public Safety rules.

The manifest intentionally sets android:autoVerify="false": this is a same-origin HTTPS deep link, not a claimed verified Android App Link. Enabling verification requires the final production application ID and signing certificate fingerprint to be published in the deployment's /.well-known/assetlinks.json.

A cross-client scenario then used headed Chrome as the requester and the public-staging Android app as the helper. The observed behavior was:

• the Android app authenticated against the public HTTPS origin and rendered the MapLibre request;
• chat messages travelled in both directions, including an Android-bound browser message appearing without reload;
• Android granted foreground location permissions, started TrackingService, and showed its persistent Stop notification;
• after Home minimized the Activity, an emulator location change from 50.46009833, 30.5334 to 50.4611, 30.5344 reached PostGIS and produced 121.97 metres of observed movement;
• the headed browser received the helper marker in real time and removed it when tracking stopped;
• notification Stop removed the Android service and notification, ended the exact tracking session, left zero active sessions, and deleted its raw current position.

The browser scenario exposed one UI defect: movement evidence was persisted and the marker updated, but the evidence badge remained stale until reload. The tracking PubSub event now carries the already-derived movement/proximity timestamps, and the LiveView updates its in-memory assignment from that same event without an extra query. Domain and two-client LiveView regression tests assert the event and badge change.

That fix was deployed with image digest sha256:1b991c1e07d98babb5151c010250cd0b9458195cf3de5a91972456ee72d75673. Both web and both worker containers used the digest, the cross-node PubSub probe passed, local and public readiness returned HTTP 200, and no application error/warning was found in the checked post-rollout logs. The before/after database counts remained 2 users / 1 matched request / 7 messages / 14 categories / 1 assignment / 4 ended tracking sessions / 0 active tracking sessions / 0 activities / 0 reports / 0 social identities.

The exact temporary E2E users, tokens, request, assignment, messages, tracking session, and audit records were removed in one scoped transaction. Queries by their exact identifiers found no remaining rows and the original database counts were restored. The headed browser windows and emulator were left open.

Operations and metrics verification

On 2026-07-18, the Compose database was archived with PostgreSQL 18 custom-format pg_dump. The published archive and SHA-256 sidecar were validated before an isolated restore. The restore drill created a database from template0, restored with pg_restore --exit-on-error, read 23 public application tables and 1106 restored rows, reported PostGIS 3.6.4, observed all 8 current migrations, and removed the exact temporary database. A follow-up catalog query returned zero remaining restore-drill databases. Source counts before and after remained 2 users / 1 help request / 7 messages / 14 categories / 1 assignment / 0 activities / 0 reports / 0 social identities.

The same rollout started 2 healthy web replicas and 2 worker replicas. Local live/readiness, public HTTPS readiness, and the public root returned HTTP 200. The cross-node PubSub probe passed across all four connected BEAM nodes. The protected metrics route returned HTTP 401 both locally and through public HTTPS without credentials, returned valid Prometheus text with the generated ignored local token, and was scraped directly from each web container. Helm 4.2.3 lint and template rendering passed.

Before the runtime image change, every BEAM container logged that libsctp.so.1 was unavailable. The rebuilt Debian trixie release contains the libsctp1 package and the exact shared library; no SCTP, application error, or application warning appeared in the post-rollout web/worker logs checked during this verification window.

Final full-stack regression

The complete local regression was repeated against application commit 0ad71f6 on 2026-07-19:

• ./scripts/quality.sh passed the full static/security/dependency gate and 167/167 Phoenix tests. The isolated external-service drill also passed on the same source and removed its exact one-run project and images.
• The fresh browser project passed its 1/1 bootstrap and all 8/8 Chromium scenarios, then removed its containers, two networks, and PostgreSQL volume.
• Android debug and public-staging builds repeated unit tests, lint, and APK assembly. The network-isolated API 37 emulator passed 5/5 device tests and was removed with its one-run image. A non-HTTPS staging/release origin failed at the dedicated Gradle guard.
• The database-scale project measured 50,000 rows per large table before and after the cursor-index migration, verified consecutive cursor pages, and removed its database volume and image.
• The 3-web/2-worker load run passed 35,316/35,316 public checks, 39,108 HTTP requests with zero failures, 1,888 authenticated write/tracking chains, and 240 heartbeat WebSockets. Exact fixture cleanup restored the pre-run application-table counts and PubSub passed.
• The Compose failure drill observed both BEAM container restarts, replaced all five replicas, passed 758 readiness samples, and completed and removed the one-failure/second-attempt Oban probe. The current-image kind rollout replaced all four pod UIDs under maxUnavailable=0, passed all 362 readiness samples, cluster/PubSub, and an empty database-count diff.
• Prometheus matched all three current web targets; Grafana provisioning passed; Alertmanager delivered firing and resolved webhooks around one exact replica stop/recovery; the database-count diff was empty.
• Restic/MinIO passed encrypted stream backup, full-data check, fresh-database restore, corrupted-repository rejection, and interruption without a snapshot. Temporary buckets/databases/containers were absent afterward, the obsolete retained bucket was removed, and only the final canonical bucket remains.
• The ordinary public-facing Compose project first passed read-only live/readiness, cluster/PubSub, public HTTPS, DNS, TLS, backup/restore, and isolated upgrade rehearsal checks. After explicit approval, it was backed up again, migrated from 8 to 10 migrations, and replaced with the current 2-web/2-worker release. Its application-table count diff remained empty; schema, health, cluster/PubSub, public HTTP/TLS/WebSocket, logs, localization, and the protected-route browser flow passed after rollout.

Known work before a public production launch

• Replace the temporary staging origin with the production-owned domain and production-sign the Android app.
• Operate PostgreSQL/PostGIS with backups, recovery testing, and the required availability model.
• Load-test representative data and traffic, then set measured pool, resource, autoscaling, and action-limit policies.
• Repeat representative authenticated write, chat, tracking, and reconnect load scenarios before deriving production capacity or autoscaling policy.
• Publish jurisdiction-specific emergency contacts, privacy, retention, prohibited-items, and voluntary-payment guidance after legal review.
• Create and configure a GitHub OAuth App, then exercise the real external provider redirect/callback in a headed browser. Until then staging keeps the provider disabled and manual links remain unverified.