simpletest/who_need_help
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prepare isolated environment release workflow

Browse Source
This commit is contained in:
SimpleTest 2026-07-23 20:06:00 +03:00
parent f18e76b201
commit 777bd779ef
33 changed files with 1375 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -209,7 +209,7 @@ load project and then run:
```
The command generates MinIO and Restic secrets only in ignored mode-`0600`
`.env.load`, streams `pg_dump` directly into an encrypted Restic repository,
`output/runtime/load.env`, streams `pg_dump` directly into an encrypted Restic repository,
restores it into a new temporary database, checks corruption and interruption
failure paths, removes those temporary buckets, and retains the successful
encrypted bucket in local MinIO. It never writes a plaintext dump to the host.
@ -460,8 +460,8 @@ two worker replicas:
./scripts/e2e-run.sh
```
On its first run it generates `.env.e2e` with independent random local secrets
and mode `0600`. Browser traffic uses the isolated Traefik HTTPS entrypoint;
On its first run it generates `output/runtime/e2e.env` with independent random
local secrets and mode `0600`. Browser traffic uses the isolated Traefik HTTPS entrypoint;
Playwright accepts only that one-run proxy's generated certificate. E2E-only
fixture and failure-injection routes are enabled by a compile-time flag that is
disabled in the ordinary production image. The suite registers users through
@ -499,8 +499,8 @@ probe or the complete minimum/current API 24/30/34/37 matrix:
./scripts/android-matrix-test.sh
```
On first run it generates the ignored `.env.android-test` with a randomized
device-loopback origin and mode `0600`. The suite covers denied and granted
On first run it generates ignored `output/runtime/android-test.env` with a
randomized device-loopback origin and mode `0600`. The suite covers denied and granted
location permission, same-origin deep links, Activity recreation, foreground
location upload while the Activity is backgrounded and destroyed, the
persistent notification Stop action, a disconnected Stop request with visible
@ -628,7 +628,8 @@ Grafana datasource and dashboard, discovers each current web container as a
separate target, and exercises a firing/resolved alert by stopping and
recovering exactly one verified load replica. It prints the loopback-only
random ports and retains non-secret evidence below `output/observability/`.
The generated Grafana password remains only in mode-`0600` `.env.load`.
The generated Grafana password remains only in mode-`0600`
`output/runtime/load.env`.
Stop only the monitoring services while leaving their local metric volumes and
the load application running:

22
android/README.md
View File

@ -77,7 +77,7 @@ source in the Phoenix environment; never put that private JSON in the Android
build.
```sh
WNH_ENV_FILE=.env.production ./scripts/android-release-build.sh
./scripts/android-release-build.sh
```
The build passes the private files with Docker BuildKit secret mounts, runs
@ -109,11 +109,12 @@ from the existing `PHX_HOST`, `PHX_SCHEME`, and `PHX_URL_PORT`, then build:
sha256sum android/dist-staging/who-need-help-staging.apk
```
This variant uses Android's generic debug signing key so it can be installed
for staging verification. It is not a production-signed artifact and must not
be published as a release. The manifest accepts same-origin HTTPS deep links,
but verified Android App Links additionally require the final signing
certificate fingerprint in the deployment's `/.well-known/assetlinks.json`.
This variant uses the dedicated stable staging key below
`~/.config/who_need_help/android-staging/`. It is not the production upload
identity and must not be published as a production release. The manifest
accepts same-origin HTTPS deep links, while verified Android App Links require
this staging certificate fingerprint in the dev deployment's
`/.well-known/assetlinks.json`.
With the temporary public origin reachable, run the API 37 emulator smoke test:
@ -151,10 +152,11 @@ repository root:
./scripts/android-matrix-test.sh
```
The command requires `/dev/kvm`. It generates an ignored
`.env.android-test` once with a randomized `http://127.0.0.1:PORT` origin and
mode `0600`; the application and its in-process fixture server both derive the
origin from that file. It then builds both APKs, boots a fresh selected emulator
The command requires `/dev/kvm`. It generates ignored
`output/runtime/android-test.env` once with a randomized
`http://127.0.0.1:PORT` origin and mode `0600`; the application and its
in-process fixture server both derive the origin from that file. It then
builds both APKs, boots a fresh selected emulator
container without external networking, injects emulator coordinates, and runs
`AndroidJUnitRunner`. `WNH_ANDROID_TEST_API` selects one supported API, while
`WNH_ANDROID_TEST_API_MATRIX` controls the matrix command.

103
docs/operations.md
View File

@ -64,6 +64,20 @@ connections, Docker volumes, public aliases, Google OAuth clients, email
delivery paths, and generated secrets. Oban queues are isolated by those
different PostgreSQL databases. There is no Redis dependency.
Generated harness state is not a deployment environment. E2E, load, and
Android instrumentation scripts keep their random local inputs below the
ignored mode-`0700` `output/runtime/` directory:
```text
output/runtime/e2e.env
output/runtime/load.env
output/runtime/android-test.env
```
Those files are generated automatically, never copied to a server, and do not
represent dev, test, or production. The one ignored `.env` at each checkout
root remains the only application/deployment configuration.
The shared Caddy edge is owned only by the production checkout and reads the
same production `.env`; it is not a third project directory or a second secret
file. Both applications intentionally share only the external
@ -116,6 +130,41 @@ to the comma-separated App Links value; the upload certificate alone does not
describe Play-delivered APKs. The production environment validator accepts
multiple SHA-256 fingerprints and rejects partial or malformed configuration.
### Release capability inputs
Each environment owns distinct external-provider credentials. Seed a new
production `.env` with `scripts/init-production-env.sh` and the corresponding
`PRODUCTION_*` process variables; use `TEST_*` only when creating the separate
test checkout. The generated file uses the ordinary runtime names:
| Capability | Values kept in that checkout's `.env` |
| --- | --- |
| Google sign-in | `GOOGLE_OAUTH_CLIENT_ID`, `GOOGLE_OAUTH_CLIENT_SECRET` |
| Browser push | three `WEB_PUSH_VAPID_*` values |
| Android Firebase client | four public `WNH_FIREBASE_*` values |
| Android delivery | `FCM_PROJECT_ID` and one private service-account source |
| Verified Android links | `ANDROID_APP_LINKS_PACKAGE_NAME`, `ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS` |
| Transactional email | `SMTP_*`, sender, and `SUPPORT_INBOX_ADDRESS` |
Do not reuse a Google client, VAPID private key, Firebase project/service
account, SMTP credential, or Android signing key between dev and production.
The public Firebase Android values are build configuration; the Base64 FCM
service-account JSON is a server secret and must never be passed into the
Android build.
Check an environment without printing its secret values:
```bash
./scripts/check-environment-readiness.sh .env
./scripts/check-environment-readiness.sh .env --require-release
```
The first command reports incomplete or local-only capabilities. The second is
a blocking release preflight and exits nonzero until Google sign-in, external
SMTP, browser Web Push, Android Firebase/FCM, App Links, support routing,
Android version/signing aliases, and the core application configuration are
all complete.
Create the test configuration inside the test checkout:
```bash
@ -349,7 +398,7 @@ The isolated load project can run a complete encrypted Restic/MinIO drill:
The script refuses the staging Compose project and validates the project and
service labels of every pre-existing container in its scope. On first use,
`scripts/ensure-local-load-env.sh` generates independent random MinIO and
Restic credentials in ignored `.env.load` and restricts that file to mode
Restic credentials in ignored `output/runtime/load.env` and restricts that file to mode
`0600`. MinIO publishes Docker-assigned ports only on `127.0.0.1`; the observed
API and console URLs are printed after a successful run.
@ -493,7 +542,7 @@ against an isolated restored copy:
The rehearsal validates the checksum and archive catalog, reads the public
origin configuration from ignored `.env`, and uses only the independently
generated credentials in ignored mode-`0600` `.env.e2e`. It builds a uniquely
generated credentials in ignored mode-`0600` `output/runtime/e2e.env`. It builds a uniquely
tagged production release, creates a uniquely named Compose project and
database from `template0`, restores the archive, records application-table
counts, and then:
@ -720,7 +769,7 @@ rewrite while preserving the target's own `instance` label.
Prometheus, Alertmanager, and Grafana are pinned by tag and digest. Their host
ports default to Docker-assigned values bound only to `127.0.0.1`; the run
prints the observed URLs. Grafana uses the random admin password generated in
ignored `.env.load`, disables anonymous signup, update checks, suggested plugin
ignored `output/runtime/load.env`, disables anonymous signup, update checks, suggested plugin
installation, and its unused built-in alert engine. The Prometheus datasource
and ten-panel dashboard are provisioned from tracked files. The dashboard
separates all discovered web and worker replicas and includes HTTP
@ -747,6 +796,54 @@ destinations, production availability, and measured alert policies remain
deployment decisions. In Kubernetes, put the metrics token in
`existingSecret`; configure the external scraper to send it as a Bearer token.
## Production release without pushing the frozen repository
The post-submission workflow keeps the public Git repository and
`test.whoneedhelp.com` untouched. A clean local commit is packaged as a
verified Git bundle and transferred directly over SSH to only
`/srv/who_need_help-production`:
```bash
./scripts/production-release.sh plan whoneedhelp
```
The default `plan` action is read-only. It verifies the exact local and remote
commits, requires a fast-forward history, checks the production checkout,
Compose scope and healthy containers, checks public readiness, opens a
read-only PostgreSQL connection, and runs the complete environment capability
preflight. It neither uploads a bundle nor creates a backup.
After reviewing the exact commit printed by the plan, execution additionally
requires an explicit per-commit confirmation:
```bash
WNH_PRODUCTION_RELEASE_CONFIRM=whoneedhelp.com:FULL_COMMIT \
./scripts/production-release.sh apply whoneedhelp
```
The apply path refuses tracked local or remote modifications. It then:
1. creates and verifies a full Git bundle for exactly that clean commit;
2. uploads only that bundle to the production checkout's ignored
`output/releases/`;
3. creates a custom-format PostgreSQL 18 backup without exposing the database
password in process arguments;
4. verifies its archive catalog and SHA-256, then copies and verifies the
backup again under local ignored `output/production-backups/`;
5. fast-forwards the production checkout without accessing or changing the
test checkout or public remote;
6. selects immutable per-commit image tags, applies migrations, starts the
application and edge, and verifies the public readiness and Android App
Links endpoints.
If application startup fails after the new image tags are selected, the script
restores the previous immutable application and Caddy image tags and attempts
to recover public readiness. It deliberately does not reverse Git source or
Ecto migrations automatically. The per-release rollback manifest and backup
paths are recorded below the production checkout's ignored `output/releases/`.
The copied backup is separate from the production host, but a long-term
encrypted off-site backup destination remains an operational requirement.
## Rollback boundary
The release image is immutable and migrations run as a separate one-shot role.

12
docs/performance.md
View File

@ -104,19 +104,19 @@ wrapper, it deliberately keeps the isolated database volume between commands.
./scripts/load-stack-up.sh
```
The generated `.env.load` is ignored, restricted to mode 600, and contains
independent PostgreSQL and application secrets. Edit its `LOAD_*` inputs to
The generated `output/runtime/load.env` is ignored, restricted to mode 600,
and contains independent PostgreSQL and application secrets. Edit its `LOAD_*` inputs to
define a specific experiment. Values in `.env.load.example` are reproducible
measurement points, not recommendations.
For a one-run duration that must not rewrite `.env.load`, pass an explicit
For a one-run duration that must not rewrite `output/runtime/load.env`, pass an explicit
experiment override:
```sh
LOAD_DURATION_OVERRIDE=10m ./scripts/load-run.sh local-soak-YYYYMMDD
```
The effective value and whether it came from `.env.load` or the override are
The effective value and whether it came from `output/runtime/load.env` or the override are
recorded in that run's `environment.txt`.
## Run and compare replica counts
@ -124,7 +124,7 @@ recorded in that run's `environment.txt`.
```sh
./scripts/load-run.sh two-web
./scripts/load-stack-up.sh 3
# Set LOAD_WEB_REPLICAS=3 in .env.load so the recorded expected topology
# Set LOAD_WEB_REPLICAS=3 in output/runtime/load.env so the recorded expected topology
# matches the running topology, then:
./scripts/load-run.sh three-web
```
@ -162,7 +162,7 @@ contexts. The pair count equals the maximum simultaneous VU count across all
scenarios, because k6 may reuse its global VU pool between parallel scenarios;
mapping each global VU identifier directly to its own pair prevents concurrent
users from sharing a tracking assignment. The random fixture password exists
only in ignored mode-600 `.env.load` and is never written to the manifest or
only in ignored mode-600 `output/runtime/load.env` and is never written to the manifest or
console.
On success and on trapped failure, cleanup deletes only the UUIDs recorded in

4
scripts/android-instrumentation-test.sh
View File

@ -3,7 +3,7 @@ set -eu
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ANDROID_ENV="$ROOT/.env"
TEST_ENV="$ROOT/.env.android-test"
TEST_ENV=${WNH_ANDROID_TEST_ENV_FILE:-"$ROOT/output/runtime/android-test.env"}
run_id=$(date -u +%Y%m%d%H%M%S)-$$
android_api=${WNH_ANDROID_TEST_API:-37.0}
# 1G is measured against the API 30/34/37 suite and is also exercised by the
@ -63,7 +63,7 @@ set -a
. "$TEST_ENV"
set +a
: "${WNH_ANDROID_TEST_BASE_URL:?Set WNH_ANDROID_TEST_BASE_URL in .env.android-test}"
: "${WNH_ANDROID_TEST_BASE_URL:?Generate Android test runtime state with scripts/ensure-local-android-test-env.sh}"
: "${WNH_TRACKING_MIN_TIME_MS:?Set WNH_TRACKING_MIN_TIME_MS in .env}"
: "${WNH_TRACKING_HTTP_TIMEOUT_MS:?Set WNH_TRACKING_HTTP_TIMEOUT_MS in .env}"

245
scripts/backup-external-postgres.sh Executable file
View File

@ -0,0 +1,245 @@
#!/usr/bin/env bash
set -euo pipefail
umask 077
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
env_file=${1:-"$ROOT/.env"}
target=${2:-}
expected_environment=${3:-}
if [[ "$env_file" != /* ]]; then
env_file="$ROOT/$env_file"
fi
if [[ ! -f "$env_file" ]]; then
echo "Environment file does not exist: $env_file" >&2
exit 2
fi
if [[ "$(stat -c '%a' "$env_file")" != 600 ]]; then
echo "Environment file must have mode 0600: $env_file" >&2
exit 2
fi
read_value() {
local key=$1
awk -v key="$key" '
index($0, key "=") == 1 {
print substr($0, length(key) + 2)
found = 1
exit
}
END { if (!found) exit 1 }
' "$env_file"
}
deployment_environment=$(read_value DEPLOYMENT_ENV)
compose_project=$(read_value COMPOSE_PROJECT_NAME)
database_mode=$(read_value DATABASE_MODE)
if [[ "$database_mode" != "external" ]]; then
echo "backup-external-postgres.sh requires DATABASE_MODE=external." >&2
exit 2
fi
if [[ -n "$expected_environment" &&
"$deployment_environment" != "$expected_environment" ]]; then
echo "Environment mismatch: expected $expected_environment." >&2
exit 2
fi
for command in awk pg_dump pg_restore psql python3 sha256sum stat; do
command -v "$command" >/dev/null 2>&1 || {
echo "Required command is unavailable: $command" >&2
exit 2
}
done
mapfile -d '' -t connection_parts < <(
python3 - "$env_file" <<'PY'
import sys
from pathlib import Path
from urllib.parse import unquote, urlsplit
values = {}
for line in Path(sys.argv[1]).read_text().splitlines():
if "=" in line and not line.startswith("#"):
key, value = line.split("=", 1)
values.setdefault(key, value)
raw_url = values.get("DATABASE_URL", "")
if raw_url.startswith("ecto://"):
raw_url = "postgresql://" + raw_url[len("ecto://"):]
parsed = urlsplit(raw_url)
required = {
"host": parsed.hostname,
"database": parsed.path.lstrip("/"),
"username": parsed.username,
"password": parsed.password,
}
missing = [name for name, value in required.items() if not value]
if missing:
raise SystemExit("DATABASE_URL is missing: " + ", ".join(missing))
parts = (
parsed.hostname,
str(parsed.port or 5432),
unquote(parsed.path.lstrip("/")),
unquote(parsed.username),
unquote(parsed.password),
values.get("DATABASE_SOCKET_DIR", ""),
)
sys.stdout.buffer.write(b"\0".join(part.encode() for part in parts) + b"\0")
PY
)
if [[ ${#connection_parts[@]} -ne 6 ]]; then
echo "Could not parse the external PostgreSQL connection without exposing it." >&2
exit 2
fi
database_host=${connection_parts[0]}
database_port=${connection_parts[1]}
database_name=${connection_parts[2]}
database_user=${connection_parts[3]}
database_password=${connection_parts[4]}
database_socket_dir=${connection_parts[5]}
connection_host=${database_socket_dir:-$database_host}
escape_pgpass() {
local escaped=$1
escaped=${escaped//\\/\\\\}
escaped=${escaped//:/\\:}
printf '%s' "$escaped"
}
pgpass_file=$(mktemp "${TMPDIR:-/tmp}/who-need-help-pgpass.XXXXXX")
partial=
checksum_partial=
metadata_partial=
cleanup() {
rm -f "$pgpass_file"
[[ -z "$partial" ]] || rm -f "$partial"
[[ -z "$checksum_partial" ]] || rm -f "$checksum_partial"
[[ -z "$metadata_partial" ]] || rm -f "$metadata_partial"
}
trap cleanup EXIT HUP INT TERM
chmod 600 "$pgpass_file"
printf '%s:%s:%s:%s:%s\n' \
"$(escape_pgpass "$database_host")" \
"$(escape_pgpass "$database_port")" \
"$(escape_pgpass "$database_name")" \
"$(escape_pgpass "$database_user")" \
"$(escape_pgpass "$database_password")" >"$pgpass_file"
psql_args=(
--host "$connection_host"
--port "$database_port"
--username "$database_user"
--dbname "$database_name"
--no-password
--no-psqlrc
--tuples-only
--no-align
--set ON_ERROR_STOP=1
)
server_version=$(
PGPASSFILE="$pgpass_file" PGOPTIONS="-c default_transaction_read_only=on" \
psql "${psql_args[@]}" --command="show server_version"
)
server_database=$(
PGPASSFILE="$pgpass_file" PGOPTIONS="-c default_transaction_read_only=on" \
psql "${psql_args[@]}" --command="select current_database()"
)
if [[ "$server_database" != "$database_name" ]]; then
echo "PostgreSQL connected to an unexpected database." >&2
exit 2
fi
printf 'Environment: %s\n' "$deployment_environment"
printf 'Compose project: %s\n' "$compose_project"
printf 'Database mode: external\n'
printf 'Database endpoint: %s:%s/%s\n' "$connection_host" "$database_port" "$database_name"
printf 'PostgreSQL server: %s\n' "$server_version"
printf 'PostgreSQL client: %s\n' "$(pg_dump --version)"
if [[ "$target" == "--check-only" ]]; then
echo "Read-only external PostgreSQL connection check passed."
exit 0
fi
if [[ -z "$target" ]]; then
echo "Usage: $0 ENV_FILE OUTPUT_DUMP [EXPECTED_ENVIRONMENT]" >&2
echo " $0 ENV_FILE --check-only [EXPECTED_ENVIRONMENT]" >&2
exit 2
fi
if [[ "$target" != /* ]]; then
target="$ROOT/$target"
fi
target_dir=$(dirname -- "$target")
target_name=$(basename -- "$target")
checksum="$target.sha256"
metadata="$target.metadata"
if [[ -e "$target" || -e "$checksum" || -e "$metadata" ]]; then
echo "Refusing to overwrite an existing backup, checksum, or metadata file." >&2
exit 2
fi
mkdir -p "$target_dir"
chmod 700 "$target_dir"
partial="$target.partial.$$"
checksum_partial="$checksum.partial.$$"
metadata_partial="$metadata.partial.$$"
printf 'Backup target: %s\n' "$target"
echo "The source database is read only; the command will create one custom-format dump plus checksum and non-secret metadata."
PGPASSFILE="$pgpass_file" pg_dump \
--host "$connection_host" \
--port "$database_port" \
--username "$database_user" \
--dbname "$database_name" \
--no-password \
--format custom \
--no-owner \
--no-acl \
--file "$partial"
[[ -s "$partial" ]] || {
echo "PostgreSQL produced an empty backup." >&2
exit 1
}
pg_restore --list "$partial" >/dev/null
hash=$(sha256sum "$partial" | awk '{print $1}')
printf '%s %s\n' "$hash" "$target_name" >"$checksum_partial"
{
printf 'deployment_environment=%s\n' "$deployment_environment"
printf 'compose_project=%s\n' "$compose_project"
printf 'database_name=%s\n' "$database_name"
printf 'server_version=%s\n' "$server_version"
printf 'client_version=%s\n' "$(pg_dump --version)"
printf 'created_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf 'sha256=%s\n' "$hash"
} >"$metadata_partial"
chmod 600 "$partial" "$checksum_partial" "$metadata_partial"
sync -f "$partial" "$checksum_partial" "$metadata_partial"
mv "$partial" "$target"
mv "$checksum_partial" "$checksum"
mv "$metadata_partial" "$metadata"
partial=
checksum_partial=
metadata_partial=
echo "External PostgreSQL backup catalog and checksum passed verification."
printf 'Backup: %s\nChecksum: %s\nMetadata: %s\n' "$target" "$checksum" "$metadata"

4
scripts/backup-s3-drill.sh
View File

@ -3,7 +3,7 @@ set -euo pipefail
umask 077
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE="$ROOT/.env.load"
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LABEL=${1:-"backup-$(date -u +%Y%m%d%H%M%S)"}
if [[ ! -f "$ENV_FILE" ]]; then
@ -22,7 +22,7 @@ for name in LOAD_PROJECT POSTGRES_DB POSTGRES_USER POSTGRES_PASSWORD DATABASE_UR
BACKUP_TIMEOUT_SECONDS BACKUP_INTERRUPTION_CHUNKS \
BACKUP_INTERRUPTION_CHUNK_BYTES BACKUP_INTERRUPTION_INTERVAL_SECONDS; do
if [[ -z "${!name:-}" ]]; then
echo "$name is missing from .env.load" >&2
echo "$name is missing from the generated load runtime environment" >&2
exit 1
fi
done

213
scripts/check-environment-readiness.sh Executable file
View File

@ -0,0 +1,213 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
env_file=${1:-"$ROOT/.env"}
mode=${2:-}
if [[ "$env_file" != /* ]]; then
env_file="$ROOT/$env_file"
fi
if [[ "$mode" != "" && "$mode" != "--require-release" ]]; then
echo "Usage: $0 [ENV_FILE] [--require-release]" >&2
exit 2
fi
if [[ ! -f "$env_file" ]]; then
echo "Environment file does not exist: $env_file" >&2
exit 2
fi
if [[ "$(stat -c '%a' "$env_file")" != 600 ]]; then
echo "Environment file must have mode 0600: $env_file" >&2
exit 2
fi
read_value() {
local key=$1
awk -v key="$key" '
index($0, key "=") == 1 {
value = substr($0, length(key) + 2)
if (value ~ /^".*"$/ || value ~ /^'\''.*'\''$/) {
value = substr(value, 2, length(value) - 2)
}
print value
found = 1
exit
}
END { if (!found) exit 1 }
' "$env_file"
}
value() {
read_value "$1" 2>/dev/null || true
}
is_set() {
[[ -n "$(value "$1")" ]]
}
all_set() {
local key
for key in "$@"; do
is_set "$key" || return 1
done
}
all_empty() {
local key
for key in "$@"; do
is_set "$key" && return 1
done
return 0
}
contains_template_marker() {
local observed=$1
[[ "$observed" == *REPLACE* || "$observed" == *GENERATE* ||
"$observed" == *example.com* || "$observed" == *example.invalid* ]]
}
failures=0
warnings=0
ready() {
printf 'READY %-24s %s\n' "$1" "$2"
}
local_only() {
printf 'LOCAL_ONLY %-24s %s\n' "$1" "$2"
warnings=$((warnings + 1))
}
missing() {
printf 'MISSING %-24s %s\n' "$1" "$2"
failures=$((failures + 1))
}
invalid() {
printf 'INVALID %-24s %s\n' "$1" "$2"
failures=$((failures + 1))
}
partial() {
printf 'PARTIAL %-24s %s\n' "$1" "$2"
failures=$((failures + 1))
}
deployment_env=$(value DEPLOYMENT_ENV)
phx_host=$(value PHX_HOST)
phx_scheme=$(value PHX_SCHEME)
phx_port=$(value PHX_URL_PORT)
base_url=$(value WNH_BASE_URL)
debug_base_url=$(value WNH_DEBUG_BASE_URL)
if all_set DEPLOYMENT_ENV PHX_HOST PHX_SCHEME PHX_URL_PORT WNH_BASE_URL WNH_DEBUG_BASE_URL &&
[[ "$base_url" == "$debug_base_url" ]] &&
[[ "$base_url" == "$phx_scheme://$phx_host" ||
"$base_url" == "$phx_scheme://$phx_host:$phx_port" ]] &&
! contains_template_marker "$base_url"; then
ready "public origin" "deployment=$deployment_env; one canonical Android/web origin"
else
invalid "public origin" "DEPLOYMENT_ENV/PHX_*/WNH_*_BASE_URL are incomplete or inconsistent"
fi
if all_set SECRET_KEY_BASE HANDOVER_SECRET RELEASE_COOKIE METRICS_TOKEN; then
ready "application secrets" "four required independent values are present"
else
missing "application secrets" "SECRET_KEY_BASE, HANDOVER_SECRET, RELEASE_COOKIE, METRICS_TOKEN"
fi
smtp_relay=$(value SMTP_RELAY)
email_delivery_provider=$(value EMAIL_DELIVERY_PROVIDER)
email_delivery_provider=${email_delivery_provider:-smtp}
if [[ "$email_delivery_provider" != "smtp" ]]; then
invalid "transactional email" "EMAIL_DELIVERY_PROVIDER must be smtp"
elif ! all_set SMTP_RELAY SMTP_PORT SMTP_AUTH SMTP_TLS SMTP_SSL EMAIL_FROM_ADDRESS; then
missing "transactional email" "SMTP transport and sender fields"
elif [[ "$smtp_relay" == "mailpit" ]]; then
local_only "transactional email" "Mailpit captures messages locally; it cannot deliver public email"
elif [[ "$(value SMTP_AUTH)" == "always" ]] && ! all_set SMTP_USERNAME SMTP_PASSWORD; then
partial "transactional email" "authenticated SMTP requires both username and password"
else
ready "transactional email" "external SMTP transport is configured"
fi
if is_set SUPPORT_INBOX_ADDRESS; then
ready "support inbox" "operator destination is configured"
else
missing "support inbox" "SUPPORT_INBOX_ADDRESS"
fi
if all_empty GOOGLE_OAUTH_CLIENT_ID GOOGLE_OAUTH_CLIENT_SECRET; then
missing "Google sign-in" "GOOGLE_OAUTH_CLIENT_ID and GOOGLE_OAUTH_CLIENT_SECRET"
elif all_set GOOGLE_OAUTH_CLIENT_ID GOOGLE_OAUTH_CLIENT_SECRET; then
ready "Google sign-in" "client ID and secret are both configured"
else
partial "Google sign-in" "client ID and secret must be configured together"
fi
if all_empty WEB_PUSH_VAPID_PUBLIC_KEY WEB_PUSH_VAPID_PRIVATE_KEY WEB_PUSH_VAPID_SUBJECT; then
missing "browser Web Push" "VAPID public/private keys and subject"
elif all_set WEB_PUSH_VAPID_PUBLIC_KEY WEB_PUSH_VAPID_PRIVATE_KEY WEB_PUSH_VAPID_SUBJECT; then
case "$(value WEB_PUSH_VAPID_SUBJECT)" in
mailto:* | https://*) ready "browser Web Push" "complete VAPID configuration" ;;
*) invalid "browser Web Push" "WEB_PUSH_VAPID_SUBJECT must use mailto: or https://" ;;
esac
else
partial "browser Web Push" "all three VAPID values are required together"
fi
if all_empty WNH_FIREBASE_APPLICATION_ID WNH_FIREBASE_API_KEY \
WNH_FIREBASE_PROJECT_ID WNH_FIREBASE_GCM_SENDER_ID; then
missing "Android Firebase client" "four WNH_FIREBASE_* Android client values"
elif all_set WNH_FIREBASE_APPLICATION_ID WNH_FIREBASE_API_KEY \
WNH_FIREBASE_PROJECT_ID WNH_FIREBASE_GCM_SENDER_ID; then
ready "Android Firebase client" "complete client configuration"
else
partial "Android Firebase client" "all four WNH_FIREBASE_* values are required together"
fi
fcm_file=$(value FCM_SERVICE_ACCOUNT_FILE)
fcm_base64=$(value FCM_SERVICE_ACCOUNT_JSON_BASE64)
if [[ -z "$(value FCM_PROJECT_ID)" && -z "$fcm_file" && -z "$fcm_base64" ]]; then
missing "Android FCM delivery" "FCM project ID and one service-account source"
elif [[ -z "$(value FCM_PROJECT_ID)" || (-n "$fcm_file" && -n "$fcm_base64") ||
(-z "$fcm_file" && -z "$fcm_base64") ]]; then
partial "Android FCM delivery" "project ID and exactly one credential source are required"
elif [[ -n "$fcm_file" ]]; then
if [[ "$fcm_file" == /* && -r "$fcm_file" ]]; then
ready "Android FCM delivery" "readable service-account file is configured"
else
invalid "Android FCM delivery" "FCM_SERVICE_ACCOUNT_FILE must be an absolute readable file"
fi
elif printf '%s' "$fcm_base64" | base64 --decode 2>/dev/null |
jq -e '.type == "service_account" and (.project_id | type == "string")' >/dev/null 2>&1; then
ready "Android FCM delivery" "valid Base64 service-account document is configured"
else
invalid "Android FCM delivery" "Base64 credential is not a service-account JSON document"
fi
if all_empty ANDROID_APP_LINKS_PACKAGE_NAME ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS; then
missing "Android App Links" "package name and signing certificate fingerprint"
elif all_set ANDROID_APP_LINKS_PACKAGE_NAME ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS; then
ready "Android App Links" "package and signing fingerprints are configured"
else
partial "Android App Links" "package and signing fingerprints must be configured together"
fi
if all_set WNH_ANDROID_VERSION_CODE WNH_ANDROID_VERSION_NAME \
WNH_ANDROID_SIGNING_KEY_ALIAS WNH_ANDROID_STAGING_SIGNING_KEY_ALIAS; then
ready "Android release inputs" "version and separate production/staging signing aliases are present"
else
missing "Android release inputs" "version code/name and both signing aliases"
fi
printf '\nEnvironment readiness: %d blocking item(s), %d local-only warning(s).\n' \
"$failures" "$warnings"
if [[ "$mode" == "--require-release" && ($failures -ne 0 || $warnings -ne 0) ]]; then
exit 1
fi

3
scripts/clean-deploy-verify.sh
View File

@ -123,8 +123,7 @@ fi
source_commit=$(git rev-parse --verify HEAD)
git archive --format=tar "$source_commit" | tar -xf - -C "$workspace"
if [[ -e "$workspace/.git" || -e "$workspace/output" || -e "$workspace/.env.load" ||
-e "$workspace/.env.e2e" ]]; then
if [[ -e "$workspace/.git" || -e "$workspace/output" ]]; then
echo "The tracked archive unexpectedly contains local state." >&2
exit 1
fi

3
scripts/e2e-run.sh
View File

@ -5,6 +5,7 @@ ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
cd "$ROOT"
"$ROOT/scripts/ensure-local-e2e-env.sh"
E2E_ENV=${WNH_E2E_ENV_FILE:-"$ROOT/output/runtime/e2e.env"}
run_id="$(date -u +%Y%m%d%H%M%S)-$$"
project="who_need_help_e2e_$run_id"
@ -44,7 +45,7 @@ export MAP_TILE_URL="/__e2e__/map-tile.png?z={z}&x={x}&y={y}"
compose() {
docker compose \
--project-name "$project" \
--env-file "$ROOT/.env.e2e" \
--env-file "$E2E_ENV" \
--file "$ROOT/compose.yaml" \
--file "$ROOT/compose.e2e.yaml" \
"$@"

8
scripts/e2e-stop.sh
View File

@ -17,8 +17,14 @@ case "$project" in
esac
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
E2E_ENV=${WNH_E2E_ENV_FILE:-"$ROOT/output/runtime/e2e.env"}
run_id=${project#who_need_help_e2e_}
if [ ! -f "$E2E_ENV" ]; then
echo "Missing generated E2E runtime state: $E2E_ENV" >&2
exit 1
fi
# Compose still resolves required interpolation values for `down`. These values
# mirror the exact per-run tags. The project label passed above remains the
# Compose resource scope.
@ -38,7 +44,7 @@ export MAP_TILE_URL="/__e2e__/map-tile.png?z={z}&x={x}&y={y}"
docker compose \
--project-name "$project" \
--env-file "$ROOT/.env.e2e" \
--env-file "$E2E_ENV" \
--file "$ROOT/compose.yaml" \
--file "$ROOT/compose.e2e.yaml" \
down --remove-orphans

11
scripts/ensure-local-android-test-env.sh
View File

@ -2,10 +2,15 @@
set -eu
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
target="$ROOT/.env.android-test"
runtime_dir=${WNH_RUNTIME_ENV_DIR:-"$ROOT/output/runtime"}
target=${WNH_ANDROID_TEST_ENV_FILE:-"$runtime_dir/android-test.env"}
mkdir -p "$runtime_dir"
chmod 700 "$runtime_dir"
if [ -f "$target" ]; then
echo ".env.android-test already exists; no setting was changed."
chmod 600 "$target"
echo "The generated Android test runtime environment already exists; no setting was changed."
exit 0
fi
@ -16,4 +21,4 @@ umask 077
} > "$target"
chmod 600 "$target"
echo "Generated .env.android-test with an isolated device-loopback origin and mode 0600."
echo "Generated isolated Android test runtime state in output/runtime/android-test.env with mode 0600."

10
scripts/ensure-local-e2e-env.sh
View File

@ -3,7 +3,11 @@ set -eu
umask 077
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
target="$ROOT/.env.e2e"
runtime_dir=${WNH_RUNTIME_ENV_DIR:-"$ROOT/output/runtime"}
target=${WNH_E2E_ENV_FILE:-"$runtime_dir/e2e.env"}
mkdir -p "$runtime_dir"
chmod 700 "$runtime_dir"
if [ -f "$target" ]; then
chmod 600 "$target"
@ -36,7 +40,7 @@ if [ -f "$target" ]; then
if [ "$updated" = true ]; then
echo "Updated non-secret E2E transport inputs; existing secrets were preserved."
else
echo ".env.e2e already exists; no secret or experiment input was changed."
echo "The generated E2E runtime environment already exists; no secret or experiment input was changed."
fi
exit 0
@ -105,4 +109,4 @@ E2E_OUTPUT_DIR=$ROOT/output/e2e/generated-per-run
EOF
chmod 600 "$target"
echo "Generated .env.e2e with independent local secrets and mode 0600."
echo "Generated isolated E2E runtime state in output/runtime/e2e.env with mode 0600."

14
scripts/ensure-local-load-env.sh
View File

@ -3,7 +3,11 @@ set -eu
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
TEMPLATE=${WNH_LOAD_ENV_TEMPLATE:-"$ROOT/.env.load.example"}
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/.env.load"}
runtime_dir=${WNH_RUNTIME_ENV_DIR:-"$ROOT/output/runtime"}
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$runtime_dir/load.env"}
mkdir -p "$runtime_dir"
chmod 700 "$runtime_dir"
for command in openssl perl; do
if ! command -v "$command" >/dev/null 2>&1; then
@ -156,7 +160,7 @@ GOOGLE_OAUTH_HTTP_RECEIVE_TIMEOUT_MS
[ "$needs_backup_interruption_chunks" = false ] &&
[ "$needs_backup_interruption_chunk_bytes" = false ] &&
[ "$needs_backup_interruption_interval" = false ]; then
echo ".env.load already exists; no secret or experiment input was changed."
echo "The generated load runtime environment already exists; no secret or experiment input was changed."
exit 0
fi
@ -356,7 +360,7 @@ GOOGLE_OAUTH_HTTP_RECEIVE_TIMEOUT_MS
needs_backup_bucket_prefix needs_backup_timeout \
needs_backup_interruption_chunks needs_backup_interruption_chunk_bytes \
needs_backup_interruption_interval missing_template_keys template_upgrade_keys
echo "Added missing deployment/queue/load/resilience/observability/backup inputs to ignored .env.load."
echo "Added missing deployment/queue/load/resilience/observability/backup inputs to the generated load runtime environment."
exit 0
fi
@ -404,7 +408,7 @@ DOCKER_SOCKET_GID_VALUE=$docker_socket_gid \
' "$TEMPLATE" >"$temporary"
if grep -Eq '^[A-Z0-9_]+=GENERATE_' "$temporary"; then
echo "A secret marker was not replaced; refusing to publish .env.load." >&2
echo "A secret marker was not replaced; refusing to publish the generated load runtime environment." >&2
exit 1
fi
@ -416,4 +420,4 @@ unset postgres_password secret_key_base handover_secret release_cookie metrics_t
backup_minio_root_user backup_minio_root_password backup_restic_password \
database_url
echo "Generated independent load-profile secrets in ignored .env.load."
echo "Generated independent load-profile runtime state in output/runtime/load.env with mode 0600."

57
scripts/init-production-env.sh
View File

@ -53,6 +53,15 @@ public_upstream_name=${PRODUCTION_PUBLIC_UPSTREAM_NAME:-who-need-help-production
codex_session_id=${PRODUCTION_CODEX_SESSION_ID:-}
google_oauth_client_id=${PRODUCTION_GOOGLE_OAUTH_CLIENT_ID:-}
google_oauth_client_secret=${PRODUCTION_GOOGLE_OAUTH_CLIENT_SECRET:-}
web_push_vapid_public_key=${PRODUCTION_WEB_PUSH_VAPID_PUBLIC_KEY:-}
web_push_vapid_private_key=${PRODUCTION_WEB_PUSH_VAPID_PRIVATE_KEY:-}
web_push_vapid_subject=${PRODUCTION_WEB_PUSH_VAPID_SUBJECT:-}
firebase_application_id=${PRODUCTION_WNH_FIREBASE_APPLICATION_ID:-}
firebase_api_key=${PRODUCTION_WNH_FIREBASE_API_KEY:-}
firebase_project_id=${PRODUCTION_WNH_FIREBASE_PROJECT_ID:-}
firebase_sender_id=${PRODUCTION_WNH_FIREBASE_GCM_SENDER_ID:-}
fcm_project_id=${PRODUCTION_FCM_PROJECT_ID:-}
fcm_service_account_json_base64=${PRODUCTION_FCM_SERVICE_ACCOUNT_JSON_BASE64:-}
android_app_links_package_name=${PRODUCTION_ANDROID_APP_LINKS_PACKAGE_NAME:-}
android_app_links_fingerprints=${PRODUCTION_ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS:-}
test_domain=${PRODUCTION_TEST_DOMAIN:-"test.$domain"}
@ -77,6 +86,34 @@ if { [ -n "$android_app_links_package_name" ] || [ -n "$android_app_links_finger
exit 1
fi
firebase_values="$firebase_application_id $firebase_api_key $firebase_project_id $firebase_sender_id"
if printf '%s\n' "$firebase_values" | grep -q '[^[:space:]]'; then
for value in "$firebase_application_id" "$firebase_api_key" \
"$firebase_project_id" "$firebase_sender_id"; do
[ -n "$value" ] || {
echo "All four production WNH_FIREBASE_* values must be configured together." >&2
exit 1
}
done
fi
vapid_values="$web_push_vapid_public_key $web_push_vapid_private_key $web_push_vapid_subject"
if printf '%s\n' "$vapid_values" | grep -q '[^[:space:]]'; then
for value in "$web_push_vapid_public_key" "$web_push_vapid_private_key" \
"$web_push_vapid_subject"; do
[ -n "$value" ] || {
echo "All three production WEB_PUSH_VAPID_* values must be configured together." >&2
exit 1
}
done
fi
if { [ -n "$fcm_project_id" ] || [ -n "$fcm_service_account_json_base64" ]; } &&
{ [ -z "$fcm_project_id" ] || [ -z "$fcm_service_account_json_base64" ]; }; then
echo "Production FCM project ID and Base64 service account must be configured together." >&2
exit 1
fi
case "$compose_project_name" in
*[!a-zA-Z0-9_-]* | '')
echo "PRODUCTION_COMPOSE_PROJECT_NAME must contain only letters, numbers, underscores, or hyphens." >&2
@ -187,6 +224,15 @@ CODEX_SESSION_ID_VALUE=$codex_session_id \
GIT_SHA_VALUE=$git_sha \
GOOGLE_OAUTH_CLIENT_ID_VALUE=$google_oauth_client_id \
GOOGLE_OAUTH_CLIENT_SECRET_VALUE=$google_oauth_client_secret \
WEB_PUSH_VAPID_PUBLIC_KEY_VALUE=$web_push_vapid_public_key \
WEB_PUSH_VAPID_PRIVATE_KEY_VALUE=$web_push_vapid_private_key \
WEB_PUSH_VAPID_SUBJECT_VALUE=$web_push_vapid_subject \
FIREBASE_APPLICATION_ID_VALUE=$firebase_application_id \
FIREBASE_API_KEY_VALUE=$firebase_api_key \
FIREBASE_PROJECT_ID_VALUE=$firebase_project_id \
FIREBASE_SENDER_ID_VALUE=$firebase_sender_id \
FCM_PROJECT_ID_VALUE=$fcm_project_id \
FCM_SERVICE_ACCOUNT_JSON_BASE64_VALUE=$fcm_service_account_json_base64 \
ANDROID_APP_LINKS_PACKAGE_NAME_VALUE=$android_app_links_package_name \
ANDROID_APP_LINKS_FINGERPRINTS_VALUE=$android_app_links_fingerprints \
EDGE_COMPOSE_PROJECT_NAME_VALUE=$edge_compose_project_name \
@ -244,6 +290,16 @@ TEST_UPSTREAM_VALUE=$test_upstream \
replacement["SUPPORT_INBOX_ADDRESS"] = ENVIRON["SUPPORT_INBOX_ADDRESS_VALUE"]
replacement["GOOGLE_OAUTH_CLIENT_ID"] = ENVIRON["GOOGLE_OAUTH_CLIENT_ID_VALUE"]
replacement["GOOGLE_OAUTH_CLIENT_SECRET"] = ENVIRON["GOOGLE_OAUTH_CLIENT_SECRET_VALUE"]
replacement["WEB_PUSH_VAPID_PUBLIC_KEY"] = ENVIRON["WEB_PUSH_VAPID_PUBLIC_KEY_VALUE"]
replacement["WEB_PUSH_VAPID_PRIVATE_KEY"] = ENVIRON["WEB_PUSH_VAPID_PRIVATE_KEY_VALUE"]
replacement["WEB_PUSH_VAPID_SUBJECT"] = ENVIRON["WEB_PUSH_VAPID_SUBJECT_VALUE"]
replacement["WNH_FIREBASE_APPLICATION_ID"] = ENVIRON["FIREBASE_APPLICATION_ID_VALUE"]
replacement["WNH_FIREBASE_API_KEY"] = ENVIRON["FIREBASE_API_KEY_VALUE"]
replacement["WNH_FIREBASE_PROJECT_ID"] = ENVIRON["FIREBASE_PROJECT_ID_VALUE"]
replacement["WNH_FIREBASE_GCM_SENDER_ID"] = ENVIRON["FIREBASE_SENDER_ID_VALUE"]
replacement["FCM_PROJECT_ID"] = ENVIRON["FCM_PROJECT_ID_VALUE"]
replacement["FCM_SERVICE_ACCOUNT_FILE"] = ""
replacement["FCM_SERVICE_ACCOUNT_JSON_BASE64"] = ENVIRON["FCM_SERVICE_ACCOUNT_JSON_BASE64_VALUE"]
replacement["ANDROID_APP_LINKS_PACKAGE_NAME"] = ENVIRON["ANDROID_APP_LINKS_PACKAGE_NAME_VALUE"]
replacement["ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS"] = ENVIRON["ANDROID_APP_LINKS_FINGERPRINTS_VALUE"]
replacement["CODEX_SESSION_ID"] = ENVIRON["CODEX_SESSION_ID_VALUE"]
@ -267,6 +323,7 @@ trap - EXIT HUP INT TERM
unset postgres_password secret_key_base handover_secret release_cookie metrics_token
unset smtp_password
unset google_oauth_client_secret
unset web_push_vapid_private_key firebase_api_key fcm_service_account_json_base64
unset android_app_links_fingerprints
echo "Generated independent deployment secrets without printing them."

57
scripts/init-test-env.sh
View File

@ -53,6 +53,15 @@ mailpit_port=${TEST_MAILPIT_PORT:-8027}
codex_session_id=${TEST_CODEX_SESSION_ID:-}
google_oauth_client_id=${TEST_GOOGLE_OAUTH_CLIENT_ID:-}
google_oauth_client_secret=${TEST_GOOGLE_OAUTH_CLIENT_SECRET:-}
web_push_vapid_public_key=${TEST_WEB_PUSH_VAPID_PUBLIC_KEY:-}
web_push_vapid_private_key=${TEST_WEB_PUSH_VAPID_PRIVATE_KEY:-}
web_push_vapid_subject=${TEST_WEB_PUSH_VAPID_SUBJECT:-}
firebase_application_id=${TEST_WNH_FIREBASE_APPLICATION_ID:-}
firebase_api_key=${TEST_WNH_FIREBASE_API_KEY:-}
firebase_project_id=${TEST_WNH_FIREBASE_PROJECT_ID:-}
firebase_sender_id=${TEST_WNH_FIREBASE_GCM_SENDER_ID:-}
fcm_project_id=${TEST_FCM_PROJECT_ID:-}
fcm_service_account_json_base64=${TEST_FCM_SERVICE_ACCOUNT_JSON_BASE64:-}
android_app_links_package_name=${TEST_ANDROID_APP_LINKS_PACKAGE_NAME:-}
android_app_links_fingerprints=${TEST_ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS:-}
support_inbox_address=${TEST_SUPPORT_INBOX_ADDRESS:-}
@ -88,6 +97,34 @@ if [[ -n "$android_app_links_package_name" || -n "$android_app_links_fingerprint
}
fi
firebase_values="$firebase_application_id $firebase_api_key $firebase_project_id $firebase_sender_id"
if grep -q '[^[:space:]]' <<<"$firebase_values"; then
for value in "$firebase_application_id" "$firebase_api_key" \
"$firebase_project_id" "$firebase_sender_id"; do
[[ -n "$value" ]] || {
echo "All four test WNH_FIREBASE_* values must be configured together." >&2
exit 1
}
done
fi
vapid_values="$web_push_vapid_public_key $web_push_vapid_private_key $web_push_vapid_subject"
if grep -q '[^[:space:]]' <<<"$vapid_values"; then
for value in "$web_push_vapid_public_key" "$web_push_vapid_private_key" \
"$web_push_vapid_subject"; do
[[ -n "$value" ]] || {
echo "All three test WEB_PUSH_VAPID_* values must be configured together." >&2
exit 1
}
done
fi
if [[ (-n "$fcm_project_id" || -n "$fcm_service_account_json_base64") &&
(-z "$fcm_project_id" || -z "$fcm_service_account_json_base64") ]]; then
echo "Test FCM project ID and Base64 service account must be configured together." >&2
exit 1
fi
for pair in "HTTP:$http_port" "MAILPIT:$mailpit_port"; do
value=${pair#*:}
if [[ ! "$value" =~ ^[0-9]+$ ]] || ((value < 1 || value > 65535)); then
@ -132,6 +169,15 @@ RELEASE_COOKIE_VALUE=$release_cookie \
METRICS_TOKEN_VALUE=$metrics_token \
GOOGLE_OAUTH_CLIENT_ID_VALUE=$google_oauth_client_id \
GOOGLE_OAUTH_CLIENT_SECRET_VALUE=$google_oauth_client_secret \
WEB_PUSH_VAPID_PUBLIC_KEY_VALUE=$web_push_vapid_public_key \
WEB_PUSH_VAPID_PRIVATE_KEY_VALUE=$web_push_vapid_private_key \
WEB_PUSH_VAPID_SUBJECT_VALUE=$web_push_vapid_subject \
FIREBASE_APPLICATION_ID_VALUE=$firebase_application_id \
FIREBASE_API_KEY_VALUE=$firebase_api_key \
FIREBASE_PROJECT_ID_VALUE=$firebase_project_id \
FIREBASE_SENDER_ID_VALUE=$firebase_sender_id \
FCM_PROJECT_ID_VALUE=$fcm_project_id \
FCM_SERVICE_ACCOUNT_JSON_BASE64_VALUE=$fcm_service_account_json_base64 \
ANDROID_APP_LINKS_PACKAGE_NAME_VALUE=$android_app_links_package_name \
ANDROID_APP_LINKS_FINGERPRINTS_VALUE=$android_app_links_fingerprints \
SUPPORT_INBOX_ADDRESS_VALUE=$support_inbox_address \
@ -185,6 +231,16 @@ CODEX_SESSION_ID_VALUE=$codex_session_id \
replacement["SUPPORT_INBOX_ADDRESS"] = ENVIRON["SUPPORT_INBOX_ADDRESS_VALUE"]
replacement["GOOGLE_OAUTH_CLIENT_ID"] = ENVIRON["GOOGLE_OAUTH_CLIENT_ID_VALUE"]
replacement["GOOGLE_OAUTH_CLIENT_SECRET"] = ENVIRON["GOOGLE_OAUTH_CLIENT_SECRET_VALUE"]
replacement["WEB_PUSH_VAPID_PUBLIC_KEY"] = ENVIRON["WEB_PUSH_VAPID_PUBLIC_KEY_VALUE"]
replacement["WEB_PUSH_VAPID_PRIVATE_KEY"] = ENVIRON["WEB_PUSH_VAPID_PRIVATE_KEY_VALUE"]
replacement["WEB_PUSH_VAPID_SUBJECT"] = ENVIRON["WEB_PUSH_VAPID_SUBJECT_VALUE"]
replacement["WNH_FIREBASE_APPLICATION_ID"] = ENVIRON["FIREBASE_APPLICATION_ID_VALUE"]
replacement["WNH_FIREBASE_API_KEY"] = ENVIRON["FIREBASE_API_KEY_VALUE"]
replacement["WNH_FIREBASE_PROJECT_ID"] = ENVIRON["FIREBASE_PROJECT_ID_VALUE"]
replacement["WNH_FIREBASE_GCM_SENDER_ID"] = ENVIRON["FIREBASE_SENDER_ID_VALUE"]
replacement["FCM_PROJECT_ID"] = ENVIRON["FCM_PROJECT_ID_VALUE"]
replacement["FCM_SERVICE_ACCOUNT_FILE"] = ""
replacement["FCM_SERVICE_ACCOUNT_JSON_BASE64"] = ENVIRON["FCM_SERVICE_ACCOUNT_JSON_BASE64_VALUE"]
replacement["ANDROID_APP_LINKS_PACKAGE_NAME"] = ENVIRON["ANDROID_APP_LINKS_PACKAGE_NAME_VALUE"]
replacement["ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS"] = ENVIRON["ANDROID_APP_LINKS_FINGERPRINTS_VALUE"]
replacement["CODEX_SESSION_ID"] = ENVIRON["CODEX_SESSION_ID_VALUE"]
@ -202,6 +258,7 @@ trap - EXIT HUP INT TERM
unset postgres_password secret_key_base handover_secret release_cookie metrics_token
unset google_oauth_client_secret
unset web_push_vapid_private_key firebase_api_key fcm_service_account_json_base64
unset android_app_links_fingerprints
"$ROOT/scripts/compose.sh" "$target" config --quiet

2
scripts/load-cycle.sh
View File

@ -3,7 +3,7 @@ set -euo pipefail
umask 077
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
BASE_ENV=${WNH_LOAD_BASE_ENV_FILE:-"$ROOT/.env.load"}
BASE_ENV=${WNH_LOAD_BASE_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LABEL=${1:-"cycle-$(date -u +%Y%m%dT%H%M%SZ)"}
MODE=${2:-load}

4
scripts/load-resilience-run.sh
View File

@ -2,7 +2,7 @@
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/.env.load"}
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LABEL=${1:-"resilience-$(date -u +%Y%m%dT%H%M%SZ)"}
if [[ ! -f "$ENV_FILE" ]]; then
@ -21,7 +21,7 @@ for name in LOAD_PROJECT LOAD_HOST LOAD_WEB_REPLICAS LOAD_WORKER_REPLICAS \
LOAD_RESILIENCE_REQUEST_TIMEOUT_SECONDS TRAEFIK_RETRY_ATTEMPTS \
TRAEFIK_API_INSECURE HTTP_PORT POSTGRES_DB METRICS_TOKEN; do
if [[ -z "${!name:-}" ]]; then
echo "$name is missing from .env.load" >&2
echo "$name is missing from the generated load runtime environment" >&2
exit 1
fi
done

6
scripts/load-run.sh
View File

@ -2,7 +2,7 @@
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/.env.load"}
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LOAD_TOOLS_IMAGE=${WNH_LOAD_TOOLS_IMAGE:-who-need-help:load-tools}
K6_IMAGE="grafana/k6@sha256:65c920dc067d5e2e00befbf982af6ad6ad0117034e8b1c65817c7975c52d4669"
PYTHON_IMAGE="python:3.14.6-alpine3.23@sha256:b165067c5afc37fa5608a3c05609cc3d51aafd808a30fbfd822ee594fef55ad4"
@ -22,7 +22,7 @@ set +a
WEB_POOL_SIZE=${WEB_POOL_SIZE:-${POOL_SIZE:-}}
WORKER_POOL_SIZE=${WORKER_POOL_SIZE:-${POOL_SIZE:-}}
duration_source=".env.load"
duration_source="output/runtime/load.env"
if [[ -n "$duration_override" ]]; then
LOAD_DURATION=$duration_override
duration_source="LOAD_DURATION_OVERRIDE"
@ -35,7 +35,7 @@ for name in LOAD_PROJECT LOAD_HOST LOAD_WEB_REPLICAS LOAD_WORKER_REPLICAS \
LOAD_AUTH_THINK_SECONDS LOAD_FIXTURE_PASSWORD POSTGRES_DB HTTP_PORT \
WEB_POOL_SIZE WORKER_POOL_SIZE; do
if [[ -z "${!name:-}" ]]; then
echo "$name is missing from .env.load" >&2
echo "$name is missing from the generated load runtime environment" >&2
exit 1
fi
done

4
scripts/load-stack-stop.sh
View File

@ -2,7 +2,7 @@
set -eu
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/.env.load"}
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LOAD_TOOLS_IMAGE=${WNH_LOAD_TOOLS_IMAGE:-who-need-help:load-tools}
if [ ! -f "$ENV_FILE" ]; then
@ -15,7 +15,7 @@ set -a
. "$ENV_FILE"
set +a
: "${LOAD_PROJECT:?LOAD_PROJECT is missing from .env.load}"
: "${LOAD_PROJECT:?LOAD_PROJECT is missing from the generated load runtime environment}"
if [ "$LOAD_PROJECT" = who_need_help ]; then
echo "Refusing to stop the staging Compose project." >&2

10
scripts/load-stack-up.sh
View File

@ -2,7 +2,7 @@
set -eu
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/.env.load"}
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LOAD_TOOLS_IMAGE=${WNH_LOAD_TOOLS_IMAGE:-who-need-help:load-tools}
REPLICAS=${1:-}
@ -13,10 +13,10 @@ set -a
. "$ENV_FILE"
set +a
: "${LOAD_PROJECT:?LOAD_PROJECT is missing from .env.load}"
: "${LOAD_HOST:?LOAD_HOST is missing from .env.load}"
: "${LOAD_WEB_REPLICAS:?LOAD_WEB_REPLICAS is missing from .env.load}"
: "${LOAD_WORKER_REPLICAS:?LOAD_WORKER_REPLICAS is missing from .env.load}"
: "${LOAD_PROJECT:?LOAD_PROJECT is missing from the generated load runtime environment}"
: "${LOAD_HOST:?LOAD_HOST is missing from the generated load runtime environment}"
: "${LOAD_WEB_REPLICAS:?LOAD_WEB_REPLICAS is missing from the generated load runtime environment}"
: "${LOAD_WORKER_REPLICAS:?LOAD_WORKER_REPLICAS is missing from the generated load runtime environment}"
if [ "$LOAD_PROJECT" = who_need_help ]; then
echo "The load profile must not use the staging Compose project." >&2

4
scripts/oban-burst-run.sh
View File

@ -2,7 +2,7 @@
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE="$ROOT/.env.load"
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
if [[ $# -ne 3 ]]; then
echo "Usage: $0 LABEL JOB_COUNT TIMEOUT_SECONDS" >&2
@ -40,7 +40,7 @@ set +a
for name in LOAD_PROJECT LOAD_WORKER_REPLICAS POSTGRES_DB; do
if [[ -z "${!name:-}" ]]; then
echo "$name is missing from .env.load." >&2
echo "$name is missing from the generated load runtime environment." >&2
exit 1
fi
done

6
scripts/observability-run.sh
View File

@ -2,7 +2,7 @@
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE="$ROOT/.env.load"
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
LABEL=${1:-"observability-$(date -u +%Y%m%dT%H%M%SZ)"}
if [[ ! -f "$ENV_FILE" ]]; then
@ -21,7 +21,7 @@ for name in LOAD_PROJECT LOAD_WEB_REPLICAS LOAD_WORKER_REPLICAS POSTGRES_DB METR
OBSERVABILITY_EVALUATION_INTERVAL OBSERVABILITY_TIMEOUT_SECONDS \
OBSERVABILITY_GRAFANA_ADMIN_USER OBSERVABILITY_GRAFANA_ADMIN_PASSWORD; do
if [[ -z "${!name:-}" ]]; then
echo "$name is missing from .env.load" >&2
echo "$name is missing from the generated load runtime environment" >&2
exit 1
fi
done
@ -658,5 +658,5 @@ trap - EXIT HUP INT TERM
printf 'Observability evidence: %s\n' "$output_dir"
printf 'Prometheus: %s\nAlertmanager: %s\nGrafana: %s/d/wnh-overview/overview\n' \
"$prometheus_url" "$alertmanager_url" "$grafana_url"
printf 'Grafana user: %s; its random password remains only in ignored .env.load.\n' \
printf 'Grafana user: %s; its random password remains only in ignored output/runtime/load.env.\n' \
"$OBSERVABILITY_GRAFANA_ADMIN_USER"

4
scripts/observability-stop.sh
View File

@ -2,7 +2,7 @@
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE="$ROOT/.env.load"
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
if [[ ! -f "$ENV_FILE" ]]; then
echo "Missing $ENV_FILE; no observability project was selected." >&2
@ -14,7 +14,7 @@ set -a
. "$ENV_FILE"
set +a
: "${LOAD_PROJECT:?LOAD_PROJECT is missing from .env.load}"
: "${LOAD_PROJECT:?LOAD_PROJECT is missing from the generated load runtime environment}"
if [[ "$LOAD_PROJECT" == "who_need_help" ]]; then
echo "Refusing to stop services in the staging Compose project." >&2

53
scripts/prepare-production-release.sh Executable file
View File

@ -0,0 +1,53 @@
#!/usr/bin/env bash
set -euo pipefail
umask 077
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
if [[ -n "$(git -C "$ROOT" status --porcelain --untracked-files=no)" ]]; then
echo "Refusing to package a release from a dirty tracked checkout." >&2
exit 1
fi
commit=$(git -C "$ROOT" rev-parse --verify HEAD)
short_commit=${commit:0:12}
release_dir="$ROOT/output/releases/$commit"
bundle="$release_dir/who_need_help-$commit.bundle"
checksum="$bundle.sha256"
manifest="$release_dir/manifest.txt"
mkdir -p "$release_dir"
chmod 700 "$ROOT/output" "$ROOT/output/releases" "$release_dir"
if [[ -e "$bundle" || -e "$checksum" || -e "$manifest" ]]; then
echo "Release package already exists; verifying it instead of overwriting it."
else
git -C "$ROOT" bundle create "$bundle" HEAD
chmod 600 "$bundle"
hash=$(sha256sum "$bundle" | awk '{print $1}')
printf '%s %s\n' "$hash" "$(basename -- "$bundle")" >"$checksum"
{
printf 'commit=%s\n' "$commit"
printf 'short_commit=%s\n' "$short_commit"
printf 'created_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf 'bundle_sha256=%s\n' "$hash"
} >"$manifest"
chmod 600 "$checksum" "$manifest"
fi
(
cd "$release_dir"
sha256sum --check "$(basename -- "$checksum")" >/dev/null
)
git -C "$ROOT" bundle verify "$bundle" >/dev/null
bundle_head=$(git -C "$ROOT" bundle list-heads "$bundle" | awk '$2 == "HEAD" {print $1}')
if [[ "$bundle_head" != "$commit" ]]; then
echo "Release bundle HEAD does not match the current commit." >&2
exit 1
fi
printf 'Release commit: %s\n' "$commit"
printf 'Bundle: %s\n' "$bundle"
printf 'Checksum: %s\n' "$checksum"
printf 'Manifest: %s\n' "$manifest"

276
scripts/production-release-remote.sh Executable file
View File

@ -0,0 +1,276 @@
#!/usr/bin/env bash
set -euo pipefail
umask 077
action=${1:-}
root=${2:-/srv/who_need_help-production}
expected_domain=${3:-whoneedhelp.com}
bundle=${4:-}
target_commit=${5:-}
backup=${6:-}
usage() {
echo "Usage: $0 plan /srv/who_need_help-production whoneedhelp.com" >&2
echo " $0 apply /srv/who_need_help-production whoneedhelp.com BUNDLE COMMIT BACKUP" >&2
}
case "$action" in
plan | apply) ;;
*) usage; exit 2 ;;
esac
root=$(realpath --canonicalize-existing "$root")
if [[ "$root" != "/srv/who_need_help-production" ]]; then
echo "Refusing a production release outside /srv/who_need_help-production." >&2
exit 2
fi
env_file="$root/.env"
if [[ ! -f "$env_file" || "$(stat -c '%a' "$env_file")" != 600 ]]; then
echo "Production .env is missing or does not have mode 0600." >&2
exit 2
fi
read_value() {
local key=$1
awk -v key="$key" '
index($0, key "=") == 1 {
print substr($0, length(key) + 2)
found = 1
exit
}
END { if (!found) exit 1 }
' "$env_file"
}
deployment_environment=$(read_value DEPLOYMENT_ENV)
compose_project=$(read_value COMPOSE_PROJECT_NAME)
database_mode=$(read_value DATABASE_MODE)
app_topology=$(read_value APP_TOPOLOGY)
phx_host=$(read_value PHX_HOST)
public_origin=$(read_value WNH_BASE_URL)
branch=$(git -C "$root" symbolic-ref --quiet --short HEAD || true)
current_commit=$(git -C "$root" rev-parse --verify HEAD)
[[ "$deployment_environment" == "production" ]] || {
echo "DEPLOYMENT_ENV is not production." >&2
exit 2
}
[[ "$compose_project" == "who_need_help_production" ]] || {
echo "Unexpected production Compose project." >&2
exit 2
}
[[ "$database_mode" == "external" ]] || {
echo "The verified single-server production workflow expects DATABASE_MODE=external." >&2
exit 2
}
[[ "$phx_host" == "$expected_domain" &&
"$public_origin" == "https://$expected_domain" ]] || {
echo "Production origin does not match the expected domain." >&2
exit 2
}
[[ "$branch" == "main" || -z "$branch" ]] || {
echo "Production checkout must be on main or detached at the deployed commit." >&2
exit 2
}
[[ -z "$(git -C "$root" status --porcelain --untracked-files=no)" ]] || {
echo "Production checkout has tracked modifications." >&2
exit 2
}
"$root/scripts/validate-production-env.sh" "$env_file" "$expected_domain" >/dev/null
"$root/scripts/compose.sh" "$env_file" config --quiet
case "$app_topology" in
compact) expected_services=(app) ;;
split) expected_services=(web worker) ;;
*) echo "Unexpected APP_TOPOLOGY." >&2; exit 2 ;;
esac
for service in "${expected_services[@]}"; do
mapfile -t containers < <("$root/scripts/compose.sh" "$env_file" ps -q "$service")
[[ ${#containers[@]} -gt 0 ]] || {
echo "Production service is not running: $service" >&2
exit 2
}
for container in "${containers[@]}"; do
state=$(docker inspect --format '{{.State.Status}}' "$container")
health=$(docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$container")
[[ "$state" == "running" && "$health" == "healthy" ]] || {
echo "Production container is not healthy: $service" >&2
exit 2
}
done
done
curl --fail --silent --show-error --max-time 15 \
"https://$expected_domain/healthz/ready" >/dev/null
printf 'Production checkout: %s\n' "$root"
printf 'Current commit: %s\n' "$current_commit"
printf 'Branch: %s\n' "${branch:-detached}"
printf 'Compose project: %s\n' "$compose_project"
printf 'Topology: %s\n' "$app_topology"
printf 'Database mode: %s\n' "$database_mode"
printf 'Public readiness: passed\n'
df -h "$root" /var/lib/docker 2>/dev/null || df -h "$root"
if [[ "$action" == "plan" ]]; then
echo "Read-only production release scope check passed."
exit 0
fi
if [[ -z "$bundle" || -z "$target_commit" || -z "$backup" ]]; then
usage
exit 2
fi
expected_confirmation="$expected_domain:$target_commit"
if [[ "${WNH_PRODUCTION_RELEASE_CONFIRM:-}" != "$expected_confirmation" ]]; then
echo "Set WNH_PRODUCTION_RELEASE_CONFIRM=$expected_confirmation for the approved release." >&2
exit 2
fi
for required_file in "$bundle" "$bundle.sha256" "$backup" "$backup.sha256" "$backup.metadata"; do
[[ -f "$required_file" ]] || {
echo "Required release evidence is missing: $required_file" >&2
exit 2
}
done
(
cd "$(dirname -- "$bundle")"
sha256sum --check "$(basename -- "$bundle.sha256")" >/dev/null
)
(
cd "$(dirname -- "$backup")"
sha256sum --check "$(basename -- "$backup.sha256")" >/dev/null
)
pg_restore --list "$backup" >/dev/null
git -C "$root" bundle verify "$bundle" >/dev/null
bundle_head=$(git -C "$root" bundle list-heads "$bundle" | awk '$2 == "HEAD" {print $1}')
[[ "$bundle_head" == "$target_commit" ]] || {
echo "Bundle HEAD does not match the approved target commit." >&2
exit 2
}
release_id="$(date -u +%Y%m%dT%H%M%SZ)-${target_commit:0:12}"
release_dir="$root/output/releases/$release_id"
mkdir -p "$release_dir"
chmod 700 "$root/output" "$root/output/releases" "$release_dir"
rollback_manifest="$release_dir/rollback-manifest.txt"
{
printf 'previous_commit=%s\n' "$current_commit"
printf 'target_commit=%s\n' "$target_commit"
printf 'APP_IMAGE=%s\n' "$(read_value APP_IMAGE)"
printf 'SOCKET_PROXY_IMAGE=%s\n' "$(read_value SOCKET_PROXY_IMAGE)"
printf 'POSTGIS_IMAGE=%s\n' "$(read_value POSTGIS_IMAGE)"
printf 'CADDY_IMAGE=%s\n' "$(read_value CADDY_IMAGE)"
printf 'database_backup=%s\n' "$backup"
printf 'started_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf 'status=started\n'
} >"$rollback_manifest"
chmod 600 "$rollback_manifest"
revision_changed=false
restore_image_revision() {
local temporary
temporary=$(mktemp "$root/.env.release-rollback.XXXXXX")
chmod 600 "$temporary"
APP_IMAGE_VALUE=$(awk -F= '$1 == "APP_IMAGE" {print substr($0,index($0,"=")+1)}' "$rollback_manifest") \
SOCKET_PROXY_IMAGE_VALUE=$(awk -F= '$1 == "SOCKET_PROXY_IMAGE" {print substr($0,index($0,"=")+1)}' "$rollback_manifest") \
POSTGIS_IMAGE_VALUE=$(awk -F= '$1 == "POSTGIS_IMAGE" {print substr($0,index($0,"=")+1)}' "$rollback_manifest") \
CADDY_IMAGE_VALUE=$(awk -F= '$1 == "CADDY_IMAGE" {print substr($0,index($0,"=")+1)}' "$rollback_manifest") \
awk '
BEGIN {
replacement["APP_IMAGE"] = ENVIRON["APP_IMAGE_VALUE"]
replacement["SOCKET_PROXY_IMAGE"] = ENVIRON["SOCKET_PROXY_IMAGE_VALUE"]
replacement["POSTGIS_IMAGE"] = ENVIRON["POSTGIS_IMAGE_VALUE"]
replacement["CADDY_IMAGE"] = ENVIRON["CADDY_IMAGE_VALUE"]
}
{
separator = index($0, "=")
key = separator > 1 ? substr($0, 1, separator - 1) : ""
print (key in replacement) ? key "=" replacement[key] : $0
}
' "$env_file" >"$temporary"
mv "$temporary" "$env_file"
chmod 600 "$env_file"
}
rollback_runtime() {
local status=$?
trap - EXIT HUP INT TERM
if [[ "$status" -ne 0 && "$revision_changed" == true ]]; then
echo "Release failed after selecting new images; restoring the previous immutable image tags." >&2
restore_image_revision
"$root/scripts/compose.sh" "$env_file" \
up -d --no-build --wait --remove-orphans || true
edge_project=$(read_value EDGE_COMPOSE_PROJECT_NAME)
docker compose \
--project-name "$edge_project" \
--project-directory "$root" \
--env-file "$env_file" \
--file "$root/compose.edge.yaml" \
up -d --no-build --wait --remove-orphans || true
curl --fail --silent --show-error --max-time 15 \
"https://$expected_domain/healthz/ready" >/dev/null || true
{
printf 'status=runtime-rolled-back\n'
printf 'failed_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf 'database_note=migrations_may_have_applied_and_were_not_reversed\n'
} >>"$rollback_manifest"
echo "The Git checkout and any applied migrations were intentionally not reversed automatically." >&2
fi
exit "$status"
}
trap rollback_runtime EXIT HUP INT TERM
release_ref="refs/wnh/releases/$target_commit"
git -C "$root" fetch "$bundle" "HEAD:$release_ref"
[[ "$(git -C "$root" rev-parse "$release_ref^{commit}")" == "$target_commit" ]] || {
echo "Fetched release ref does not match the approved commit." >&2
exit 1
}
git -C "$root" merge-base --is-ancestor "$current_commit" "$target_commit" || {
echo "Production updates must be a fast-forward from the deployed commit." >&2
exit 1
}
if [[ "$branch" == "main" ]]; then
git -C "$root" merge --ff-only "$release_ref"
else
git -C "$root" checkout --detach "$release_ref"
fi
"$root/scripts/set-deployment-revision.sh" "$env_file"
revision_changed=true
"$root/scripts/validate-production-env.sh" "$env_file" "$expected_domain"
"$root/scripts/check-environment-readiness.sh" "$env_file" --require-release
"$root/scripts/deploy-up.sh" "$env_file"
"$root/scripts/edge-up.sh" "$env_file"
curl --fail --silent --show-error --max-time 30 \
"https://$expected_domain/healthz/ready" >/dev/null
curl --fail --silent --show-error --max-time 30 \
"https://$expected_domain/.well-known/assetlinks.json" >/dev/null
{
printf 'status=success\n'
printf 'completed_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} >>"$rollback_manifest"
revision_changed=false
trap - EXIT HUP INT TERM
printf 'Production release completed: %s\n' "$target_commit"
printf 'Rollback/runtime evidence: %s\n' "$rollback_manifest"
printf 'Database backup: %s\n' "$backup"

124
scripts/production-release.sh Executable file
View File

@ -0,0 +1,124 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
action=${1:-plan}
ssh_target=${2:-whoneedhelp}
remote_root=${WNH_PRODUCTION_REMOTE_ROOT:-/srv/who_need_help-production}
expected_domain=${WNH_PRODUCTION_DOMAIN:-whoneedhelp.com}
case "$action" in
plan | apply) ;;
*)
echo "Usage: $0 [plan|apply] [SSH_TARGET]" >&2
exit 2
;;
esac
for command in git pg_restore scp sha256sum ssh; do
command -v "$command" >/dev/null 2>&1 || {
echo "Required command is unavailable: $command" >&2
exit 2
}
done
local_commit=$(git -C "$ROOT" rev-parse --verify HEAD)
remote_commit=$(
ssh -o BatchMode=yes "$ssh_target" \
"git -C '$remote_root' rev-parse --verify HEAD"
)
printf 'Local candidate commit: %s\n' "$local_commit"
printf 'Current production commit: %s\n' "$remote_commit"
if git -C "$ROOT" cat-file -e "$remote_commit^{commit}" 2>/dev/null; then
git -C "$ROOT" merge-base --is-ancestor "$remote_commit" "$local_commit" || {
echo "The local candidate is not a fast-forward from the production commit." >&2
exit 2
}
printf 'Pending commits: %s\n' \
"$(git -C "$ROOT" rev-list --count "$remote_commit..$local_commit")"
else
echo "The production commit is not present in the local object database." >&2
exit 2
fi
plan_failed=0
if ! ssh -o BatchMode=yes "$ssh_target" \
"bash -s -- plan '$remote_root' '$expected_domain'" \
<"$ROOT/scripts/production-release-remote.sh"; then
plan_failed=1
fi
if ! ssh -o BatchMode=yes "$ssh_target" \
"bash -s -- '$remote_root/.env' --check-only production" \
<"$ROOT/scripts/backup-external-postgres.sh"; then
plan_failed=1
fi
if ! ssh -o BatchMode=yes "$ssh_target" \
"bash -s -- '$remote_root/.env' --require-release" \
<"$ROOT/scripts/check-environment-readiness.sh"; then
plan_failed=1
fi
if [[ "$plan_failed" -ne 0 ]]; then
echo "Production release plan has blocking checks; no remote state was changed." >&2
exit 1
fi
if [[ "$action" == "plan" ]]; then
echo "Production release plan passed; no remote state was changed."
exit 0
fi
if [[ -n "$(git -C "$ROOT" status --porcelain --untracked-files=no)" ]]; then
echo "Refusing to release a dirty tracked checkout." >&2
exit 2
fi
confirmation="$expected_domain:$local_commit"
if [[ "${WNH_PRODUCTION_RELEASE_CONFIRM:-}" != "$confirmation" ]]; then
echo "Release execution requires explicit approval in this exact process:" >&2
echo "WNH_PRODUCTION_RELEASE_CONFIRM=$confirmation $0 apply $ssh_target" >&2
exit 2
fi
"$ROOT/scripts/prepare-production-release.sh"
release_dir="$ROOT/output/releases/$local_commit"
bundle="$release_dir/who_need_help-$local_commit.bundle"
remote_release_dir="$remote_root/output/releases/incoming"
remote_bundle="$remote_release_dir/$(basename -- "$bundle")"
timestamp=$(date -u +%Y%m%dT%H%M%SZ)
remote_backup="$remote_root/output/backups/production/pre-$timestamp-${local_commit:0:12}.dump"
ssh -o BatchMode=yes "$ssh_target" \
"install -d -m 700 '$remote_release_dir'"
scp -p "$bundle" "$bundle.sha256" "$ssh_target:$remote_release_dir/"
ssh -o BatchMode=yes "$ssh_target" \
"bash -s -- '$remote_root/.env' '$remote_backup' production" \
<"$ROOT/scripts/backup-external-postgres.sh"
local_backup_dir="$ROOT/output/production-backups/$timestamp-${local_commit:0:12}"
mkdir -p "$local_backup_dir"
chmod 700 "$ROOT/output" "$ROOT/output/production-backups" "$local_backup_dir"
scp -p \
"$ssh_target:$remote_backup" \
"$ssh_target:$remote_backup.sha256" \
"$ssh_target:$remote_backup.metadata" \
"$local_backup_dir/"
(
cd "$local_backup_dir"
sha256sum --check "$(basename -- "$remote_backup.sha256")" >/dev/null
)
pg_restore --list "$local_backup_dir/$(basename -- "$remote_backup")" >/dev/null
echo "Copied and independently verified the pre-release backup outside the production server."
quoted_confirmation=$(printf '%q' "$confirmation")
ssh -o BatchMode=yes "$ssh_target" \
"WNH_PRODUCTION_RELEASE_CONFIRM=$quoted_confirmation bash -s -- apply '$remote_root' '$expected_domain' '$remote_bundle' '$local_commit' '$remote_backup'" \
<"$ROOT/scripts/production-release-remote.sh"
echo "Production release and public health verification completed."

22
scripts/quality.sh
View File

@ -116,6 +116,10 @@ WNH_LOAD_ENV_FILE="$legacy_load_env" \
test "$(sha256sum "$legacy_load_env" | awk '{print $1}')" = "$legacy_load_hash"
echo "Checking independent test and production environment initialization"
quality_fcm_base64=$(
printf '%s' '{"type":"service_account","project_id":"quality-production"}' |
base64 -w 0
)
test_env="$scan_dir/test.env"
if ./scripts/init-test-env.sh test.help.test \
"$scan_dir/test.missing-codex.env" >/dev/null 2>&1; then
@ -186,11 +190,22 @@ PRODUCTION_EMAIL_FROM_ADDRESS=contact@help.test \
PRODUCTION_CODEX_SESSION_ID=00000000-0000-0000-0000-000000000001 \
PRODUCTION_GOOGLE_OAUTH_CLIENT_ID=quality-production-client \
PRODUCTION_GOOGLE_OAUTH_CLIENT_SECRET=quality-production-secret \
PRODUCTION_WEB_PUSH_VAPID_PUBLIC_KEY=quality-vapid-public \
PRODUCTION_WEB_PUSH_VAPID_PRIVATE_KEY=quality-vapid-private \
PRODUCTION_WEB_PUSH_VAPID_SUBJECT=mailto:contact@help.test \
PRODUCTION_WNH_FIREBASE_APPLICATION_ID=1:123456789:android:quality \
PRODUCTION_WNH_FIREBASE_API_KEY=quality-public-client-key \
PRODUCTION_WNH_FIREBASE_PROJECT_ID=quality-production \
PRODUCTION_WNH_FIREBASE_GCM_SENDER_ID=123456789 \
PRODUCTION_FCM_PROJECT_ID=quality-production \
PRODUCTION_FCM_SERVICE_ACCOUNT_JSON_BASE64="$quality_fcm_base64" \
PRODUCTION_ANDROID_APP_LINKS_PACKAGE_NAME=org.whoneedhelp.mobile \
PRODUCTION_ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS=A5:74:2B:AE:70:C6:D0:34:E3:75:44:B6:2E:37:A3:75:C0:E0:05:64:74:50:F4:0F:29:B2:A9:84:F9:FD:B8:FB \
PRODUCTION_SUPPORT_INBOX_ADDRESS=support@help.test \
./scripts/init-production-env.sh help.test "$production_env" >/dev/null
test "$(stat -c '%a' "$production_env")" = 600
./scripts/validate-production-env.sh "$production_env" help.test >/dev/null
./scripts/check-environment-readiness.sh "$production_env" --require-release >/dev/null
grep -Fx 'COMPOSE_PROJECT_NAME=who_need_help_production' "$production_env" >/dev/null
grep -E '^APP_IMAGE=who-need-help:production-[0-9a-f]{12}$' "$production_env" >/dev/null
grep -Fx 'EDGE_COMPOSE_PROJECT_NAME=who_need_help_edge' "$production_env" >/dev/null
@ -333,6 +348,13 @@ if ./scripts/init-production-env.sh help.test "$production_env" >/dev/null 2>&1;
echo "Production environment initializer overwrote an existing file." >&2
exit 1
fi
if PRODUCTION_CODEX_SESSION_ID=00000000-0000-0000-0000-000000000001 \
PRODUCTION_WNH_FIREBASE_PROJECT_ID=partial-firebase \
./scripts/init-production-env.sh \
help.test "$scan_dir/.env.production.partial-firebase" >/dev/null 2>&1; then
echo "Production initializer accepted partial Firebase Android configuration." >&2
exit 1
fi
incomplete_production_env="$scan_dir/.env.production.incomplete"
PRODUCTION_CODEX_SESSION_ID=00000000-0000-0000-0000-000000000001 \
./scripts/init-production-env.sh help.test "$incomplete_production_env" >/dev/null

2
scripts/rotate-load-metrics-token.sh
View File

@ -2,7 +2,7 @@
set -eu
ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
ENV_FILE="$ROOT/.env.load"
ENV_FILE=${WNH_LOAD_ENV_FILE:-"$ROOT/output/runtime/load.env"}
if [ ! -f "$ENV_FILE" ]; then
echo "Missing $ENV_FILE. Run scripts/ensure-local-load-env.sh first." >&2

6
scripts/upgrade-rehearsal-compose.sh
View File

@ -26,7 +26,7 @@ fi
"$ROOT/scripts/ensure-local-e2e-env.sh" >/dev/null
PUBLIC_ENV_FILE="$ROOT/.env"
ENV_FILE="$ROOT/.env.e2e"
ENV_FILE=${WNH_E2E_ENV_FILE:-"$ROOT/output/runtime/e2e.env"}
if [ ! -f "$PUBLIC_ENV_FILE" ]; then
echo "Missing $PUBLIC_ENV_FILE." >&2
@ -65,8 +65,8 @@ set -a
. "$ENV_FILE"
set +a
: "${POSTGRES_USER:?POSTGRES_USER is missing from .env.e2e}"
: "${DATABASE_URL:?DATABASE_URL is missing from .env.e2e}"
: "${POSTGRES_USER:?POSTGRES_USER is missing from the generated E2E runtime environment}"
: "${DATABASE_URL:?DATABASE_URL is missing from the generated E2E runtime environment}"
dump_dir=$(CDPATH='' cd -- "$(dirname -- "$dump")" && pwd)
dump_name=$(basename -- "$dump")

81
scripts/validate-production-env.sh
View File

@ -110,6 +110,16 @@ email_from_address=$(require_value EMAIL_FROM_ADDRESS)
support_inbox_address=$(optional_value SUPPORT_INBOX_ADDRESS)
google_oauth_client_id=$(optional_value GOOGLE_OAUTH_CLIENT_ID)
google_oauth_client_secret=$(optional_value GOOGLE_OAUTH_CLIENT_SECRET)
web_push_vapid_public_key=$(optional_value WEB_PUSH_VAPID_PUBLIC_KEY)
web_push_vapid_private_key=$(optional_value WEB_PUSH_VAPID_PRIVATE_KEY)
web_push_vapid_subject=$(optional_value WEB_PUSH_VAPID_SUBJECT)
firebase_application_id=$(optional_value WNH_FIREBASE_APPLICATION_ID)
firebase_api_key=$(optional_value WNH_FIREBASE_API_KEY)
firebase_project_id=$(optional_value WNH_FIREBASE_PROJECT_ID)
firebase_sender_id=$(optional_value WNH_FIREBASE_GCM_SENDER_ID)
fcm_project_id=$(optional_value FCM_PROJECT_ID)
fcm_service_account_file=$(optional_value FCM_SERVICE_ACCOUNT_FILE)
fcm_service_account_json_base64=$(optional_value FCM_SERVICE_ACCOUNT_JSON_BASE64)
android_app_links_package_name=$(optional_value ANDROID_APP_LINKS_PACKAGE_NAME)
android_app_links_fingerprints=$(optional_value ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS)
codex_session_id=$(require_value CODEX_SESSION_ID)
@ -318,6 +328,77 @@ if [[ -n "$google_oauth_client_id" || -n "$google_oauth_client_secret" ]]; then
reject_marker GOOGLE_OAUTH_CLIENT_SECRET "$google_oauth_client_secret"
fi
firebase_values=(
"$firebase_application_id"
"$firebase_api_key"
"$firebase_project_id"
"$firebase_sender_id"
)
firebase_nonempty=0
for candidate in "${firebase_values[@]}"; do
[[ -z "$candidate" ]] || firebase_nonempty=$((firebase_nonempty + 1))
done
if ((firebase_nonempty != 0 && firebase_nonempty != ${#firebase_values[@]})); then
echo "All four WNH_FIREBASE_* Android client values must be configured together." >&2
exit 1
fi
vapid_values=(
"$web_push_vapid_public_key"
"$web_push_vapid_private_key"
"$web_push_vapid_subject"
)
vapid_nonempty=0
for candidate in "${vapid_values[@]}"; do
[[ -z "$candidate" ]] || vapid_nonempty=$((vapid_nonempty + 1))
done
if ((vapid_nonempty != 0 && vapid_nonempty != ${#vapid_values[@]})); then
echo "All three WEB_PUSH_VAPID_* values must be configured together." >&2
exit 1
fi
if ((vapid_nonempty == ${#vapid_values[@]})) &&
[[ ! "$web_push_vapid_subject" =~ ^(mailto:|https://) ]]; then
echo "WEB_PUSH_VAPID_SUBJECT must start with mailto: or https://." >&2
exit 1
fi
if [[ -n "$fcm_service_account_file" && -n "$fcm_service_account_json_base64" ]]; then
echo "Set only one FCM service-account credential source." >&2
exit 1
fi
if [[ -n "$fcm_project_id" || -n "$fcm_service_account_file" ||
-n "$fcm_service_account_json_base64" ]]; then
[[ -n "$fcm_project_id" ]] || {
echo "FCM_PROJECT_ID is required with FCM credentials." >&2
exit 1
}
[[ -n "$fcm_service_account_file" || -n "$fcm_service_account_json_base64" ]] || {
echo "One FCM service-account credential source is required with FCM_PROJECT_ID." >&2
exit 1
}
if [[ -n "$fcm_service_account_file" ]]; then
[[ "$fcm_service_account_file" == /* && -r "$fcm_service_account_file" ]] || {
echo "FCM_SERVICE_ACCOUNT_FILE must be an absolute readable file." >&2
exit 1
}
else
for command in base64 jq; do
command -v "$command" >/dev/null 2>&1 || {
echo "Required command is unavailable for FCM validation: $command" >&2
exit 1
}
done
printf '%s' "$fcm_service_account_json_base64" |
base64 --decode 2>/dev/null |
jq -e '.type == "service_account" and (.project_id | type == "string")' \
>/dev/null 2>&1 || {
echo "FCM_SERVICE_ACCOUNT_JSON_BASE64 is not a service-account JSON document." >&2
exit 1
}
fi
fi
if [[ -n "$android_app_links_package_name" || -n "$android_app_links_fingerprints" ]]; then
[[ -n "$android_app_links_package_name" && -n "$android_app_links_fingerprints" ]] || {
echo "Android App Links package and fingerprints must either both be set or both be empty." >&2

52
scripts/validate-test-env.sh
View File

@ -124,6 +124,58 @@ if [[ -n "$google_id" || -n "$google_secret" ]]; then
}
fi
firebase_values=(
"$(read_value WNH_FIREBASE_APPLICATION_ID 2>/dev/null || true)"
"$(read_value WNH_FIREBASE_API_KEY 2>/dev/null || true)"
"$(read_value WNH_FIREBASE_PROJECT_ID 2>/dev/null || true)"
"$(read_value WNH_FIREBASE_GCM_SENDER_ID 2>/dev/null || true)"
)
firebase_nonempty=0
for candidate in "${firebase_values[@]}"; do
[[ -z "$candidate" ]] || firebase_nonempty=$((firebase_nonempty + 1))
done
if ((firebase_nonempty != 0 && firebase_nonempty != ${#firebase_values[@]})); then
echo "All four test WNH_FIREBASE_* Android client values must be configured together." >&2
exit 1
fi
vapid_values=(
"$(read_value WEB_PUSH_VAPID_PUBLIC_KEY 2>/dev/null || true)"
"$(read_value WEB_PUSH_VAPID_PRIVATE_KEY 2>/dev/null || true)"
"$(read_value WEB_PUSH_VAPID_SUBJECT 2>/dev/null || true)"
)
vapid_nonempty=0
for candidate in "${vapid_values[@]}"; do
[[ -z "$candidate" ]] || vapid_nonempty=$((vapid_nonempty + 1))
done
if ((vapid_nonempty != 0 && vapid_nonempty != ${#vapid_values[@]})); then
echo "All three test WEB_PUSH_VAPID_* values must be configured together." >&2
exit 1
fi
if ((vapid_nonempty == ${#vapid_values[@]})) &&
[[ ! "${vapid_values[2]}" =~ ^(mailto:|https://) ]]; then
echo "WEB_PUSH_VAPID_SUBJECT must start with mailto: or https://." >&2
exit 1
fi
fcm_project_id=$(read_value FCM_PROJECT_ID 2>/dev/null || true)
fcm_service_account_file=$(read_value FCM_SERVICE_ACCOUNT_FILE 2>/dev/null || true)
fcm_service_account_json_base64=$(
read_value FCM_SERVICE_ACCOUNT_JSON_BASE64 2>/dev/null || true
)
if [[ -n "$fcm_service_account_file" && -n "$fcm_service_account_json_base64" ]]; then
echo "Set only one test FCM service-account credential source." >&2
exit 1
fi
if [[ -n "$fcm_project_id" || -n "$fcm_service_account_file" ||
-n "$fcm_service_account_json_base64" ]]; then
[[ -n "$fcm_project_id" &&
(-n "$fcm_service_account_file" || -n "$fcm_service_account_json_base64") ]] || {
echo "Test FCM project ID and exactly one credential source are required together." >&2
exit 1
}
fi
android_package=$(read_value ANDROID_APP_LINKS_PACKAGE_NAME 2>/dev/null || true)
android_fingerprints=$(read_value ANDROID_APP_LINKS_SHA256_CERT_FINGERPRINTS 2>/dev/null || true)
if [[ -n "$android_package" || -n "$android_fingerprints" ]]; then

13
test/who_need_help/mutual_aid_flow_test.exs
View File

@ -138,13 +138,16 @@ defmodule WhoNeedHelp.MutualAidFlowTest do
assert {:ok, replacement_assignment} = Help.accept_request(replacement_scope, request.id)
assert replacement_assignment.id != assignment.id
assert [old_assignment, active_assignment] =
Assignment
|> where([current], current.request_id == ^request.id)
|> order_by([current], asc: current.inserted_at)
|> Repo.all()
assert Repo.aggregate(
from(current in Assignment, where: current.request_id == ^request.id),
:count
) == 2
old_assignment = Repo.get!(Assignment, assignment.id)
active_assignment = Repo.get!(Assignment, replacement_assignment.id)
refute old_assignment.active
assert old_assignment.status == :cancelled
assert active_assignment.active
assert active_assignment.helper_id == replacement.id
end