simpletest/who_need_help
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test: exercise authenticated load and database writes

Browse Source
This commit is contained in:
SimpleTest 2026-07-19 05:59:26 +03:00
parent a4414f09e4
commit 909e16c969
12 changed files with 1081 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.env.load.example
View File

@ -51,3 +51,7 @@ LOAD_DURATION=30s
LOAD_WS_HOLD_MS=5000
LOAD_WS_CONNECT_TIMEOUT_MS=3000
LOAD_HTTP_THINK_SECONDS=0.1
LOAD_AUTH_VUS=8
LOAD_AUTH_WS_TIMEOUT_MS=5000
LOAD_AUTH_THINK_SECONDS=0.1
LOAD_FIXTURE_PASSWORD=GENERATE_LOAD_FIXTURE_PASSWORD

7
Dockerfile
View File

@ -165,6 +165,11 @@ RUN mix dialyzer --plt
CMD ["mix", "dialyzer"]
FROM builder AS load_tools
CMD ["mix", "wnh.load_fixtures"]
# Keep the production release as the default build result while exposing the
# dedicated `test` and `quality` targets to local verification scripts.
# dedicated `test`, `quality`, and isolated `load_tools` targets to local
# verification scripts.
FROM final AS release

5
compose.load.yaml
View File

@ -14,6 +14,11 @@ services:
labels:
- traefik.http.routers.${TRAEFIK_APP_NAME}.middlewares=${TRAEFIK_APP_NAME}-forwarded
- traefik.http.middlewares.${TRAEFIK_APP_NAME}-forwarded.headers.customrequestheaders.X-Forwarded-Proto=https
- traefik.http.routers.${TRAEFIK_APP_NAME}-tls.rule=${TRAEFIK_ROUTER_RULE}
- traefik.http.routers.${TRAEFIK_APP_NAME}-tls.entrypoints=websecure
- traefik.http.routers.${TRAEFIK_APP_NAME}-tls.service=${TRAEFIK_APP_NAME}
- traefik.http.routers.${TRAEFIK_APP_NAME}-tls.middlewares=${TRAEFIK_APP_NAME}-forwarded
- traefik.http.routers.${TRAEFIK_APP_NAME}-tls.tls=true
worker:
image: who-need-help:load

1
compose.yaml
View File

@ -37,6 +37,7 @@ services:
- --providers.docker.exposedbydefault=false
- --providers.docker.constraints=Label(`com.docker.compose.project`,`${TRAEFIK_PROJECT_CONSTRAINT:-who_need_help}`)
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --entrypoints.web.forwardedheaders.trustedips=${TRAEFIK_TRUSTED_IPS:-127.0.0.1/32}
ports:
- "${HTTP_PORT:-4010}:80"

13
docs/local-hardening-plan.md
View File

@ -10,13 +10,13 @@ item below unless the evidence column explicitly describes a local mock.
| Browser E2E | Manual headed-Chrome scenarios exist; no committed browser suite | A fresh uniquely named Compose project runs two-user urgent help, Activity, moderation, privacy, and error paths; traces are retained on failure; its exact volume is removed |
| Android UI | Two JVM unit-test files; no `androidTest` source set | Emulator instrumentation covers deep links, permissions, foreground tracking, notification Stop, lifecycle, and network failure |
| CI and quality | No tracked CI workflow or static/security analysis dependencies | The same containerized gates pass locally and are represented in a validated CI workflow |
| Localization and accessibility | Completed locally: product copy and custom validation messages are extracted; EN/UK/RU catalogs and localized category descriptions/structured values are implemented | 508 default and 40 error messages are current; RU/UK have no empty/fuzzy entries; 153 backend tests and all 8 browser specs pass, including locale persistence, keyboard, axe, themes, responsive widths, and reconnect |
| Localization and accessibility | Completed locally: product copy and custom validation messages are extracted; EN/UK/RU catalogs and localized category descriptions/structured values are implemented | 508 default and 40 error messages are current; RU/UK have no empty/fuzzy entries; 161 backend tests and all 8 browser specs pass, including locale persistence, keyboard, axe, themes, responsive widths, and reconnect |
| Database scale | Core discovery/chat/moderation lists call unbounded `Repo.all()` | Cursor-bounded queries pass behavior tests and measured `EXPLAIN ANALYZE` checks on an isolated generated dataset |
| Load and resilience | Public/readiness/heartbeat k6 profile exists | Authenticated writes, chat, tracking, reconnect, rolling replacement, and worker retry profiles pass without touching staging data |
| Observability | Protected Prometheus text endpoint exists | Local Prometheus/Grafana/Alertmanager profile scrapes every replica and an induced isolated failure exercises alert delivery |
| Backup | Validated local custom-format dump and restore drill exist | An encrypted artifact is uploaded to local S3-compatible MinIO and restored into a fresh database; corruption and interrupted-upload checks fail closed |
| External boundaries | Mailpit and a fake GitHub strategy cover parts of SMTP/OAuth | Local protocol-level SMTP/OAuth mocks and the applicable push adapter boundary cover success, rejection, retry, replay, and timeout |
| Final regression | 153 Phoenix tests plus reproducible browser and Android device suites | Browser, Android, API, DB, WebSocket, backup, monitoring, failure, cleanup, docs, and clean Git are verified from the final commits |
| Final regression | 161 Phoenix tests plus reproducible browser and Android device suites | Browser, Android, API, DB, WebSocket, backup, monitoring, failure, cleanup, docs, and clean Git are verified from the final commits |
The goal remains open while any row lacks reproducible local evidence.
@ -55,7 +55,7 @@ The goal remains open while any row lacks reproducible local evidence.
- The containerized `scripts/quality.sh` gate passes ShellCheck, Hadolint,
actionlint, all Compose renders, Helm lint, a Trivy scan of tracked source and
the rendered Kubernetes manifest, compiler/xref/Credo/Sobelow/Dialyzer/Hex
checks, 153 Phoenix tests, both npm audits, and a Trivy scan of the production
checks, 161 Phoenix tests, both npm audits, and a Trivy scan of the production
release image. It creates random one-run database credentials and removes its
exact volume, networks, images, and source snapshot.
- The checked-in GitHub Actions workflow runs the same isolated backend/security
@ -71,5 +71,12 @@ The goal remains open while any row lacks reproducible local evidence.
authenticated LiveViews; category descriptions and structured values follow
that browser locale. The localization browser specs run in the same isolated
two-web/two-worker gate as the accessibility checks.
- The isolated three-web/two-worker load profile now uses real password login,
authenticated LiveView joins, tracking start, HTTP position writes, private
chat writes, and explicit tracking stop. A measured 30-second run completed
all 1,880 authenticated chains and 240 heartbeat sockets with no functional
failure, retained no current positions after stop, passed cross-node PubSub
and readiness, and restored every tracked table count after exact fixture
cleanup.
- The remaining rows above are still pending; this document is not a
completion claim for the entire hardening goal.

69
docs/performance.md
View File

@ -13,11 +13,14 @@ the load profile cannot join the staging router. It exercises:
- dynamic public home and Safety pages;
- database-backed readiness queries;
- Phoenix WebSocket upgrades and protocol heartbeats;
- real password login and authenticated LiveView joins;
- consent-driven tracking start, authenticated location writes, private chat
writes, and explicit tracking stop/position deletion;
- the configured number of web and worker replicas;
- cross-node Phoenix PubSub after each run.
It does not authenticate users, create requests, send chat messages, update
locations, render a real browser, or reproduce an unknown future production
It does not render a real browser, retain a browser tracking session for an
unknown real-world trip duration, or reproduce an unknown future production
traffic mix. It also does not sample the k6 load-generator container itself.
Its results only describe the recorded host, container versions, replica
counts, database state, and experiment inputs.
@ -50,10 +53,29 @@ Each run writes ignored evidence below `output/performance/<label>/`:
- a machine-readable k6 summary and complete console report;
- periodic Docker CPU/memory/network/block-I/O/PID samples for every load
stack container;
- before/after application table counts and their diff;
- before/after application table counts, exact authenticated-write evidence,
fixture cleanup output, and a cleanup diff;
- final Compose state, readiness response, application logs, and cross-node
PubSub result.
Before k6 starts, the production-compiled fixture task verifies the exact
configured load database name and a deliberate confirmation marker. It creates
isolated requester/helper/request/assignment pairs through the application
contexts. The pair count equals the maximum simultaneous VU count across all
scenarios, because k6 may reuse its global VU pool between parallel scenarios;
mapping each global VU identifier directly to its own pair prevents concurrent
users from sharing a tracking assignment. The random fixture password exists
only in ignored mode-600 `.env.load` and is never written to the manifest or
console.
On success and on trapped failure, cleanup deletes only the UUIDs recorded in
the validated run manifest. A successful run additionally requires every
tracked application-table count after cleanup to equal its pre-fixture count.
In a separate negative check, replacing one helper UUID with another valid UUID
made cleanup exit nonzero before mutation; all 2 users, 1 request, and 1
assignment remained. Restoring the original manifest then removed those exact
rows and returned all three counts to zero.
The k6 image is pinned to version 2.1.0 by digest and runs locally with anonymous
usage reporting disabled. No Grafana Cloud account, API token, OpenAI API, or
usage-based service is involved.
@ -110,6 +132,47 @@ Ignored evidence:
- `output/performance/two-web-final-20260718/`
- `output/performance/three-web-final-20260718/`
## Observed authenticated write run
Observed on 2026-07-19 on the same recorded Docker host with 3 web replicas,
2 worker replicas, 40 paced public HTTP VUs, 40 five-second Phoenix heartbeat
VUs, and 8 authenticated mutual-aid VUs for 30 seconds. The fixture tool
created 88 isolated pairs so every possible global VU had a unique account and
assignment.
| Observation | Result |
| --- | ---: |
| Public HTTP checks | 35,286 / 35,286 passed |
| All HTTP failures | 0 / 39,062 |
| Phoenix heartbeat opened / replies | 240 / 240 |
| Authenticated logins | 8 |
| Authenticated pages / LiveView joins | 1,880 / 1,880 |
| Tracking starts / position writes / stops | 1,880 / 1,880 / 1,880 |
| Private messages written | 1,880 |
| Authenticated or WebSocket errors | 0 |
| HTTP request duration p95 | 5.30 ms |
| Authenticated HTTP duration p95 | 8.18 ms |
| WebSocket connect p95 | 3.55 ms |
The database evidence counted 1,880 chat rows, 1,880 tracking sessions with
1,880 total samples, 8 login tokens, and zero current positions after explicit
stop. Cleanup removed the 176 fixture users, 88 requests, 88 assignments,
1,880 messages, 1,880 tracking sessions, and their audit rows. The post-cleanup
application-table diff was empty. Readiness and the five-node web/worker PubSub
probe passed, and the run-scoped proxy/web/worker logs contained no application
error, warning, HTTP 4xx/5xx, or database deadlock.
Eleven sequential resource samples observed average CPU of 139.84%, 138.12%,
and 141.90% for the three web containers, with maximum memory of 290.2, 309.4,
and 292.3 MiB respectively. PostgreSQL averaged 38.23% CPU and reached
237.1 MiB; Traefik averaged 45.53% CPU and reached 82.39 MiB. These are
short local observations without container resource limits, not minimums,
production capacity, or autoscaling inputs.
Ignored evidence:
- `output/performance/auth-final-20260719i/`
Stop the isolated containers without deleting their database volume:
```sh

17
docs/verification.md
View File

@ -16,7 +16,7 @@ results from product limits and unknown production properties.
| Consent-driven live tracking | Implemented and cross-client verified | On API 37, Android started `TrackingService` as a location foreground service with a persistent Stop notification. After Home minimized the Activity, an emulator coordinate change reached PostGIS. Notification Stop removed the service, notification, active session, and raw position. | Browsers stop with the page. Android has no `ACCESS_BACKGROUND_LOCATION`, unattended start, or route history. |
| Privacy settings | Implemented and browser-verified | The profile exposed hidden, approximate public, exact for active match, and explicit exact-public options. Blocking and current-position cleanup have automated tests. | Exact public location remains a user opt-in; legal privacy and retention text still requires jurisdiction-specific review before launch. |
| Reputation and anti-abuse | Implemented at MVP level | Handover codes, two-party completion, double-blind reviews, unique-counterpart ranking, optional movement/proximity evidence, reports, blocks, abuse signals, and moderator audit paths have automated tests. | The system is not bot-proof and does not claim identity verification. No punitive numeric policy is enabled without measured and approved thresholds. |
| Social profiles | Manual links implemented; GitHub verification implemented and automated-tested | Manual links cannot set verification fields. The optional GitHub flow uses state, PKCE, a user-bound one-time session, unique provider ownership, and an audit record; 153 tests pass, including callback replay/state checks. No access-token field exists and the controller receives only normalized identity attributes. | The staging operator has not supplied GitHub OAuth credentials, so the real external provider redirect/callback remains disabled and has not been browser-verified. Other providers remain manual/unverified. |
| Social profiles | Manual links implemented; GitHub verification implemented and automated-tested | Manual links cannot set verification fields. The optional GitHub flow uses state, PKCE, a user-bound one-time session, unique provider ownership, and an audit record; 161 tests pass, including callback replay/state checks. No access-token field exists and the controller receives only normalized identity attributes. | The staging operator has not supplied GitHub OAuth credentials, so the real external provider redirect/callback remains disabled and has not been browser-verified. Other providers remain manual/unverified. |
| Voluntary thanks | Implemented as an external optional link | A helper can expose an optional link after completion; the UI states that the platform does not process the payment. | The platform does not provide payments, escrow, refunds, tax reporting, or payment guarantees. |
| Android client | Local and public-staging clients implemented and emulator-verified | The native packages `org.whoneedhelp.mobile.debug` and `org.whoneedhelp.mobile.staging` launch the same authenticated LiveView app. Public HTTPS login, map, two-way chat, permission prompts, minimized foreground-service location updates, notification Stop, deep-link routing, and server cleanup were exercised on API 37. | Production signing, Play Store publication, verified Android App Links, unattended/background-permission tracking, and iOS are not implemented. |
| Multiple web/worker instances | Implemented and locally verified | Docker Compose and kind each ran 2 web and 2 worker replicas. The project probes cross-node Phoenix PubSub using different BEAM nodes. Kubernetes web/worker pods were Ready with zero restarts at the final observation. | Local PostGIS is a single instance. Production database HA, backups, and recovery are operator work and are not claimed complete. |
@ -186,11 +186,16 @@ for that reverse lookup selected the index. The table contained no block rows,
so this confirms query shape, not production performance.
Exact production capacity, minimum CPU/RAM, and scaling thresholds are unknown:
there is no representative load dataset or target-environment measurement. An
isolated 40 HTTP VU / 40 WebSocket VU local comparison is recorded in
`docs/performance.md`; it tested a paced public/readiness/heartbeat workload
with 2 and 3 web replicas but did not find a saturation point. The Helm chart
therefore does not invent resource limits or an HPA policy.
there is no representative production dataset, traffic model, or
target-environment measurement. The isolated measurements in
`docs/performance.md` include the earlier paced public/readiness/heartbeat
comparison and a later 3-web/2-worker run with real authentication, LiveView,
chat writes, tracking writes, and explicit position deletion. The latter
completed 1,880 authenticated chains and 240 heartbeat sockets without a
functional failure and restored every tracked application table count after
fixture cleanup. Neither short run found a saturation point or represents
production traffic, so the Helm chart does not invent resource limits or an HPA
policy.
## Public staging observation

420
lib/mix/tasks/wnh.load_fixtures.ex Normal file
View File

@ -0,0 +1,420 @@
defmodule Mix.Tasks.Wnh.LoadFixtures do
use Mix.Task
import Ecto.Query
alias WhoNeedHelp.Accounts.{Scope, User}
alias WhoNeedHelp.Catalog
alias WhoNeedHelp.Help
alias WhoNeedHelp.Help.{Assignment, HelpRequest}
alias WhoNeedHelp.Messaging.Message
alias WhoNeedHelp.Repo
alias WhoNeedHelp.Tracking.{Position, TrackingSession}
alias WhoNeedHelp.Trust.{AbuseSignal, AuditEvent, Report, Review}
@shortdoc "Prepares or removes isolated authenticated load fixtures"
@confirmation "isolated-load-fixtures"
@impl Mix.Task
def run([action]) when action in ["prepare", "cleanup"] do
Mix.Task.run("app.start")
context = verified_context()
case action do
"prepare" -> prepare(context)
"cleanup" -> cleanup(context)
end
end
def run(_args) do
Mix.raise("usage: mix wnh.load_fixtures prepare|cleanup")
end
defp verified_context do
run_id = required_env!("WNH_LOAD_FIXTURE_RUN_ID")
expected_database = required_env!("WNH_LOAD_EXPECTED_DATABASE")
manifest_path = required_env!("WNH_LOAD_FIXTURE_PATH") |> Path.expand()
unless Regex.match?(~r/^[A-Za-z0-9._-]+$/, run_id) do
Mix.raise(
"WNH_LOAD_FIXTURE_RUN_ID may contain only letters, numbers, dot, underscore, and dash"
)
end
unless System.get_env("WNH_LOAD_FIXTURE_CONFIRM") == @confirmation do
Mix.raise("WNH_LOAD_FIXTURE_CONFIRM must equal #{@confirmation}")
end
unless String.starts_with?(manifest_path, "/output/") do
Mix.raise("WNH_LOAD_FIXTURE_PATH must resolve below /output")
end
%Postgrex.Result{rows: [[actual_database]]} =
Repo.query!("SELECT current_database()", [], log: false)
unless actual_database == expected_database do
Mix.raise(
"refusing load-fixture mutation: expected database #{inspect(expected_database)}, " <>
"observed #{inspect(actual_database)}"
)
end
%{
run_id: run_id,
database: actual_database,
manifest_path: manifest_path
}
end
defp prepare(context) do
count = positive_integer_env!("WNH_LOAD_FIXTURE_COUNT")
password = required_env!("LOAD_FIXTURE_PASSWORD")
unless byte_size(password) in 12..72 do
Mix.raise("LOAD_FIXTURE_PASSWORD must contain between 12 and 72 bytes")
end
email_prefix = email_prefix(context.run_id)
if Repo.exists?(
from(user in User,
where: fragment("left(?, ?) = ?", user.email, ^byte_size(email_prefix), ^email_prefix)
)
) do
Mix.raise("fixtures already exist for run #{inspect(context.run_id)}")
end
category = Catalog.seed_defaults()
password_hash = Bcrypt.hash_pwd_salt(password)
now = DateTime.utc_now(:second)
{:ok, fixtures} =
Repo.transaction(fn ->
Enum.map(1..count, fn index ->
requester =
insert_user!(
"#{email_prefix}requester-#{index}@example.invalid",
"Load requester #{index}",
password_hash,
now
)
helper =
insert_user!(
"#{email_prefix}helper-#{index}@example.invalid",
"Load helper #{index}",
password_hash,
now
)
latitude = 50.45 + index / 100_000
longitude = 30.52 + index / 100_000
request =
Help.create_request(Scope.for_user(requester), %{
"title" => "Load medicine pickup #{context.run_id} #{index}",
"description" =>
"Isolated authenticated load fixture for a reserved legal medicine pickup.",
"pickup_instructions" => "Synthetic load fixture; no real pickup.",
"structured_data" => %{"pickup_status" => "reserved"},
"location_label" => "Isolated load location #{index}",
"latitude" => Float.to_string(latitude),
"longitude" => Float.to_string(longitude),
"urgency" => "now",
"location_visibility" => "exact_for_active_match",
"expires_at" => DateTime.add(now, 86_400, :second),
"category_id" => category.id
})
|> unwrap!("create request")
assignment =
Help.accept_request(Scope.for_user(helper), request.id)
|> unwrap!("accept request")
%{
"index" => index,
"requester_id" => requester.id,
"helper_id" => helper.id,
"helper_email" => helper.email,
"request_id" => request.id,
"assignment_id" => assignment.id,
"latitude" => latitude,
"longitude" => longitude
}
end)
end)
manifest = %{
"schema_version" => 1,
"run_id" => context.run_id,
"database" => context.database,
"fixtures" => fixtures
}
context.manifest_path
|> Path.dirname()
|> File.mkdir_p!()
File.write!(context.manifest_path, Jason.encode_to_iodata!(manifest, pretty: true))
# The parent evidence directory is mode 0700 on the host. The manifest
# contains only temporary IDs and emails (never the password) and must be
# readable by the separately containerized, unprivileged k6 process.
File.chmod!(context.manifest_path, 0o644)
Mix.shell().info(
"prepared #{length(fixtures)} fixture pairs for #{context.run_id} in #{context.database}"
)
end
defp cleanup(context) do
manifest = context.manifest_path |> File.read!() |> Jason.decode!()
unless manifest["schema_version"] == 1 and manifest["run_id"] == context.run_id and
manifest["database"] == context.database do
Mix.raise("fixture manifest does not match the requested run and database")
end
fixtures = manifest["fixtures"]
unless is_list(fixtures) and fixtures != [] do
Mix.raise("fixture manifest has no non-empty fixture list")
end
user_ids = Enum.flat_map(fixtures, &[&1["requester_id"], &1["helper_id"]])
request_ids = Enum.map(fixtures, & &1["request_id"])
assignment_ids = Enum.map(fixtures, & &1["assignment_id"])
unless Enum.all?(user_ids ++ request_ids ++ assignment_ids, &uuid?/1) do
Mix.raise("fixture manifest contains an invalid identifier")
end
validate_manifest_ownership!(context, fixtures)
{:ok, deleted} =
Repo.transaction(fn ->
tracking_session_ids =
TrackingSession
|> where([session], session.assignment_id in ^assignment_ids)
|> select([session], session.id)
|> Repo.all()
message_ids =
Message
|> where([message], message.assignment_id in ^assignment_ids)
|> select([message], message.id)
|> Repo.all()
report_count =
Report
|> where(
[report],
report.reporter_id in ^user_ids or report.request_id in ^request_ids or
report.assignment_id in ^assignment_ids or report.message_id in ^message_ids
)
|> Repo.delete_all()
|> elem(0)
audit_count =
AuditEvent
|> where(
[event],
event.actor_id in ^user_ids or
(event.target_type == "request" and event.target_id in ^request_ids) or
(event.target_type == "assignment" and event.target_id in ^assignment_ids)
)
|> Repo.delete_all()
|> elem(0)
position_count =
Position
|> where([position], position.tracking_session_id in ^tracking_session_ids)
|> Repo.delete_all()
|> elem(0)
session_count =
TrackingSession
|> where([session], session.id in ^tracking_session_ids)
|> Repo.delete_all()
|> elem(0)
message_count =
Message
|> where([message], message.assignment_id in ^assignment_ids)
|> Repo.delete_all()
|> elem(0)
abuse_count =
AbuseSignal
|> where(
[signal],
signal.subject_id in ^user_ids or signal.assignment_id in ^assignment_ids
)
|> Repo.delete_all()
|> elem(0)
review_count =
Review
|> where(
[review],
review.assignment_id in ^assignment_ids or review.reviewer_id in ^user_ids or
review.reviewee_id in ^user_ids
)
|> Repo.delete_all()
|> elem(0)
assignment_count =
Assignment
|> where([assignment], assignment.id in ^assignment_ids)
|> Repo.delete_all()
|> elem(0)
request_count =
HelpRequest
|> where([request], request.id in ^request_ids)
|> Repo.delete_all()
|> elem(0)
user_count =
User
|> where([user], user.id in ^user_ids)
|> Repo.delete_all()
|> elem(0)
%{
reports: report_count,
audit_events: audit_count,
tracking_positions: position_count,
tracking_sessions: session_count,
messages: message_count,
abuse_signals: abuse_count,
reviews: review_count,
assignments: assignment_count,
requests: request_count,
users: user_count
}
end)
Mix.shell().info(
"removed fixtures for #{context.run_id} from #{context.database}: #{inspect(deleted)}"
)
end
defp insert_user!(email, display_name, password_hash, now) do
%User{}
|> User.registration_changeset(%{
"email" => email,
"display_name" => display_name,
"locale" => "en",
"terms_accepted" => true
})
|> Ecto.Changeset.put_change(:hashed_password, password_hash)
|> Ecto.Changeset.put_change(:confirmed_at, now)
|> Repo.insert!()
end
defp validate_manifest_ownership!(context, fixtures) do
indexes = Enum.map(fixtures, & &1["index"])
expected_indexes = Enum.to_list(1..length(fixtures))
unless Enum.all?(indexes, &(is_integer(&1) and &1 > 0)) and
Enum.sort(indexes) == expected_indexes do
Mix.raise("fixture manifest indexes are not the exact prepared sequence")
end
prefix = email_prefix(context.run_id)
expected_users =
fixtures
|> Enum.flat_map(fn fixture ->
requester_email = "#{prefix}requester-#{fixture["index"]}@example.invalid"
helper_email = "#{prefix}helper-#{fixture["index"]}@example.invalid"
unless fixture["helper_email"] == helper_email do
Mix.raise("fixture manifest helper identity does not match its run and index")
end
[
{fixture["requester_id"], requester_email},
{fixture["helper_id"], helper_email}
]
end)
|> MapSet.new()
expected_requests =
fixtures
|> Enum.map(&{&1["request_id"], &1["requester_id"]})
|> MapSet.new()
expected_assignments =
fixtures
|> Enum.map(&{&1["assignment_id"], &1["request_id"], &1["helper_id"]})
|> MapSet.new()
user_ids = Enum.map(expected_users, &elem(&1, 0))
request_ids = Enum.map(expected_requests, &elem(&1, 0))
assignment_ids = Enum.map(expected_assignments, &elem(&1, 0))
observed_users =
User
|> where([user], user.id in ^user_ids)
|> select([user], {user.id, user.email})
|> Repo.all()
|> MapSet.new()
observed_requests =
HelpRequest
|> where([request], request.id in ^request_ids)
|> select([request], {request.id, request.requester_id})
|> Repo.all()
|> MapSet.new()
observed_assignments =
Assignment
|> where([assignment], assignment.id in ^assignment_ids)
|> select(
[assignment],
{assignment.id, assignment.request_id, assignment.helper_id}
)
|> Repo.all()
|> MapSet.new()
expected_pair_count = length(fixtures)
unless MapSet.size(expected_users) == expected_pair_count * 2 and
MapSet.size(expected_requests) == expected_pair_count and
MapSet.size(expected_assignments) == expected_pair_count and
observed_users == expected_users and
observed_requests == expected_requests and
observed_assignments == expected_assignments do
Mix.raise("fixture manifest ownership does not exactly match the load database")
end
end
defp unwrap!({:ok, value}, _operation), do: value
defp unwrap!({:error, reason}, operation) do
Repo.rollback("#{operation} failed: #{inspect(reason)}")
end
defp positive_integer_env!(name) do
case Integer.parse(required_env!(name)) do
{value, ""} when value > 0 -> value
_ -> Mix.raise("#{name} must be a positive integer")
end
end
defp required_env!(name) do
case System.get_env(name) do
value when is_binary(value) and value != "" -> value
_ -> Mix.raise("#{name} is required")
end
end
defp email_prefix(run_id), do: "wnh-load-#{String.downcase(run_id)}-"
defp uuid?(value) when is_binary(value) do
match?({:ok, _binary}, Ecto.UUID.dump(value))
end
defp uuid?(_value), do: false
end

350
load/k6/public-and-websocket.js
View File

@ -1,5 +1,6 @@
import { check, sleep } from "k6";
import http from "k6/http";
import exec from "k6/execution";
import { Counter } from "k6/metrics";
import { WebSocket } from "k6/websockets";
@ -34,34 +35,79 @@ function nonNegativeNumber(name) {
}
const baseUrl = required("BASE_URL").replace(/\/+$/, "");
const publicOrigin = required("PUBLIC_ORIGIN").replace(/\/+$/, "");
if (!/^https?:\/\/[^/?#]+$/.test(baseUrl)) {
throw new Error("BASE_URL must be an HTTP(S) origin without a path");
}
if (!/^https?:\/\/[^/?#]+$/.test(publicOrigin)) {
throw new Error("PUBLIC_ORIGIN must be an HTTP(S) origin without a path");
}
const websocketUrl =
baseUrl.replace(/^http/, "ws") + "/live/websocket?vsn=2.0.0";
const websocketHoldMs = positiveInteger("WS_HOLD_MS");
const websocketConnectTimeoutMs = positiveInteger("WS_CONNECT_TIMEOUT_MS");
const httpThinkSeconds = nonNegativeNumber("HTTP_THINK_SECONDS");
const httpVus = positiveInteger("HTTP_VUS");
const websocketVus = positiveInteger("WS_VUS");
const authenticatedVus = positiveInteger("AUTH_VUS");
const maxVus = httpVus + websocketVus + authenticatedVus;
const authWebsocketTimeoutMs = positiveInteger("AUTH_WS_TIMEOUT_MS");
const authThinkSeconds = nonNegativeNumber("AUTH_THINK_SECONDS");
const fixturePassword = required("FIXTURE_PASSWORD");
const fixtureManifest = JSON.parse(open("/fixtures/fixtures.json"));
const fixtureRunId = required("FIXTURE_RUN_ID");
if (
fixtureManifest.schema_version !== 1 ||
fixtureManifest.run_id !== fixtureRunId ||
!Array.isArray(fixtureManifest.fixtures) ||
fixtureManifest.fixtures.length < maxVus
) {
throw new Error("The authenticated-load fixture manifest is invalid or too small");
}
const websocketOpened = new Counter("wnh_websocket_opened");
const websocketErrors = new Counter("wnh_websocket_errors");
const heartbeatReplies = new Counter("wnh_websocket_heartbeat_replies");
const authenticatedLogins = new Counter("wnh_authenticated_logins");
const authenticatedPages = new Counter("wnh_authenticated_pages");
const trackingUpdates = new Counter("wnh_tracking_updates");
const liveViewJoins = new Counter("wnh_liveview_joins");
const liveViewTrackingStarts = new Counter("wnh_liveview_tracking_starts");
const liveViewTrackingStops = new Counter("wnh_liveview_tracking_stops");
const liveViewMessages = new Counter("wnh_liveview_messages");
const authenticatedErrors = new Counter("wnh_authenticated_errors");
let authenticated = false;
let fixture;
export const options = {
discardResponseBodies: true,
// The isolated Traefik profile uses its generated local-only certificate.
// Transport remains TLS; trust verification is scoped to this k6 process.
insecureSkipTLSVerify: true,
// Each authenticated VU represents one continuing user session.
noCookiesReset: true,
scenarios: {
public_http: {
executor: "constant-vus",
exec: "publicHttp",
vus: positiveInteger("HTTP_VUS"),
vus: httpVus,
duration: required("DURATION"),
},
phoenix_websocket: {
executor: "constant-vus",
exec: "phoenixWebsocket",
vus: positiveInteger("WS_VUS"),
vus: websocketVus,
duration: required("DURATION"),
},
authenticated_mutual_aid: {
executor: "constant-vus",
exec: "authenticatedMutualAid",
vus: authenticatedVus,
duration: required("DURATION"),
},
},
@ -152,3 +198,303 @@ export function phoenixWebsocket() {
}
});
}
function htmlAttribute(tag, name) {
const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const match = tag.match(new RegExp(`\\s${escaped}=(?:"([^"]*)"|'([^']*)')`));
if (!match) {
return null;
}
return (match[1] ?? match[2])
.replaceAll("&amp;", "&")
.replaceAll("&quot;", '"')
.replaceAll("&#39;", "'");
}
function csrfToken(body) {
const metaTags = body.match(/<meta\b[^>]*>/gi) || [];
const tag = metaTags.find((candidate) => htmlAttribute(candidate, "name") === "csrf-token");
return tag ? htmlAttribute(tag, "content") : null;
}
function liveViewRoot(body) {
const tags = body.match(/<[^!/][^>]*>/g) || [];
const tag = tags.find((candidate) => /\sdata-phx-main(?:\s|=|>)/.test(candidate));
if (!tag) {
return null;
}
const root = {
id: htmlAttribute(tag, "id"),
session: htmlAttribute(tag, "data-phx-session"),
static: htmlAttribute(tag, "data-phx-static"),
};
return root.id && root.session ? root : null;
}
function fixtureForVu() {
const index = exec.vu.idInInstance - 1;
const selected = fixtureManifest.fixtures[index];
if (!selected) {
throw new Error(`No fixture exists for VU ${exec.vu.idInInstance}`);
}
return selected;
}
function authenticateActor(actor) {
const loginPage = http.get(`${baseUrl}/users/log-in`, {
responseType: "text",
tags: { endpoint: "auth_login_page" },
});
const csrf = typeof loginPage.body === "string" ? csrfToken(loginPage.body) : null;
if (loginPage.status !== 200 || !csrf) {
authenticatedErrors.add(1);
return false;
}
const login = http.post(
`${baseUrl}/users/log-in`,
{
_csrf_token: csrf,
"user[email]": actor.helper_email,
"user[password]": fixturePassword,
},
{
redirects: 0,
tags: { endpoint: "auth_password_login" },
},
);
const jar = http.cookieJar();
const cookies = jar.cookiesForURL(baseUrl);
const succeeded =
login.status === 302 &&
Array.isArray(cookies._who_need_help_key) &&
cookies._who_need_help_key.length > 0;
if (succeeded) {
authenticatedLogins.add(1);
} else {
authenticatedErrors.add(1);
}
return succeeded;
}
export function authenticatedMutualAid() {
if (!fixture) {
fixture = fixtureForVu();
}
if (!authenticated) {
authenticated = authenticateActor(fixture);
}
if (!authenticated) {
sleep(authThinkSeconds);
return;
}
const requestPath = `/requests/${fixture.request_id}`;
const page = http.get(`${baseUrl}${requestPath}`, {
responseType: "text",
tags: { endpoint: "authenticated_request_page" },
});
const body = typeof page.body === "string" ? page.body : "";
const csrf = csrfToken(body);
const root = liveViewRoot(body);
if (page.status !== 200 || !csrf || !root) {
authenticatedErrors.add(1);
sleep(authThinkSeconds);
return;
}
authenticatedPages.add(1);
const socketUrl =
baseUrl.replace(/^http/, "ws") +
`/live/websocket?_csrf_token=${encodeURIComponent(csrf)}` +
"&client_type=browser&vsn=2.0.0";
const socket = new WebSocket(socketUrl, [], {
jar: http.cookieJar(),
headers: { Origin: publicOrigin },
tags: { endpoint: "authenticated_liveview" },
});
const joinRef = `${exec.vu.idInInstance}-${__ITER}-join`;
const trackingRef = `${exec.vu.idInInstance}-${__ITER}-tracking`;
const eventRef = `${exec.vu.idInInstance}-${__ITER}-message`;
const stopRef = `${exec.vu.idInInstance}-${__ITER}-stop`;
const topic = `lv:${root.id}`;
let completed = false;
const timer = setTimeout(() => {
if (!completed) {
authenticatedErrors.add(1);
}
socket.close();
}, authWebsocketTimeoutMs);
socket.addEventListener("open", () => {
socket.send(
JSON.stringify([
joinRef,
joinRef,
topic,
"phx_join",
{
url: `${publicOrigin}${requestPath}`,
params: {
_csrf_token: csrf,
client_type: "browser",
_mounts: 0,
_mount_attempts: 0,
_track_static: [],
},
session: root.session,
static: root.static || null,
},
]),
);
});
socket.addEventListener("message", (event) => {
let frame;
try {
frame = JSON.parse(event.data);
} catch (_error) {
return;
}
if (!Array.isArray(frame) || frame[2] !== topic || frame[3] !== "phx_reply") {
return;
}
if (frame[1] === joinRef && frame[4]?.status === "ok") {
liveViewJoins.add(1);
socket.send(
JSON.stringify([
joinRef,
trackingRef,
topic,
"event",
{
type: "click",
event: "start-tracking",
value: {},
cid: null,
},
]),
);
} else if (frame[1] === trackingRef && frame[4]?.status === "ok") {
liveViewTrackingStarts.add(1);
const trackingStartResponse = frame[4]?.response ?? null;
const latitude = fixture.latitude + ((__ITER % 20) + 1) / 1_000_000;
const longitude = fixture.longitude + ((__ITER % 20) + 1) / 1_000_000;
const tracking = http.post(
`${baseUrl}/mobile/tracking/${fixture.assignment_id}/position`,
{
latitude: String(latitude),
longitude: String(longitude),
accuracy_meters: "5",
captured_at: new Date().toISOString(),
},
{
headers: { "x-csrf-token": csrf },
responseType: "text",
tags: { endpoint: "authenticated_tracking_update" },
},
);
if (tracking.status !== 204) {
console.error(
`authenticated tracking update failed: ${JSON.stringify({
vu_id_in_instance: exec.vu.idInInstance,
vu_id_in_test: exec.vu.idInTest,
fixture_index: fixture.index,
iteration: __ITER,
status: tracking.status,
response_body: tracking.body,
tracking_start_response: trackingStartResponse,
})}`,
);
completed = true;
authenticatedErrors.add(1);
clearTimeout(timer);
socket.close();
return;
}
trackingUpdates.add(1);
const message =
`load:${fixtureRunId}:vu-${exec.vu.idInInstance}:iteration-${__ITER}`;
const formValue = `message%5Bbody%5D=${encodeURIComponent(message)}`;
socket.send(
JSON.stringify([
joinRef,
eventRef,
topic,
"event",
{
type: "form",
event: "send-message",
value: formValue,
meta: { _target: "undefined" },
uploads: {},
cid: null,
},
]),
);
} else if (frame[1] === eventRef && frame[4]?.status === "ok") {
liveViewMessages.add(1);
socket.send(
JSON.stringify([
joinRef,
stopRef,
topic,
"event",
{
type: "click",
event: "stop-tracking",
value: {},
cid: null,
},
]),
);
} else if (frame[1] === stopRef && frame[4]?.status === "ok") {
completed = true;
liveViewTrackingStops.add(1);
clearTimeout(timer);
socket.close();
} else if (
frame[1] === joinRef ||
frame[1] === trackingRef ||
frame[1] === eventRef ||
frame[1] === stopRef
) {
completed = true;
authenticatedErrors.add(1);
clearTimeout(timer);
socket.close();
}
});
socket.addEventListener("error", () => {
if (!completed) {
completed = true;
authenticatedErrors.add(1);
}
clearTimeout(timer);
socket.close();
});
socket.addEventListener("close", () => clearTimeout(timer));
sleep(authThinkSeconds);
}

35
scripts/ensure-local-load-env.sh
View File

@ -5,12 +5,6 @@ ROOT=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd)
TEMPLATE="$ROOT/.env.load.example"
ENV_FILE="$ROOT/.env.load"
if [ -f "$ENV_FILE" ]; then
chmod 600 "$ENV_FILE"
echo ".env.load already exists; no secret or experiment input was changed."
exit 0
fi
for command in openssl perl; do
if ! command -v "$command" >/dev/null 2>&1; then
echo "Required command is unavailable: $command" >&2
@ -18,6 +12,29 @@ for command in openssl perl; do
fi
done
if [ -f "$ENV_FILE" ]; then
chmod 600 "$ENV_FILE"
if grep -q '^LOAD_FIXTURE_PASSWORD=' "$ENV_FILE"; then
echo ".env.load already exists; no secret or experiment input was changed."
exit 0
fi
umask 077
load_fixture_password=$(openssl rand -hex 24)
{
printf '\n# Added by the authenticated-load profile upgrade.\n'
printf 'LOAD_AUTH_VUS=8\n'
printf 'LOAD_AUTH_WS_TIMEOUT_MS=5000\n'
printf 'LOAD_AUTH_THINK_SECONDS=0.1\n'
printf 'LOAD_FIXTURE_PASSWORD=%s\n' "$load_fixture_password"
} >>"$ENV_FILE"
chmod 600 "$ENV_FILE"
unset load_fixture_password
echo "Added authenticated-load inputs and a random fixture password to ignored .env.load."
exit 0
fi
if [ ! -f "$TEMPLATE" ]; then
echo "Missing tracked template: $TEMPLATE" >&2
exit 1
@ -29,6 +46,7 @@ secret_key_base=$(openssl rand -hex 64)
handover_secret=$(openssl rand -hex 64)
release_cookie=$(openssl rand -hex 64)
metrics_token=$(openssl rand -hex 32)
load_fixture_password=$(openssl rand -hex 24)
database_url="ecto://wnh_load:${postgres_password}@db/who_need_help_load"
temporary=$(mktemp "${ENV_FILE}.XXXXXX")
trap 'rm -f "$temporary"' EXIT HUP INT TERM
@ -39,6 +57,7 @@ SECRET_KEY_BASE_VALUE=$secret_key_base \
HANDOVER_SECRET_VALUE=$handover_secret \
RELEASE_COOKIE_VALUE=$release_cookie \
METRICS_TOKEN_VALUE=$metrics_token \
LOAD_FIXTURE_PASSWORD_VALUE=$load_fixture_password \
perl -0pe '
s/GENERATE_POSTGRES_PASSWORD/$ENV{POSTGRES_PASSWORD_VALUE}/g;
s/GENERATE_DATABASE_URL/$ENV{DATABASE_URL_VALUE}/g;
@ -46,6 +65,7 @@ METRICS_TOKEN_VALUE=$metrics_token \
s/GENERATE_HANDOVER_SECRET/$ENV{HANDOVER_SECRET_VALUE}/g;
s/GENERATE_RELEASE_COOKIE/$ENV{RELEASE_COOKIE_VALUE}/g;
s/GENERATE_METRICS_TOKEN/$ENV{METRICS_TOKEN_VALUE}/g;
s/GENERATE_LOAD_FIXTURE_PASSWORD/$ENV{LOAD_FIXTURE_PASSWORD_VALUE}/g;
' "$TEMPLATE" >"$temporary"
if grep -Eq '^[A-Z0-9_]+=GENERATE_' "$temporary"; then
@ -56,6 +76,7 @@ fi
chmod 600 "$temporary"
mv "$temporary" "$ENV_FILE"
trap - EXIT HUP INT TERM
unset postgres_password secret_key_base handover_secret release_cookie metrics_token database_url
unset postgres_password secret_key_base handover_secret release_cookie metrics_token \
load_fixture_password database_url
echo "Generated independent load-profile secrets in ignored .env.load."

192
scripts/load-run.sh
View File

@ -16,15 +16,27 @@ set -a
. "$ENV_FILE"
set +a
for name in LOAD_PROJECT LOAD_HOST LOAD_WEB_REPLICAS LOAD_HTTP_VUS LOAD_WS_VUS \
for name in LOAD_PROJECT LOAD_HOST LOAD_WEB_REPLICAS LOAD_WORKER_REPLICAS \
LOAD_HTTP_VUS LOAD_WS_VUS \
LOAD_DURATION LOAD_WS_HOLD_MS LOAD_WS_CONNECT_TIMEOUT_MS \
LOAD_HTTP_THINK_SECONDS; do
LOAD_HTTP_THINK_SECONDS LOAD_AUTH_VUS LOAD_AUTH_WS_TIMEOUT_MS \
LOAD_AUTH_THINK_SECONDS LOAD_FIXTURE_PASSWORD POSTGRES_DB HTTP_PORT; do
if [[ -z "${!name:-}" ]]; then
echo "$name is missing from .env.load" >&2
exit 1
fi
done
for name in LOAD_WEB_REPLICAS LOAD_WORKER_REPLICAS LOAD_HTTP_VUS LOAD_WS_VUS \
LOAD_AUTH_VUS; do
if [[ ! "${!name}" =~ ^[1-9][0-9]*$ ]]; then
echo "$name must be a positive integer." >&2
exit 1
fi
done
fixture_count=$((LOAD_HTTP_VUS + LOAD_WS_VUS + LOAD_AUTH_VUS))
if [[ "$LOAD_PROJECT" == "who_need_help" ]]; then
echo "The load profile must not use the staging Compose project." >&2
exit 1
@ -63,6 +75,24 @@ if [[ -z "$network_id" ]]; then
exit 1
fi
internal_network_id=$(
docker network ls \
--filter "label=com.docker.compose.project=$LOAD_PROJECT" \
--filter "label=com.docker.compose.network=internal" \
--quiet |
head -n 1
)
if [[ -z "$internal_network_id" ]]; then
echo "The isolated load-profile internal network was not found." >&2
exit 1
fi
if ! docker image inspect who-need-help:load-tools >/dev/null 2>&1; then
echo "The isolated load-tools image is missing. Run scripts/load-stack-up.sh first." >&2
exit 1
fi
mapfile -t measured_containers < <(
"${compose[@]}" ps -q web worker db proxy
)
@ -75,6 +105,7 @@ fi
output_dir="$ROOT/output/performance/$LABEL"
mkdir -p "$output_dir"
chmod 700 "$ROOT/output" "$ROOT/output/performance" "$output_dir"
run_started_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)
running_marker="$output_dir/.sampling"
resource_log="$output_dir/docker-stats.jsonl"
touch "$running_marker"
@ -89,19 +120,95 @@ snapshot_database() {
>"$destination" <<'SQL'
BEGIN READ ONLY;
SELECT 'users' AS table_name, count(*) AS row_count FROM users
UNION ALL SELECT 'users_tokens', count(*) FROM users_tokens
UNION ALL SELECT 'help_requests', count(*) FROM help_requests
UNION ALL SELECT 'messages', count(*) FROM messages
UNION ALL SELECT 'categories', count(*) FROM categories
UNION ALL SELECT 'category_proposals', count(*) FROM category_proposals
UNION ALL SELECT 'category_votes', count(*) FROM category_votes
UNION ALL SELECT 'help_assignments', count(*) FROM help_assignments
UNION ALL SELECT 'activities', count(*) FROM activities
UNION ALL SELECT 'activity_participants', count(*) FROM activity_participants
UNION ALL SELECT 'activity_messages', count(*) FROM activity_messages
UNION ALL SELECT 'reports', count(*) FROM reports
UNION ALL SELECT 'reviews', count(*) FROM reviews
UNION ALL SELECT 'blocks', count(*) FROM blocks
UNION ALL SELECT 'audit_events', count(*) FROM audit_events
UNION ALL SELECT 'abuse_signals', count(*) FROM abuse_signals
UNION ALL SELECT 'rate_limit_buckets', count(*) FROM rate_limit_buckets
UNION ALL SELECT 'social_identities', count(*) FROM social_identities
UNION ALL SELECT 'tracking_sessions', count(*) FROM tracking_sessions
UNION ALL SELECT 'tracking_positions', count(*) FROM tracking_positions
ORDER BY table_name;
COMMIT;
SQL
}
run_fixture_tool() {
local action=$1
docker run --rm \
--network "$internal_network_id" \
--env-file "$ENV_FILE" \
--env APP_ROLE=migrate \
--env "WNH_LOAD_EXPECTED_DATABASE=$POSTGRES_DB" \
--env WNH_LOAD_FIXTURE_CONFIRM=isolated-load-fixtures \
--env "WNH_LOAD_FIXTURE_RUN_ID=$LABEL" \
--env "WNH_LOAD_FIXTURE_COUNT=$fixture_count" \
--env "WNH_LOAD_FIXTURE_PATH=/output/fixtures.json" \
--volume "$output_dir:/output" \
who-need-help:load-tools \
mix wnh.load_fixtures "$action"
}
validate_authenticated_writes() {
# The run label was restricted to a conservative character set above and is
# passed as a psql variable, not interpolated into SQL syntax.
# shellcheck disable=SC2016
"${compose[@]}" exec -T db sh -c \
'psql --no-psqlrc --tuples-only --no-align --set ON_ERROR_STOP=1 \
--set run_id="$1" --username "$POSTGRES_USER" --dbname "$POSTGRES_DB"' \
sh "$LABEL" >"$output_dir/authenticated-writes.json" <<'SQL'
SELECT json_build_object(
'messages',
(SELECT count(*) FROM messages WHERE body LIKE 'load:' || :'run_id' || ':%'),
'tracking_samples',
(SELECT coalesce(sum(session.sample_count), 0)
FROM tracking_sessions AS session
JOIN users AS actor ON actor.id = session.user_id
WHERE left(actor.email, length('wnh-load-' || lower(:'run_id') || '-')) =
'wnh-load-' || lower(:'run_id') || '-'),
'tracking_sessions',
(SELECT count(*)
FROM tracking_sessions AS session
JOIN users AS actor ON actor.id = session.user_id
WHERE left(actor.email, length('wnh-load-' || lower(:'run_id') || '-')) =
'wnh-load-' || lower(:'run_id') || '-'),
'tracking_positions',
(SELECT count(*)
FROM tracking_positions AS position
JOIN tracking_sessions AS session ON session.id = position.tracking_session_id
JOIN users AS actor ON actor.id = session.user_id
WHERE left(actor.email, length('wnh-load-' || lower(:'run_id') || '-')) =
'wnh-load-' || lower(:'run_id') || '-'),
'session_tokens',
(SELECT count(*)
FROM users_tokens AS token
JOIN users AS actor ON actor.id = token.user_id
WHERE left(actor.email, length('wnh-load-' || lower(:'run_id') || '-')) =
'wnh-load-' || lower(:'run_id') || '-')
);
SQL
jq -e '
.messages > 0 and
.tracking_samples > 0 and
.tracking_sessions > 0 and
.tracking_positions == 0 and
.session_tokens > 0
' "$output_dir/authenticated-writes.json" >/dev/null
}
sample_resources() {
while [[ -e "$running_marker" ]]; do
observed_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)
@ -120,10 +227,34 @@ cleanup_sampler() {
fi
}
trap cleanup_sampler EXIT HUP INT TERM
fixtures_prepared=false
cleanup_on_exit() {
local status=$?
local cleanup_status=0
trap - EXIT HUP INT TERM
cleanup_sampler
if [[ "$fixtures_prepared" == true ]]; then
set +e
run_fixture_tool cleanup >>"$output_dir/fixture-cleanup.log" 2>&1
cleanup_status=$?
set -e
fi
if [[ "$cleanup_status" -ne 0 ]]; then
echo "Fixture cleanup failed; inspect $output_dir/fixture-cleanup.log." >&2
status=1
fi
exit "$status"
}
trap cleanup_on_exit EXIT HUP INT TERM
{
printf 'observed_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf 'observed_at=%s\n' "$run_started_at"
printf 'k6_image=%s\n' "$K6_IMAGE"
printf 'load_project=%s\n' "$LOAD_PROJECT"
printf 'web_replicas=%s\n' "$LOAD_WEB_REPLICAS"
@ -133,12 +264,19 @@ trap cleanup_sampler EXIT HUP INT TERM
printf 'websocket_hold_ms=%s\n' "$LOAD_WS_HOLD_MS"
printf 'websocket_connect_timeout_ms=%s\n' "$LOAD_WS_CONNECT_TIMEOUT_MS"
printf 'http_think_seconds=%s\n' "$LOAD_HTTP_THINK_SECONDS"
printf 'authenticated_vus=%s\n' "$LOAD_AUTH_VUS"
printf 'fixture_count=%s\n' "$fixture_count"
printf 'authenticated_websocket_timeout_ms=%s\n' "$LOAD_AUTH_WS_TIMEOUT_MS"
printf 'authenticated_think_seconds=%s\n' "$LOAD_AUTH_THINK_SECONDS"
docker info --format 'docker_cpus={{.NCPU}} docker_memory_bytes={{.MemTotal}} docker_server={{.ServerVersion}}'
docker compose version
uname -a
} >"$output_dir/environment.txt"
snapshot_database "$output_dir/database-before.txt"
snapshot_database "$output_dir/database-pre-fixtures.txt"
run_fixture_tool prepare >"$output_dir/fixture-prepare.log" 2>&1
fixtures_prepared=true
snapshot_database "$output_dir/database-before-load.txt"
sample_resources &
sampler_pid=$!
@ -147,14 +285,21 @@ docker run --rm \
--user "$(id -u):$(id -g)" \
--network "$network_id" \
--volume "$ROOT/load/k6:/scripts:ro" \
--volume "$output_dir:/fixtures:ro" \
--volume "$output_dir:/output" \
--env "BASE_URL=http://$LOAD_HOST" \
--env "BASE_URL=https://$LOAD_HOST" \
--env "PUBLIC_ORIGIN=https://$LOAD_HOST" \
--env "HTTP_VUS=$LOAD_HTTP_VUS" \
--env "WS_VUS=$LOAD_WS_VUS" \
--env "DURATION=$LOAD_DURATION" \
--env "WS_HOLD_MS=$LOAD_WS_HOLD_MS" \
--env "WS_CONNECT_TIMEOUT_MS=$LOAD_WS_CONNECT_TIMEOUT_MS" \
--env "HTTP_THINK_SECONDS=$LOAD_HTTP_THINK_SECONDS" \
--env "AUTH_VUS=$LOAD_AUTH_VUS" \
--env "AUTH_WS_TIMEOUT_MS=$LOAD_AUTH_WS_TIMEOUT_MS" \
--env "AUTH_THINK_SECONDS=$LOAD_AUTH_THINK_SECONDS" \
--env "FIXTURE_PASSWORD=$LOAD_FIXTURE_PASSWORD" \
--env "FIXTURE_RUN_ID=$LABEL" \
"$K6_IMAGE" run \
--no-usage-report \
--summary-mode=full \
@ -167,11 +312,11 @@ set -e
cleanup_sampler
sampler_pid=
trap - EXIT HUP INT TERM
snapshot_database "$output_dir/database-after.txt"
diff -u "$output_dir/database-before.txt" "$output_dir/database-after.txt" \
>"$output_dir/database-diff.txt" || true
snapshot_database "$output_dir/database-after-load.txt"
diff -u "$output_dir/database-before-load.txt" "$output_dir/database-after-load.txt" \
>"$output_dir/database-load-diff.txt" || true
validate_authenticated_writes
COMPOSE_PROJECT_NAME=$LOAD_PROJECT "$ROOT/scripts/verify-realtime-cluster.sh" compose \
>"$output_dir/pubsub-probe.txt"
@ -180,7 +325,8 @@ curl --fail --silent --show-error \
"http://localhost:$HTTP_PORT/healthz/ready" \
>"$output_dir/readiness-after.json"
"${compose[@]}" ps -a >"$output_dir/compose-after.txt"
"${compose[@]}" logs --since 10m proxy web worker >"$output_dir/application.log" 2>&1
"${compose[@]}" logs --since "$run_started_at" proxy web worker \
>"$output_dir/application.log" 2>&1
if ! jq -e '
def metric($name):
@ -193,7 +339,18 @@ if ! jq -e '
metric("wnh_websocket_opened").count > 0 and
metric("wnh_websocket_opened").count ==
metric("wnh_websocket_heartbeat_replies").count and
(metric("wnh_websocket_errors").count // 0) == 0
(metric("wnh_websocket_errors").count // 0) == 0 and
metric("wnh_authenticated_logins").count > 0 and
metric("wnh_authenticated_pages").count > 0 and
metric("wnh_tracking_updates").count > 0 and
metric("wnh_liveview_joins").count > 0 and
metric("wnh_liveview_joins").count ==
metric("wnh_liveview_tracking_starts").count and
metric("wnh_liveview_joins").count == metric("wnh_tracking_updates").count and
metric("wnh_liveview_joins").count == metric("wnh_liveview_messages").count and
metric("wnh_liveview_joins").count ==
metric("wnh_liveview_tracking_stops").count and
(metric("wnh_authenticated_errors").count // 0) == 0
' "$output_dir/k6-summary.json" >/dev/null; then
echo "Functional load checks failed; evidence is in $output_dir." >&2
exit 1
@ -204,4 +361,15 @@ if [[ "$k6_status" -ne 0 ]]; then
exit "$k6_status"
fi
run_fixture_tool cleanup >"$output_dir/fixture-cleanup.log" 2>&1
fixtures_prepared=false
snapshot_database "$output_dir/database-after-cleanup.txt"
if ! diff -u "$output_dir/database-pre-fixtures.txt" \
"$output_dir/database-after-cleanup.txt" >"$output_dir/database-cleanup-diff.txt"; then
echo "Fixture cleanup did not restore the pre-run table counts; evidence is in $output_dir." >&2
exit 1
fi
trap - EXIT HUP INT TERM
printf 'Load evidence: %s\n' "$output_dir"

2
scripts/load-stack-up.sh
View File

@ -34,6 +34,8 @@ case "$LOAD_WEB_REPLICAS:$LOAD_WORKER_REPLICAS" in
esac
cd "$ROOT"
docker build --target load_tools --tag who-need-help:load-tools .
docker compose \
--env-file "$ENV_FILE" \
-p "$LOAD_PROJECT" \