Harden launch E2E and record launch gates

This commit is contained in:
SimpleTest 2026-08-20 22:44:41 +03:00
parent 8e5073a7a5
commit 1f9c5d74d5
7 changed files with 148 additions and 22 deletions

View File

@ -200,10 +200,11 @@ block publishing changes, including Store Listing, Pricing, and Distribution.
- [ ] Tester feedback and fixes documented. - [ ] Tester feedback and fixes documented.
- [ ] Production-access questionnaire completed from actual evidence. - [ ] Production-access questionnaire completed from actual evidence.
On 2026-08-14 Play Console showed Closed testing locked until the one remaining On 2026-08-20 Play Console still showed Closed testing locked until the one
app-setup task, public contact details, is complete and reported `0 testers remaining app-setup task, public contact details, is complete and reported `0
currently opted-in`. No Closed or Production release was created during this testers currently opted-in`. The Console still required at least 12 opted-in
inspection. testers for at least 14 days. No Closed or Production release was created
during this inspection.
## Publishing ## Publishing

View File

@ -36,6 +36,21 @@ does not authorize saving fields in Play Console or publishing a release.
This was a read-only observation. No Console value, track, release, production This was a read-only observation. No Console value, track, release, production
deployment, frozen test deployment, or public Git remote was changed. deployment, frozen test deployment, or public Git remote was changed.
## Read-only recheck on 2026-08-20
- The Dashboard still reports **10 of 11 complete**.
- **Select an app category and provide contact details** remains the only
incomplete setup task. The category is **Social**. Public email, phone, and
website remain empty in Store settings.
- Closed testing remains locked until app setup is complete and still reports
`0 testers currently opted-in`.
- The production-access section currently requires a published closed-testing
release, at least 12 opted-in testers, and a closed test lasting at least 14
days.
This recheck was read-only. No Console field, release, track, deployment,
frozen test environment, or public Git remote was changed.
These are direct observations from the authenticated Play Console session on These are direct observations from the authenticated Play Console session on
that date. Recheck the Console before applying because its fields and policy that date. Recheck the Console before applying because its fields and policy
requirements can change. requirements can change.

View File

@ -121,6 +121,10 @@ The following decisions are intentionally not generated by code:
allocation, alerts, and any scaling thresholds. allocation, alerts, and any scaling thresholds.
- [ ] Backup ownership, encryption-key custody, off-site destination, restore - [ ] Backup ownership, encryption-key custody, off-site destination, restore
procedure, and measured recovery objectives have been approved. procedure, and measured recovery objectives have been approved.
- [x] The independent stale-backup alert uses the operator-selected maximum
age of 36 hours (`129600` seconds). This is an alert threshold for the
latest restore-verified heartbeat, not approval of retention, RPO, RTO,
capacity, or encryption-key custody.
Until the retention decision and tested executor exist, account-deletion cases Until the retention decision and tested executor exist, account-deletion cases
remain an audited operator workflow and automatic erasure stays disabled. The remain an audited operator workflow and automatic erasure stays disabled. The

View File

@ -6,12 +6,35 @@ import {
projectEmail, projectEmail,
} from "./helpers"; } from "./helpers";
async function submitSupportRequest(
page: import("@playwright/test").Page,
requesterEmail: string,
kind: "privacy_request" | "data_export",
subject: string,
details: string,
): Promise<void> {
await page.goto("/support");
await page.getByLabel("What do you need help with?").selectOption(kind);
await expect(page.getByLabel("Contact email")).toHaveValue(requesterEmail);
await expect(page.getByLabel("Contact email")).toHaveAttribute(
"readonly",
"",
);
await page.getByLabel("Subject").fill(subject);
await page.getByLabel("Describe the problem").fill(details);
await page.getByRole("button", { name: "Send support request" }).click();
await expect(page).toHaveURL(/\/support\/received\?reference=SUP-/);
await expect(
page.getByRole("heading", { name: "Support request created" }),
).toBeVisible();
}
test.skip( test.skip(
process.env.E2E_PRODUCTION_READ_ONLY !== "1", process.env.E2E_PRODUCTION_RUN_SCOPED !== "1",
"This read-only staff queue check requires the production run-scoped fixture", "This support intake and staff queue check requires the production run-scoped fixture",
); );
test("run-scoped support and legal fixtures are visible to production staff", async ({ test("run-scoped support intake and legal fixture reach production staff", async ({
browser, browser,
}, testInfo) => { }, testInfo) => {
const runID = process.env.E2E_RUN_ID; const runID = process.env.E2E_RUN_ID;
@ -27,13 +50,69 @@ test("run-scoped support and legal fixtures are visible to production staff", as
const supportSubject = `Production E2E support ${runID}`; const supportSubject = `Production E2E support ${runID}`;
const supportDetails = const supportDetails =
"Run-scoped read-only browser fixture for the production support queue."; "Run-scoped read-only browser fixture for the production support queue.";
const privacySubject = `Production E2E privacy ${runID}`;
const dataExportSubject = `Production E2E data export ${runID}`;
const accountDeletionSubject = "Delete my Who Need Help account";
const removalExplanation = const removalExplanation =
"Run-scoped read-only browser fixture for the production legal review queue."; "Run-scoped read-only browser fixture for the production legal review queue.";
const requester = await loginWithPassword(
browser,
requesterEmail,
fixturePassword,
);
const admin = await loginWithPassword(browser, adminEmail, fixturePassword); const admin = await loginWithPassword(browser, adminEmail, fixturePassword);
const assertRequesterClean = captureBrowserFailures(requester.page);
const assertAdminClean = captureBrowserFailures(admin.page); const assertAdminClean = captureBrowserFailures(admin.page);
await submitSupportRequest(
requester.page,
requesterEmail,
"privacy_request",
privacySubject,
"Run-scoped production browser verification for the authenticated privacy-request intake.",
);
await submitSupportRequest(
requester.page,
requesterEmail,
"data_export",
dataExportSubject,
"Run-scoped production browser verification for the authenticated data-export intake.",
);
await requester.page.goto("/account/delete");
await expect(requester.page.getByLabel("Account email")).toHaveValue(
requesterEmail,
);
await expect(requester.page.getByLabel("Account email")).toHaveAttribute(
"readonly",
"",
);
await requester.page
.getByLabel("Additional information")
.fill(
"Run-scoped production browser verification for the account-deletion support workflow.",
);
await requester.page
.getByRole("button", { name: "Request account deletion" })
.click();
await expect(requester.page).toHaveURL(/\/support\/received\?reference=SUP-/);
await gotoLiveView(admin.page, "/support/operations?queue=support"); await gotoLiveView(admin.page, "/support/operations?queue=support");
await admin.page
.locator("#support-case-filters")
.getByLabel("Search")
.fill(requesterEmail);
const supportRows = admin.page.locator("main tbody tr");
await expect(supportRows).toHaveCount(4);
await expect(supportRows.filter({ hasText: privacySubject })).toHaveCount(1);
await expect(supportRows.filter({ hasText: dataExportSubject })).toHaveCount(
1,
);
await expect(
supportRows.filter({ hasText: accountDeletionSubject }),
).toHaveCount(1);
await admin.page await admin.page
.locator("#support-case-filters") .locator("#support-case-filters")
.getByLabel("Search") .getByLabel("Search")
@ -62,6 +141,8 @@ test("run-scoped support and legal fixtures are visible to production staff", as
admin.page.getByText(removalExplanation, { exact: true }), admin.page.getByText(removalExplanation, { exact: true }),
).toBeVisible(); ).toBeVisible();
assertRequesterClean();
assertAdminClean(); assertAdminClean();
await requester.context.close();
await admin.context.close(); await admin.context.close();
}); });

View File

@ -498,8 +498,8 @@ defmodule Mix.Tasks.Wnh.StagingFullE2e do
defp validate_precreated_records!(manifest, support_request_ids, removal_notice_ids) do defp validate_precreated_records!(manifest, support_request_ids, removal_notice_ids) do
records = manifest["precreated_records"] records = manifest["precreated_records"]
unless support_request_ids == [records["support_request"]] and unless records["support_request"] in support_request_ids and
removal_notice_ids == [records["content_removal_notice"]] do records["content_removal_notice"] in removal_notice_ids do
Mix.raise("full staging E2E precreated records do not match the manifest") Mix.raise("full staging E2E precreated records do not match the manifest")
end end
end end

View File

@ -19,9 +19,9 @@ Usage:
./scripts/production-full-e2e.sh run [run-id] ./scripts/production-full-e2e.sh run [run-id]
The run creates only run-scoped synthetic users and records, exercises the The run creates only run-scoped synthetic users and records, exercises the
two-user help, moderated activity, and read-only staff support/legal browser two-user help, moderated activity, authenticated privacy/data/deletion intake,
flows, and removes the exact fixture on success, failure, or interrupt. It and read-only staff support/legal browser flows, and removes the exact fixture
never resets the database. on success, failure, or interrupt. It never resets the database.
EOF EOF
} }
@ -59,7 +59,7 @@ import shlex
import sys import sys
path = sys.argv[1] path = sys.argv[1]
wanted = { required = {
"COMPOSE_PROJECT_NAME", "COMPOSE_PROJECT_NAME",
"DATABASE_MODE", "DATABASE_MODE",
"DEPLOYMENT_ENV", "DEPLOYMENT_ENV",
@ -67,6 +67,8 @@ wanted = {
"POSTGRES_DB", "POSTGRES_DB",
"WNH_BASE_URL", "WNH_BASE_URL",
} }
optional = {"SUPPORT_OPERATOR_EMAIL_MODE"}
wanted = required | optional
values = {} values = {}
with open(path, encoding="utf-8") as handle: with open(path, encoding="utf-8") as handle:
for raw_line in handle: for raw_line in handle:
@ -79,12 +81,12 @@ with open(path, encoding="utf-8") as handle:
parsed = shlex.split(value, comments=False, posix=True) parsed = shlex.split(value, comments=False, posix=True)
values[key] = parsed[0] if parsed else "" values[key] = parsed[0] if parsed else ""
missing = sorted(key for key in wanted if not values.get(key)) missing = sorted(key for key in required if not values.get(key))
if missing: if missing:
raise SystemExit("Missing production identity settings: " + ", ".join(missing)) raise SystemExit("Missing production identity settings: " + ", ".join(missing))
for key in sorted(wanted): for key in sorted(wanted):
print(f"{key.lower()}={values[key]}") print(f"{key.lower()}={values.get(key, '')}")
PY PY
commit=$(git -C "$root" rev-parse HEAD) commit=$(git -C "$root" rev-parse HEAD)
@ -147,12 +149,15 @@ commit=$(value commit)
container=$(value container) container=$(value container)
database=$(value postgres_db) database=$(value postgres_db)
project=$(value compose_project_name) project=$(value compose_project_name)
support_operator_email_mode=$(value support_operator_email_mode)
if [[ "$(value deployment_env)" != production ]] || if [[ "$(value deployment_env)" != production ]] ||
[[ "$(value phx_host)" != whoneedhelp.com ]] || [[ "$(value phx_host)" != whoneedhelp.com ]] ||
[[ "$(value wnh_base_url)" != "$BASE_URL" ]] || [[ "$(value wnh_base_url)" != "$BASE_URL" ]] ||
[[ "$(value database_mode)" != external ]] || [[ "$(value database_mode)" != external ]] ||
[[ "$project" != who_need_help_production ]] || [[ "$project" != who_need_help_production ]] ||
[[ "$(value support_operator_email_mode)" != "" &&
"$(value support_operator_email_mode)" != disabled ]] ||
[[ "$(value health)" != healthy ]] || [[ "$(value health)" != healthy ]] ||
[[ "$(value fixture_prefix_count)" != 0 ]] || [[ "$(value fixture_prefix_count)" != 0 ]] ||
[[ ! "$commit" =~ ^[0-9a-f]{40}$ ]] || [[ ! "$commit" =~ ^[0-9a-f]{40}$ ]] ||
@ -195,14 +200,17 @@ Verified production target:
commit: $commit commit: $commit
app container: $container app container: $container
restart count: $(value restart_count) restart count: $(value restart_count)
support operator email mode: ${support_operator_email_mode:-disabled}
existing run-scoped fixture users: 0 existing run-scoped fixture users: 0
Exact temporary mutation scope: Exact temporary mutation scope:
- six confirmed synthetic users under wnh-staging-e2e-$RUN_ID-*; - six confirmed synthetic users under wnh-staging-e2e-$RUN_ID-*;
- their help requests, assignments, chats, positions, handover, reviews; - their help requests, assignments, chats, positions, handover, reviews;
- their activity, participation, group chat, report and category proposal; - their activity, participation, group chat, report and category proposal;
- one directly inserted support row and one directly inserted legal row, - one directly inserted support row plus three authenticated privacy,
read through the staff UI without submitting or moderating either record; data-export and account-deletion requests submitted through the public UI;
- one directly inserted legal row, read through the staff UI without
submitting or moderating a content-removal notice;
- their notifications, audit events and associated Oban jobs; - their notifications, audit events and associated Oban jobs;
- no database reset, migration, real-user role/status change, email delivery, - no database reset, migration, real-user role/status change, email delivery,
Caddy change, test-project change, payment, iOS, or KYC action. Caddy change, test-project change, payment, iOS, or KYC action.
@ -318,10 +326,10 @@ cleanup() {
! jq -e ' ! jq -e '
.cleanup_verified == true and .cleanup_verified == true and
(.cleanup_targets.users | length) >= 6 and (.cleanup_targets.users | length) >= 6 and
(.cleanup_targets.support_requests | length) == 1 and (.cleanup_targets.support_requests | length) >= 1 and
(.cleanup_targets.content_removal_notices | length) == 1 and (.cleanup_targets.content_removal_notices | length) == 1 and
(.cleanup_deleted_counts.users | type) == "number" and (.cleanup_deleted_counts.users | type) == "number" and
.cleanup_deleted_counts.support_requests == 1 and .cleanup_deleted_counts.support_requests == (.cleanup_targets.support_requests | length) and
.cleanup_deleted_counts.content_removal_notices == 1 and .cleanup_deleted_counts.content_removal_notices == 1 and
.cleanup_verified_at != null .cleanup_verified_at != null
' "$output_dir/fixture.json" >/dev/null; then ' "$output_dir/fixture.json" >/dev/null; then
@ -431,7 +439,7 @@ docker run --rm \
--env "E2E_RUN_ID=$RUN_ID" \ --env "E2E_RUN_ID=$RUN_ID" \
--env "E2E_FIXTURE_PASSWORD=$fixture_password" \ --env "E2E_FIXTURE_PASSWORD=$fixture_password" \
--env "E2E_ADMIN_EMAIL=wnh-staging-e2e-$RUN_ID-admin@example.invalid" \ --env "E2E_ADMIN_EMAIL=wnh-staging-e2e-$RUN_ID-admin@example.invalid" \
--env E2E_PRODUCTION_READ_ONLY=1 \ --env E2E_PRODUCTION_RUN_SCOPED=1 \
--env E2E_DETERMINISTIC_MAP_TILES=1 \ --env E2E_DETERMINISTIC_MAP_TILES=1 \
--env HOME=/tmp \ --env HOME=/tmp \
--volume "$output_dir/browser:/work/output" \ --volume "$output_dir/browser:/work/output" \

View File

@ -53,6 +53,22 @@ defmodule WhoNeedHelp.StagingFullE2ECleanupTest do
manifest["precreated_records"]["content_removal_notice"] manifest["precreated_records"]["content_removal_notice"]
) )
browser_created_support_request =
%WhoNeedHelp.Support.SupportRequest{
reference: "SUP-BROWSER-#{String.upcase(run_id)}",
requester_id: prepared_support_request.requester_id,
contact_verified_at: DateTime.utc_now(:second),
status: :open
}
|> WhoNeedHelp.Support.SupportRequest.submission_changeset(%{
"kind" => "data_export",
"contact_email" => prepared_support_request.contact_email,
"subject" => "Browser-created production E2E support #{run_id}",
"details" =>
"This run-owned support record proves cleanup accepts browser-created fixture data."
})
|> Repo.insert!()
assert manifest["schema_version"] == 2 assert manifest["schema_version"] == 2
assert prepared_support_request.subject == "Production E2E support #{run_id}" assert prepared_support_request.subject == "Production E2E support #{run_id}"
assert prepared_support_request.contact_verified_at assert prepared_support_request.contact_verified_at
@ -89,13 +105,14 @@ defmodule WhoNeedHelp.StagingFullE2ECleanupTest do
manifest = manifest_path |> File.read!() |> Jason.decode!() manifest = manifest_path |> File.read!() |> Jason.decode!()
cleaned_job_ids = manifest["cleanup_targets"]["push_jobs"] cleaned_job_ids = manifest["cleanup_targets"]["push_jobs"]
assert manifest["cleanup_targets"]["support_requests"] == [prepared_support_request.id] assert Enum.sort(manifest["cleanup_targets"]["support_requests"]) ==
Enum.sort([prepared_support_request.id, browser_created_support_request.id])
assert manifest["cleanup_targets"]["content_removal_notices"] == [ assert manifest["cleanup_targets"]["content_removal_notices"] == [
prepared_removal_notice.id prepared_removal_notice.id
] ]
assert manifest["cleanup_deleted_counts"]["support_requests"] == 1 assert manifest["cleanup_deleted_counts"]["support_requests"] == 2
assert manifest["cleanup_deleted_counts"]["content_removal_notices"] == 1 assert manifest["cleanup_deleted_counts"]["content_removal_notices"] == 1
assert support_job.id in cleaned_job_ids assert support_job.id in cleaned_job_ids
assert legal_job.id in cleaned_job_ids assert legal_job.id in cleaned_job_ids