Run modes and the KPI

Two things the word "mode" means

The platform uses "mode" in two unrelated places, and keeping them apart is the fastest way to understand the architecture. Run-modes are a deployment and operations concept: the same backend binary runs as several differently privileged workloads. Delivery modes are a trainee-experience concept: the shape of the exercise you actually sit down to work.

This page covers both, and then the operational KPI that the delivery modes feed. The two are linked, because the way a trainee is placed into a lab decides both the isolation the run needs and the moment the KPI clock starts.

Run-mode deployables: one binary, separate privileges

Under ADR-001 the backend `academy-api` is a modular monolith: one Go binary and one image, with bounded contexts enforced at the package level that talk only through in-process interfaces and never through cross-module database access. That single artefact is deployed as several Kubernetes workloads selected by a `run-mode` flag, each with its own workload identity, node placement and least-privilege scope. Privilege separation is achieved by identity and deployment topology rather than by maintaining a second codebase.

This keeps a small team productive while giving real blast-radius isolation. The internet-facing `api` pod holds no cloud-provisioning credentials at all, so a compromise there cannot mint range writes or read the custody-signing key, which belongs to acquisition. It does hold the object-store credential, because serving a lab's evidence to the trainee working it is its job, so evidence already captured is within reach of that pod and the isolation claim is about the range and the ledger rather than about the evidence store.

The range itself is treated as hostile, and the trust model is honest-asymmetric, not "zero trust into the range". The control plane reaches into the range only through a small set of sanctioned crossings, and the range never opens an inbound connection back to the control plane. See ADR-001 for the run-mode decision and the isolation model for the crossings.

• `api` - internet-facing and stateless; serves the SPA, enqueues signed provisioning intents, holds no range credentials.
• `provisioner` - runs on an isolated, tainted node pool and is the sole writer into the range; consumes only authorised queue intents (ADR-009).
• `collector` - read-only telemetry ingest; normalises range-native logs into stored, OCSF-shaped events.
• `acquisition` - assembles and seals evidence a run already captured, holding the custody-signing key; the WORM evidence vault is a production-path control not enabled here; it carries no range credential, so there is no range write path and no live read from the range either.
• `gateway` - the interactive-shell broker for live labs (ADR-031); a non-internet workload with its own identity that dials a pseudo-terminal into only the ticketed container. Reached by nobody unless the api's `LAB_SHELL` is on: the api mints no ticket while it is off (ADR-222).

Delivery modes: what a trainee sits down to

ADR-024 splits the training model into purpose-built delivery modes because their concurrency, isolation and cost profiles differ by orders of magnitude. Across all of them one rule holds: a run is a lease against a pool of standing, reset-between-runs environments, never a per-trainee cloud account or tenant (ADR-010).

From the trainee's seat, a lab takes one of three shapes.

These shapes map onto delivery contexts: self-paced individual labs (frozen or live), facilitated team purple-team exercises on one shared live range, and the closing public CTF event scheduled for February 2027. Team mode is primarily instructional and observational, because detection credit cannot be attributed cleanly to one analyst in a shared live range, so it is not treated as a per-trainee KPI source.

A facilitated session is scheduled with a category (SOC, forensics, threat hunting, CTF, attack simulation, incident response) and, if you want one, a time cap: a window in the calendar or a fixed length from the moment it opens (ADR-069). The cap bounds how long the shared range is held, so a ninety-minute exercise does not pay for two hours of environment. It does not end the exercise for you: the range is released around the cap, and you close the session when the room is finished with it.

• Frozen investigation - a per-trainee view over a pre-captured, read-only artefact set. Nothing detonates live; the evidence is already present and the work is to find it.
• Live lab - a per-trainee incident injection into a resource-group-scoped lease inside a pooled subscription. Two trainees on the same lab never share a detonation, which is what makes a per-trainee KPI clock valid. Interactive shell access, where a deployment turns it on, is brokered by the `gateway` run-mode.
• CTF - static challenges scored against pre-captured artefact sets, graded in-process with no live-range calls on the hot path (ADR-038); runs are individual and self-paced.

Grading from captured state, not keystrokes

ADR-025 fixes the grading hot path against telemetry that is already normalised and stored, or against the lab's frozen artefacts, rather than synchronous live-cloud reads. Two independent reasons drive this. Provider APIs such as Microsoft Graph, the Unified Audit Log and CloudTrail are rate-limited and would throttle under load; and cloud audit logs are eventually consistent, surfacing events minutes to hours later, which would produce false negatives indistinguishable from a wrong answer.

Grading credits the end state a trainee leaves behind, not their keystrokes. Three grading modes exist, in priority order.

• Static flag - a submitted token matches an expected value.
• Captured-telemetry assertion - the submission is checked against already-ingested, OCSF-normalised events or frozen artefacts, with no live cloud calls. This is the default hot path and the only mode used at CTF scale.
• Live range-state assertion - for example, confirming that a compromised session is now revoked. Reserved for low-concurrency self-paced labs, rate-limited, with an explicit timeout and a fallback verdict so a slow or throttled provider never blocks scoring.

Measuring MTTD and MTTR

ADR-026 redefines the funded KPI so it measures detection and response skill, not idle time or cloud-log latency. The clock starts per trainee at incident-injection or lab-start (`t0`), recorded by the `runs` state machine. A single shared-detonation clock would be invalid for a self-paced model, since a trainee who begins an hour after detonation would otherwise be scored as taking an hour to detect when they had simply not yet started.

For frozen labs the telemetry is pre-staged, which removes the ingestion lag from the measurement entirely. The trainee is scored on how fast they find evidence that is already present, not on how fast a provider's pipeline delivered a log.

"Platform-observed" is load-bearing. Detection is credited from a graded submission or an evidence-tagging action recorded by the platform, never from a trainee self-report, so timings are auditable and reproducible rather than gameable.

• MTTD = first correct platform-observed detection minus `t0`.
• MTTR = first correct platform-observed containment verification minus detection.

The honest ≥25% framing and the privacy line

Per-trainee measurements roll up into a documented paired pre/post cohort protocol: each trainee is measured on a baseline assessment before training and an equivalent assessment after, and the paired difference is the reported improvement.

The brief's target of a ≥25% MTTD improvement is reported as a measured programme KPI with a transparent, stated methodology and directional cohort results. It is not a statistically powered or statistically significant claim. At around 40 trainees across three to four pilot organisations the sample is not powered for significance, so the honesty sits in the claim rather than the method; the measurement machinery itself is sound.

Per-trainee MTTD and MTTR are personal performance data and are in scope for the GDPR. The platform records only the timestamps the definitions require, not keystroke or surveillance-grade behavioural telemetry; the `analytics` module reads them from the run row rather than owning a store of its own, and this data lives only in the mutable Postgres store so it stays erasable. It never enters the evidence store or the custody ledger, which reference pseudonymous actor identifiers only (ADR-023).