05 Records, deployment, security, tests and operations
- Status
- direction
- Conceptual layer
- ② Context store and Plant Box ops
- Repo layer
- L2
universal-repositary, L1 edge - Source
- architecture section 7.3, section 13 degraded modes · D20 · ADR 008 · Index: README.
1. L2 production tables#
One migration in universal-repositary/packages/migrate/sql/, extending the existing context and outcome records (telemetry.stop_event, features.process_batch, features.quality_status, features.shift_roster).
Table: 16 rows by table
| Table | Schema | One row per | Idempotent key |
|---|---|---|---|
fast_reading | telemetry | Batch of fast readings for one asset (compressed) | plant, asset, batch start, sequence |
billet_push | telemetry | Billet pushed | plant, line, push time, counter |
forged_part | telemetry | Part forged | plant, line, part time, counter |
part_link | features | Billet-to-part link with method and confidence | part |
part_flag | features | Flag or score on a part | part, flag, model version |
time_bin | features | Part and time window | plant, line, part, bin start |
ht_basket | features | Heat-treatment basket and its record | plant, furnace, quench time |
ht_test | features | Strength test linked to a charge | plant, charge, sample |
rejection_row | features | Register row with match status | plant, source ref, row hash |
twin_state | features | Twin checkpoint | plant, line, asset, event time |
write_log | ledger | Write request and result, hash-chained | writer, sequence |
follow_through | ledger | Outcome of each action or signal | action |
missed_savings | ledger | Expected loss and outcome | action |
param_row | baselines | Versioned parameter row | context, version |
param_promotion | baselines | Promotion record | context, version |
part_alias | graph | Raw part name to part key | plant, raw name |
Every insert is an upsert on its key. fast_reading is kept at least 13 months so a full-year replay is possible; records and the write log are never compacted by Stamped. Producers and consumers per table: 07.
2. Topics#
All under stamped/v1/{org}/{plant}/; the full registry with publishers, readers and ACLs is in 07.
| Topic | Broker | Content |
|---|---|---|
fast/{line}/{asset}/{signal} | Plant only | Fast readings |
twin/state/{line} | Plant only | Plant state and forecasts |
twin/write/request | Plant only, twin publishes | Write requests |
writer/write/result | Plant only, writer publishes | Results and refusals |
twin/heartbeat | Plant only, twin publishes | Heartbeat for the writer |
records/{type} | Plant to cloud, buffered | Record rows, versioned JSON Schema |
messages/out | Plant only, twin to L5 relay | Alerts, actions, signals |
control/in | Plant only, sync agent publishes | Verified config bundle from the cloud |
Schemas live in contracts/ with contract tests on both sides.
3. Deployment#
| Mode | Cloud | Plant Box | Allowed |
|---|---|---|---|
| Cloud only | Twin, L2–L6 | Edge agent | Replay; messages only if the link is reliable |
| Hybrid | L2–L6, scheduled L3, L4 | Edge agent, sync agent, broker, twin, writer, L5 relay | Mandatory from AL2 |
Plant BoxPlant-side computer for the fast loop (direction; D4) (D4): fanless industrial PC on a UPS, with an imaged spare on site. Images are signed; the twin is Nuitka-compiled; the writer and edge agent are Go binaries. No updates during a shift with writes on; the writer starts switched off after any update. L2 hosting follows D20.
4. Security#
Zones and conduits in the style of IEC 62443: plant OT (PLCs, SCADA) ↔ Stamped Plant Box (read conduit for readers; write conduit for the writer only, to allow-listed tags) ↔ Stamped cloud (outbound only from the plant). The writer has its own PLC credentials; broker ACLs let only the twin publish write requests and only the writer read them. Remote support is a time-limited outbound tunnel a plant person switches on. No secrets in git.
5. Monitoring targets#
Twin loop p99 under 100 ms; fast-tag freshness under 2 s; signals event-to-phone under 5 s; P1 alerts under 5 min; zero writer read-back failures; no heartbeat gap over 5 s; edge buffer drains within an hour of link return; rising register match rate; about 1 alert per person per hour.
6. Replay#
Months of plant history replay through the twin in shadow, on event time, from a harness in intelligence-evals. Output per line, role and threshold: forecast error, interval coverage, alerts, actions and signals per person per shift, flag shares, expected value per procedure.
7. Test strategy#
| Layer | Pass condition |
|---|---|
| Unit | Normal CI |
| Reference tests | Rewritten modules match analysis-script outputs within tolerance |
| Shadow replay | Budgets per role met; forecast and coverage targets met; no floods |
| Fault injection | Gaps, frozen values, restarts, lost heartbeat, read-back mismatch, operator override, clock jumps, broker restart, part change mid-run: no stale message; writer restores and stops |
Writer against plant-sim | Forbidden writes refused; limits hold; override wins; restore on stop |
| Contract tests | Producers and consumers agree |
| Template check | No hold, release, accept, reject or bypass wording |
No write path reaches a real PLC before its simulator and fault-injection tests pass.
8. Low-data lines#
Lines with only push and part counters run a low-data mode: stop and start detection, stop/Ready/first-part messages, and first-parts-after-a-stop flags. No heater twin, no temperature flags, no writes. Each line is its own site-pack entry with mode: low_data, so upgrading it later is configuration.
9. Runbooks#
| Situation | Automatic | Person |
|---|---|---|
| Plant box down | PLC watchdog drops the machine switch; operator values stand; no signals | Swap to the spare; writes stay off until re-enabled |
| Cloud link down | Plant box keeps running; signals still go out; records buffer | Check the buffer drains |
| PLC program changed | Writer sees the checksum or tag map change and switches off | Repeat site checks for affected tags |
| Register late | Matching waits; P3 to admin | Quality head sends it; thresholds stay on the last approved version |
| Model drift | Affected tag drops from AL3 to AL2 | Stamped team reviews the refit |
| Clock jump | Writer stops; state uncertain | Fix time sync; re-enable writes |
| Alert flood | One summary per person; P3 to admin | Admin checks for a broken tag; may shelve with reason |
Page history: last 3 changes
- docs(technical): rewrite fast-loop/; all architecture diagrams in house style
7330f47 - docs(fast-loop): interfaces and ownership; one topic registry; control-today wording; amend ADR-033 and ADR-036
df796e9 - docs(decisions): add ADR-033..038 (twin runtime, fast read path, plant-side writer, message classes, alerts and quality-to-lot link, part-keyed parameters), fast-loop technical set, rebuilt index with renumbering map; fix bare-number link text and ranges
22e2872