L3 — Rulepacks and evals
- Status
- as-built
- Conceptual layer
- ③ Analytics · governance: eval scorecards → L3 registry (D9)
- Repo layer
- L3
intelligence-rulepacks,intelligence-evals - Source
- architecture section 2.2 RP / EV, section 3.6.7
Rulepacks (intelligence-rulepacks)#
Filesystem catalog of versioned YAML rulepacks, vertical priors, and DISCOM HT tables. Not a runtime. Core loads them with RULEPACK_PATH pointing at the rulepacks repo root.
| Owns | Does not own |
|---|---|
Thresholds, formula ids, suppressions, ops_clearance defaults, vertical overlays | Detector Python; outbox; Lab UI |
rulepack://{pack}/{semver}#{rule_id} citations on Findings | Setting delivery / status (core owns dual-lane) |
Math runs in intelligence-core. Rule YAML is sheet music; engines are the orchestra.
Typical layout: domain/{pack}/{semver}/ · verticals/{id}/params.yaml · tariff tables · schemas/formula_registry.json · schemas/catalog_index.json.
Counts drift — prefer schemas/catalog_index.json in the rulepacks repo over copied numbers here.
Load order (as-built)#
- Domain pack for the engine’s
pack_id/rule_id - Optional vertical overlay (
VERTICAL_ID) - Shared suppressions + metric registry
Plant secrets stay outside the rulepacks repo.
Evals (intelligence-evals)#
Offline scores for L3 RunArtifacts plus an internal Lab UI. Observation only — no promote Lab → L4. Backtests write scorecards into the L3 registry (D9, RECOMMENDED).
| Surface | Job |
|---|---|
CLI stamped-l3-eval | Backtest goldens; gate check (precision_min in config/gates.yaml) |
| Lab UI | Triage candidates; L4 board = delivery=l4 ∧ status=emitted; Discovery keeps the rest |
Default corpus is checked-in RunArtifact 1.1.0 goldens — no live L2 required. Optional attach to core Lab export is secondary.
Split of duty#
| Piece | Role |
|---|---|
rulepacks | thresholds / citations |
core | engines / dual-lane / outbox |
evals | offline score / Lab triage |
Packs, procedures, parameters and replay (direction)#
| Item | Home | Note |
|---|---|---|
| Platform pack | intelligence-core twin/kit/ | Estimators, gate, Monte Carlo, calibration, replay, drift |
| Sector pack (forging first) | intelligence-rulepacks | Physics priors per alloy family; procedure catalog as versioned YAML; alarm rule templates |
| Site pack | Private site workspace, signed | Tag bindings, model cards, parameter rows, part aliases, plant-written limits, write allow-list (signed by the production head) |
| Parameter registry | L2 baselines.param_row · param_promotion | Context-keyed rows; promotions per ADR-011 |
| Replay harness | intelligence-evals | Leave-one-run-out replay; gates on error, alarm precision and recall, coverage, message budget |
| Reference tests | intelligence-core CI | Reproduce analysis-script outputs on recorded data within tolerances; synthetic fixtures in repo |
Detail: 05-twin-and-fast-loop.md sections 4–6.
Next: 04-finding-contract.md.
Page history: last 3 changes
- docs(technical): rewrite l3/ to the architecture
4a28bbd - docs(l3): twin runtime, twin-record engines, packs and replay as direction
28255e9 - docs(l3): document rulepacks, evals, and L4 intake
8013d9c