EIC code displayed by LXR master/​swf-monitor/​docs/​AI_PROPOSALS.md	Source navigation Diff markup Identifier search General search
	Version: master test Revert   Change

Warning, /swf-monitor/docs/AI_PROPOSALS.md is written in an unsupported language. File is not indexed.

 # epicprod AI Proposals
 
 AI proposals are how the LLM acts as a change agent in epicprod with a human
 in the loop: a proposer emits concrete actions, a human reviews and decides,
 and approval executes the change. Proposals make full use of the LLM for
 automation — not only assessment — while keeping every mutation under
 deterministic human control.
 
 ## The invariant: deterministic after authoring
 
 A proposal is an **executable payload**, not prose: the action identifier and
 the exact validated arguments of the service call it wants (for the pilot,
 `dataset_propagation_set` — state, `replaced_by`, comment). The LLM is
 consulted exactly once, at authoring time. From the moment the proposal
 exists it is frozen data plus existing code, passing three deterministic
 checkpoints:
 
 1. **Propose** — the payload is validated by the same validators the service
    uses. An unexecutable proposal is refused at birth; nothing reaches review
    that cannot run.
 2. **Decide** — preconditions are re-checked against current state
    (revalidation on touch, the `precondition` anchor). A record that changed
    since the proposal saw it is marked stale and withdrawn, never silently
    re-interpreted.
 3. **Execute** — the identical service call an operator makes by hand, with
    the approving human as `changed_by`. No LLM variance anywhere past
    checkpoint 1.
 
 AI surfaces get only the *propose* verb; the decide path demands an
 authenticated human. This is the credential-boundary pattern applied to
 authority: the AI surface holds no mutation right, structurally.
 
 ## The AI proposal list: the Proposal table is canonical
 
 The AI subsystem is its own Django app, `src/ai/` — a sibling of `pcs`,
 owning the proposal model, services, pages, and API, plus the AI
 assessment helpers and the corun-ai client (`ai/assessments.py`,
 `ai/corun_client.py`; the legacy `AIContent` model remains in
 `monitor_app` until its planned post-backfill retirement). Executors stay
 in their domain apps; `ai` dispatches to them.
 
 Every proposal is a `Proposal` row (`ai_proposal`): action, subject
 (type + key, `counterpart_key` for relation subjects such as a
 questionnaire–task match), frozen `payload`, required `comment`,
 `confidence`, proposer identity + scan version, `batch_id`, **executor**
 (`service` now; `agent_message` reserved for prod-ops-agent actions),
 `precondition` (the deterministic decide-time guard, capturing every prior
 value an undo must restore), input hash, status,
 and the decision record (`decided_by`, `decided_at`, optional `quality`,
 executed action-stream log id).
 
 Status vocabulary: `proposed → executed | denied | withdrawn | stale`, with
 `approved_pending_execution` reserved for asynchronous executors. Undoing
 an executed proposal returns it to `proposed` — pending again, decision
 fields cleared, render projection restored — while the origin-stamped
 execution and undo events carry the record. **Terminal rows are retained** — the AI proposal list is where AI-proposed activity
 and its decisions stay visible: the queue (pending), the decision history,
 per-proposer track records, and the standing metric of what fraction of
 production mutations originate with AI. Operators can delete list rows
 (test or noise entries), logged with counts — a cleanup verb, never a
 decision verb.
 
 Every proposal has a **ref** — its category prefix plus row id (`cp-123`
 for campaign propagation) — the one identifier humans use across surfaces:
 the web list's Ref column, the bot listing, and decide instructions. The
 prefix doubles as corroboration: a decide call whose prefix mismatches the
 row's category is refused, never reinterpreted, so a garbled or mis-relayed
 reference cannot decide the wrong proposal. Prefixes are declared per
 category in `ACTION_REF_PREFIXES` (`ai/models.py`).
 
 Records a proposal targets carry a **render projection** in their metadata
 (`Dataset.metadata['proposal']`), written and cleared in the same
 transactions by the same services — so the catalog badges, filters, and
 compose detail render without joins while the table holds truth. Denial
 memory is a proposal-list query (denied row with the same subject and input
 hash), not record state.
 
 ## Adding a proposal category
 
 Every proposal category completes the same checklist, in order; the
 propagation pilot is the worked example of each item.
 
 1. **Executor first.** A validated service in the domain app: one call is
    one operator action, it accepts `origin`, logs one origin-stamped
    action-stream event (action id registered with sublevel and live
    defaults), and returns the event's log id. Humans and approvals make
    the identical call — without the executor there is nothing to propose.
 2. **Precondition.** The deterministic decide-time anchor: the fields
    that prove the record has not moved, including every prior value an
    undo must restore (for propagation: `prev_state`, `prev_replaced_by`).
 3. **Propose service.** Validates the payload with the executor's own
    validators (an unexecutable proposal is refused at birth), captures
    the precondition, computes the input hash, honors denial memory, and
    writes the Proposal rows and the render projection.
 4. **Decide dispatch.** The action's revalidation rule and executor call
    registered in the decide path, and the category's ref prefix declared
    in `ACTION_REF_PREFIXES`. (The pilot hard-wires `proposal_decide` to
    the propagation action; the second category generalizes this into a
    per-action registry.)
 5. **Comment template.** Defined per action: code fills the facts, the
    model fills only the judgment fragment — or nothing.
 6. **Review surfaces.** The render projection badge and an AI-proposed
    filter on the domain list page in the `.ai-attr` treatment, with
    approve/deny in its bulk bar. The proposals page picks the rows up
    without further work.
 7. **Reversibility class.** Declared undoable, mitigable, or
    irreversible-by-design; review ceremony follows the class, and the
    approve control states its undo story.
 8. **Scan lifecycle.** Proposer identity, scan version, batch id;
    heartbeat refresh semantics if the proposer recurs; the per-action
    SysConfig off-switch, present in the SysConfig document at its
    default.
 9. **Weakest consumer.** If bots or small models read or relay the
    category, the retrieval surface is sized for the smallest model.
 10. **Documentation.** The category added to this document: its
     precondition, comment template, and reversibility class.
 
 ## Review surfaces
 
 Proposals are **visible but inert** on the open face: anyone can see a
 recommendation; only an authenticated approver can act. Everything
 AI-attributable renders in the universal AI presence scheme — purple ink
 on lavender ground, defined once (`--ai-fg`, `--ai-bg`, `--ai-border`,
 `.ai-attr` / `.ai-attr-text` / `.ai-fill`) in the base template. Used with
 restraint: it marks AI material embedded in non-AI surfaces and the titles
 of AI pages, so the color registers as "this is AI" without saturating.
 The marker follows artifact kind, not authorship: proposals and
 assessments are inherently AI products and are marked; artifacts of a
 kind humans equally produce — narratives, documents, code — dress plain,
 whoever drafted them.
 
 - **The catalog list** — the origin surface, a first-class decision site:
   proposals sit purple-marked in the working material. AI filters isolate
   proposed and executed rows; pending rows badge the proposed transition
   and executed rows carry an executed mark; Approve AI / Deny AI / Undo AI
   in the bulk bar act on the selection, with the quality selector recorded
   on the decision.
 - **The compose detail**: the same proposal block with the same controls —
   the record is server-side, so list and detail cannot fork.
 - **The proposals page** (`/ai/proposals/`, on the production home under
   AI assessments): the pending queue and the decision history, filterable by
   status, action, proposer, and batch, with bulk decide and per-proposer
   track records.
 - **The bot surface** (MCP: `ai_list_proposals`, `ai_decide_proposal`): one
   flat list over every category — refs plus server-rendered description
   lines the bot shows verbatim — and a decide relay carrying the human's
   ref, verdict, and identity. The relaying LLM contributes no
   interpretation; deciders must be on the SysConfig approver list
   (`ai_proposal_mcp_approvers`, default empty), and the executed event's
   filter field records the relaying surface (`mcp:<ref>`).
 
 A decision optionally carries a **quality tag** from the shared review
 vocabulary (`wrong | poor | ok | good` — the same scale as AI assessments):
 set the tag in the quality selector, then approve or deny, and it is
 recorded with the decision. A deny tagged `wrong` records a miscalibrated
 proposal and weighs against the proposer's track record; a plain deny
 records a sound proposal decided against.
 
 Execution events and history entries are **origin-stamped**
 (`origin: ai_proposal`, proposer, scan version, batch, proposed-at)
 alongside the approving `username`, so every mutation is classifiable as
 human-originated or AI-proposed/human-approved.
 
 ## Lifecycle: the scan heartbeat
 
 Recurring proposers refresh their pending proposals: each scan withdraws
 its unacted proposals and re-derives from current inputs — what still holds
 returns freshly validated, what no longer holds does not return, new
 findings appear. Pending staleness is bounded by the scan cadence, an
 unreviewed proposal returns at each scan until decided, and denial is the
 only stop (a denied input hash is never re-proposed until the proposer's
 inputs change).
 Withdrawals are logged and counted, never silent. One-shot proposal sets
 persist until decided; revalidation on touch covers their staleness.
 Per-action-type proposing is switchable off in SysConfig.
 
 ## Proposers
 
 The subsystem is **proposer-agnostic**: rule-based detectors, LLMs, and
 hybrids are all legitimate proposers — a deterministic rule ("this family's
 EVGEN inputs vanished from Rucio → propose hold") emits through the same
 surface as an LLM scan.
 
 **Comments are templated, not freeform.** Each proposer's harness defines a
 per-action comment template: code fills the facts (what changes, counts,
 replacement target, source and date), the model fills only the judgment
 fragment — some proposers fill nothing. Facts in comments therefore cannot
 be hallucinated, and the human reviews a uniform comment shape per action
 type. The questionnaire automatch is the worked hybrid example: structured
 frame plus the model's one-line reason.
 
 **Weakest-consumer design.** The propose surface is one flat call with
 deterministic validation, so a small-model proposer's worst case is a
 denied proposal. The proposal list's
 per-proposer approval and wrong-rates make proposal rights earnable and
 revocable by data: a proposer whose wrong-rate climbs gets its proposing
 switched off. LLM *readers* of proposal state (bots, assessors) get a
 retrieval tool sized for the smallest model: summarize-first, prescriptive
 docstring.
 
 ## Ceremony proportional to stakes
 
 Review friction matches irreversibility, in both directions:
 
 - **Undoable** actions (catalog state: propagation, tags, matches) approve
   in one tap, and the system offers **Undo** — a computed compensating
   action with full provenance (`origin: undo` carrying the proposal id),
   never erasure; history stays append-only. The undo offer expires when
   the record moves on, by the same precondition machinery. Propagation is
   the worked example: Undo on an executed proposal restores the
   precondition state — including the prior `replaced_by` — through the
   same executor, with a templated comment naming the proposal, and
   returns the proposal to the pending queue.
 - **Mitigable** actions (external effects: a submitted PanDA task can be
   killed or superseded by `.tryN`, not unsubmitted) offer the named
   mitigation, labeled honestly: it does not restore the prior world.
 - **Irreversible-by-design** actions (locking a tag, publishing a
   narrative revision) are labeled as such — their immutability is a
   guarantee other records depend on — and get ceremony: dry-run preview
   before approve.
 
 Every approve control states its undo story. Reversibility class is
 declared per action type in the same registry family as sublevel and live
 defaults.
 
 **Dry run** is the trust instrument for consequential and composite
 actions: execute the frozen payload with writes off and show the concrete
 before→after; approve then executes what was just shown. Dry runs write
 nothing, so they are open to all readers, not only approvers. Composite
 executors (campaign creation) must be
 dry-runnable; a composite proposal's review surface is its dry run, diffed
 against the preview frozen at propose time — staleness made visible at a
 scale a single `prev_state` anchor cannot guard.
 
 ## Growth path
 
 Designed-for extensions, reserved now, built when their consumer arrives:
 
 - **Asynchronous executors** (`executor: agent_message`): approval queues
   the payload to the prod-ops agent; the proposal holds
   `approved_pending_execution` until the completion event closes it.
 - **Recordless subjects**: system-scoped proposals ("re-run catalog_sync")
   carry `subject_type: system` and no render projection — the queue page is
   their only surface.
 - **Creation subjects**: proposals that create records ("compose a draft
   task from request #202") anchor on idempotency keys rather than
   `prev_state`; the intake surface already provides them.
 - **Second consumer, first adaptation**: questionnaire–task matching — the
   automatch's `suggested` matches land as proposals with confidence
   (relation subjects, `counterpart_key`), high/medium confidence keeps
   auto-accepting with origin stamps: the graduation ladder
   (`manual → auto+notify`) already in production.
 - **Beyond epicprod**: the subsystem is system-wide — testbed and
   streaming actions are expected consumers. Proposal events currently log
   to the epicprod action stream; where non-epicprod proposals log is
   decided when the first such consumer arrives.
 
 ## Action-stream records
 
 | action | when |
 |---|---|
 | `proposal_created` | one per propose call, with count and batch |
 | `proposal_denied` | one per deny decision, with count and quality |
 | `proposal_expired` | one per heartbeat withdrawal, with count |
 | (execution) | the executed action's own event, origin-stamped |
 | (undo) | the compensating action's own event, `origin: undo` with the proposal id |
 
 ## Related
 
 - [PCS.md](https://github.com/BNLNPPS/swf-epicprod/blob/main/docs/PCS.md) — the propagation disposition model the pilot proposes on.
 - [ACTION_STREAM.md](ACTION_STREAM.md) — the event record.
 - [EPICPROD_NARRATIVES.md](https://github.com/BNLNPPS/swf-epicprod/blob/main/docs/EPICPROD_NARRATIVES.md) — the context proposers
   reason against.
 - [EPICPROD_TASK_CATALOG.md](https://github.com/BNLNPPS/swf-epicprod/blob/main/docs/EPICPROD_TASK_CATALOG.md) — the review surface.
 - [EPICPROD_OPS.md](https://github.com/BNLNPPS/swf-epicprod/blob/main/docs/EPICPROD_OPS.md) — the shared review-quality vocabulary.

This page was automatically generated by the 2.3.7 LXR engine. The LXR team