Warning, /swf-monitor/docs/PANDA_BOT.md is written in an unsupported language. File is not indexed.
0001 # DISpatcher Mattermost Bot
0002
0003 DISpatcher (formerly the PanDA bot) is the ePIC production Mattermost bot, an MCP
0004 client for production monitoring questions.
0005 The SWF Monitor MCP server overview is [MCP.md](MCP.md), client setup is
0006 [MCP_CLIENTS.md](MCP_CLIENTS.md), and the tool catalog is
0007 [MCP_TOOL_REFERENCE.md](MCP_TOOL_REFERENCE.md).
0008
0009 DISpatcher (`monitor_app/panda/bot.py`) is an MCP **client**. It answers production-monitoring questions in Mattermost by selecting and calling tools across multiple MCP servers.
0010
0011 **Architecture:**
0012 - Listens on a Mattermost channel via WebSocket (`mattermostdriver`)
0013 - Holds connections to the local swf-monitor MCP (HTTP — the `swf_*`, `pcs_*`, `panda_*` tools) plus stdio-launched external servers: **LXR** (EIC code browser cross-reference), **uproot** (ROOT file analysis), **GitHub**, **Zenodo**, **XRootD**, **JLab-Rucio**, **BNL-Rucio**, and **corun-ai/codoc**
0014 - The corun-ai MCP server is a standalone stdio server that calls prod corun-ai over REST (`CORUN_BASE_URL + /api/v1/...`). It queues generation jobs asynchronously; the bot exposes submit/generate/status/page tools but deliberately does not expose the long-polling `wait_for_job` tool, so a long corun-ai generation does not hold the bot's single response lock. Tool surface and config detail: [corun-mcp-server](https://github.com/eic/corun-mcp-server) README.
0015 - On startup, if `CORUN_API_TOKEN` is configured, the bot ensures a corun-ai notification subscription exists. corun-ai sends terminal job callbacks to `/swf-monitor/api/corun-callback/`; swf-monitor posts simple completion/failure/cancel notices for visible result pages to the fixed `#dispatcher` Mattermost channel. A callback carrying `result_page_ui_visible: false` still drives machine-side assessment enforcement but produces no Mattermost notice. There is no bot-side polling tracker. Callback payload schema: corun-ai `docs/job-system.md` § Job Notifications.
0016 - Registers in-process **epicdoc** tools (`epic_doc_search`, `epic_doc_contents`) backed by a ChromaDB vector store of ePIC docs — runs inside the bot process, not as a separate MCP server
0017 - **Bamboo** log analysis is used via the `panda_study_job` and `panda_harvester_workers` swf-monitor MCP tools, not as a separate MCP server
0018 - For JLab Rucio campaign dataset queries, the bot should search the `epic` scope first (for example `scope="epic", name="*26.04.1*", type="DATASET"`) and must not infer absence from a single empty scope or from an XRootD permission error
0019 - For Rucio dataset placement questions, replication rules are authoritative for managed RSE placement; replica/PFN listings may include transient staging endpoints and should not be summarized as persistent placement without checking rules
0020 - System prompt is externalized to a file and re-read per message, so prompt iteration doesn't require a bot restart
0021 - **3-tier tool awareness**: every tool is visible by name+one-liner in the system prompt so the LLM knows the full catalog; detailed schemas are fetched only for tools the LLM explicitly selects via `select_tools`; the bot preserves server and suggestion context across thread turns so follow-ups don't re-select from scratch
0022 - **Progressive tool loading via semantic similarity**: for each user question the bot embeds the question and ranks tools by server-prefixed cosine similarity, auto-truncating at a score cliff — the LLM sees a small, relevant set rather than all hundreds of tools
0023 - **DPID (Data Provenance ID) anti-fabrication**: for questions about specific jobs/tasks, the bot verifies the LLM cited a real DPID from tool output, strips the DPID from the user-facing reply, and warns if verification fails
0024 - Bot-created AI assessments are stamped before the MCP call: `username` is `bot`, `ai` is the exact Claude model used for the message, and `data.origin` includes `type: "bot"`, `client: "mattermost"`, `harness: "bot"`, and the model
0025 - Remembers recent Q&A exchanges (via `swf_record_ai_memory`) to improve responses over time. Memory is collective — the bot does not track or remember who asked what
0026 - `/panda` slash commands for direct queries without LLM involvement (status, errors, jobs/tasks by filter, site detail)
0027 - Server-side matplotlib plots rendered in Mattermost
0028
0029 **Running:** `manage.py panda_bot`
0030
0031 **Environment variables:**
0032 - `MATTERMOST_URL` (default: `chat.epic-eic.org`)
0033 - `MATTERMOST_TOKEN` (required)
0034 - `MATTERMOST_TEAM` (default: `main`)
0035 - `MATTERMOST_CHANNEL` (default: `dispatcher`)
0036 - `MCP_URL` (default: `http://127.0.0.1:8001/swf-monitor/mcp/`)
0037 - `ANTHROPIC_API_KEY` (required, used by the Anthropic SDK)
0038 - `CORUN_BASE_URL` (optional, default: `https://epic-devcloud.org/doc`)
0039 - `CORUN_API_TOKEN` (optional; when set, enables the corun-ai MCP server)
0040 - `CORUN_CALLBACK_URL` (optional, default: `https://pandaserver02.sdcc.bnl.gov/swf-monitor/api/corun-callback/`)
0041 - `CORUN_SUBSCRIPTION_NAME` (optional, default: `pandabot-swf-testbed`)
0042
0043 **Naming:** Renamed from PanDA bot to the ePIC-chosen DISpatcher (bot account
0044 `@dispatcher`, channel `#dispatcher`) on 2026-07-05. Internal identifiers keep
0045 the historical name for state continuity: `MEMORY_USERNAME='pandabot'` (AI
0046 memory), the `pandabot_active_threads` state key, `CORUN_SUBSCRIPTION_NAME`
0047 default `pandabot-swf-testbed`, and the `swf-panda-bot.service` systemd unit.
0048
0049 **#epicprod-live:** the bot is a passive member of the live action-stream
0050 channel. The publisher (`publish_epicprod_live`) posts events as the dedicated
0051 `epicprod` bot account; plain channel posts never wake DISpatcher — it engages
0052 only when @mentioned, typically in a thread under an event post, and follows up
0053 from the record link and subject carried in the post (recipe in the system
0054 prompt).
0055
0056 **MCP transport:** The bot uses a minimal HTTP POST client (`MCPClient`) that sends JSON-RPC requests to the local MCP endpoint. Each user question gets a fresh stateless request/response exchange.
