Warning, /swf-monitor/docs/CACHED_PRODUCTS.md is written in an unsupported language. File is not indexed.
0001 # Cached products — uniform long-build caching
0002
0003 A cached product is any expensive-to-build, read-often result: an
0004 aggregation over a large table, a remote-service rollup, a rendered
0005 fragment. This is the one pattern for serving them; new caching of a
0006 long build uses this mechanism rather than adding another hand-rolled
0007 variant.
0008
0009 ## Contract
0010
0011 - **A request always serves the stored product immediately**, stamped
0012 with its build time. Nothing expensive builds in the request path —
0013 the generalization of the no-remote-calls-in-render rule.
0014 - **Staleness rebuilds behind the response.** A request that finds the
0015 product older than its TTL returns it anyway and triggers one
0016 background rebuild; `building_since` on the store row is the
0017 cross-worker lock, so concurrent requests never stampede.
0018 - **Explicit update rebuilds synchronously.** The uniform Update button
0019 passes `refresh=1`; the user chose to wait, and gets fresh data back.
0020 - **The first-ever fill builds synchronously** — there is nothing to
0021 serve; it happens once per key.
0022 - **Failures surface.** A broken builder logs with its key and clears
0023 the lock; it never presents as silently stale data.
0024
0025 ## Mechanism
0026
0027 - Store: the `CachedProduct` row (`swf_cached_product`) — key, JSON
0028 value, `built_at`, `build_seconds`, `building_since`.
0029 - API: `monitor_app.cached_product.get_product(key, builder,
0030 ttl_seconds, refresh=False)` returning `{value, built_at,
0031 age_seconds, refreshing, built_now}`.
0032 - Executor: pure-database builders run in a background thread here.
0033 Credentialed or very heavy builds belong on the prod-ops agent with
0034 an SSE completion push (`swf-epicprod/docs/EPICPROD_OPS_AGENT.md`) —
0035 the agent is the heavy half of this same serve-cached-always pattern.
0036 - UI: DataTables pages get the freshness chip for free — return
0037 `create_response(..., extra={'product_built_at': ...,
0038 'product_age_seconds': ..., 'product_refreshing': ...})` and
0039 `_datatable_base.html` shows "Data as of HH:MM · Update". Non-table
0040 pages render the same fields from the `get_product` result.
0041
0042 ## Products on this mechanism
0043
0044 | Key | Builder | TTL |
0045 |---|---|---|
0046 | `panda_errors:v2:<days>:<user>:<site>:<source>` | PanDA error summary aggregation | 300 s |
0047 | `panda_tasks_window:<days>` | PanDA tasks list full-window aggregation | 120 s |
0048 | `prod_hub_corun_counts` | corun-ai assessment/narrative counts | 600 s |
0049
0050 ## Migration targets
0051
0052 Hand-rolled predecessors that should fold onto this mechanism as they
0053 are next touched: the catalog table fragment cache
0054 (`pcs.views.rebuild_current_task_list_html_cache`), the campaign
0055 progress snapshot refresh, and the per-request Rucio snapshot rollups.
0056 The Rucio snapshot fetch itself stays on the prod-ops agent (it is
0057 credentialed); its serve-side reads already follow the serve-cached
0058 contract.
