Warning, /swf-monitor/docs/FASTMON_FILES.md is written in an unsupported language. File is not indexed.
0001 # FastMon Files (Time Frames) Monitoring
0002
0003 ## Overview
0004
0005 FastMon Files represent Time Frame (TF) files that are subsampled from Super Time Frame (STF) files for rapid monitoring and quality assessment. These smaller data samples enable real-time monitoring of data quality without processing entire STF files.
0006
0007 ## Purpose
0008
0009 The FastMon system provides:
0010 - **Rapid Quality Assessment**: Quick validation of detector data through sampled Time Frames
0011 - **Real-time Monitoring**: Immediate feedback on data collection quality
0012 - **Efficient Processing**: Analyze representative samples without full STF processing overhead
0013 - **Early Problem Detection**: Identify issues during data taking rather than post-run
0014
0015 ## Data Flow
0016
0017 1. **STF Generation**: DAQ simulator creates Super Time Frame files during physics data taking
0018 2. **TF Sampling**: FastMon agent samples Time Frames from each STF (typically 7 TFs per STF at 15% size fraction)
0019 3. **Registration**: TF files are registered in the FastMonFile database table
0020 4. **Broadcasting**: TF availability is broadcast via ActiveMQ for monitoring clients
0021 5. **Analysis**: Fast monitoring clients process TFs for quality metrics
0022
0023 ## Database Schema
0024
0025 ### FastMonFile Model
0026 - `tf_file_id`: Unique identifier for the Time Frame file
0027 - `stf_file`: Reference to parent STF file
0028 - `tf_filename`: Unique filename following pattern: `{stf_base}_tf_{sequence}.tf`
0029 - `file_size_bytes`: Size of the TF file
0030 - `tf_first`: Index of the first Time Frame in the sample
0031 - `tf_last`: Index of the last Time Frame in the sample
0032 - `tf_count`: Number of Time Frames in the sample (typically ~15% of `StfFile.tf_count`)
0033 - `status`: Processing status (registered, processing, processed, failed, done)
0034 - `metadata`: JSON field containing:
0035 - Simulation parameters
0036 - Creation timestamp
0037 - Agent information
0038 - Machine state and substate
0039 - Timing information
0040
0041 ## Web Interface
0042
0043 The FastMon Files view provides:
0044
0045 ### Filtering Options
0046 - **By Status**: Filter TFs by processing status
0047 - **By Parent STF**: View all TFs from a specific STF file
0048 - **By Run Number**: View all TFs from a specific run
0049
0050 ### Table Columns
0051 - TF Filename
0052 - Parent STF File (clickable to STF detail page)
0053 - Run Number (clickable to run detail page)
0054 - File Size
0055 - Status
0056 - Creation Time
0057
0058 ### Features
0059 - Server-side pagination for large datasets
0060 - Global search across filenames and run numbers
0061 - Quick filter links for easy navigation
0062 - Export capabilities for offline analysis
0063
0064 ## API Endpoints
0065
0066 ### REST API
0067 - `GET /api/fastmon-files/` - List all TF files with filtering
0068 - `POST /api/fastmon-files/` - Register new TF file
0069 - `GET /api/fastmon-files/{id}/` - Get specific TF file details
0070 - `PATCH /api/fastmon-files/{id}/` - Update TF file status
0071
0072 ### DataTables AJAX
0073 - `/fastmon-files/datatable/` - Server-side processing endpoint for web interface
0074
0075 ## Monitoring Metrics
0076
0077 The system tracks:
0078 - Total TF files generated per run
0079 - TF processing success rate
0080 - Average TF file size
0081 - Time from STF creation to TF availability
0082 - Distribution of TFs across machine states
