Back to home page

EIC code displayed by LXR

 
 

    


Warning, /swf-testbed/docs/operations.md is written in an unsupported language. File is not indexed.

0001 # Operations Guide
0002 
0003 Running and managing the SWF Testbed services.
0004 
0005 ## Running the Testbed
0006 
0007 The testbed supports two deployment modes:
0008 
0009 **Development Mode**: Docker manages PostgreSQL and ActiveMQ
0010 **System Mode**: System services manage PostgreSQL and ActiveMQ (e.g., pandaserver02)
0011 
0012 Use `python report_system_status.py` to check which services are available and determine the appropriate mode.
0013 
0014 ### Development Mode (Docker-managed)
0015 
0016 This is the recommended approach as it provides a consistent, cross-platform
0017 environment.
0018 
0019 **Prerequisites:**
0020 
0021 - Docker Desktop installed and running.
0022 
0023 **Usage:**
0024 
0025 The `swf-testbed` CLI provides commands to manage the entire testbed, including
0026 the Docker containers and the Python agents managed by Supervisor.
0027 
0028 - `swf-testbed start`: Starts the Docker containers (PostgreSQL, ActiveMQ) and
0029   then starts all Python agents.
0030 - `swf-testbed stop`: Stops and removes the Docker containers, and stops all
0031   Python agents.
0032 - `swf-testbed status`: Shows the status of the Docker containers and the
0033   Python agents.
0034 
0035 ### System Mode (System-managed services)
0036 
0037 This mode is for environments where PostgreSQL and ActiveMQ are managed as system services.
0038 
0039 **Prerequisites:**
0040 
0041 You are responsible for installing and running PostgreSQL and ActiveMQ.
0042 
0043 - **PostgreSQL:**
0044   - **macOS (using Homebrew):**
0045 
0046     ```bash
0047     brew install postgresql
0048     brew services start postgresql
0049     # Optional: Create a user and database
0050     # createuser -s admin
0051     # createdb -O admin swfdb
0052     ```
0053 
0054   - **Other Systems:** Follow the official installation guide for your
0055     operating system.
0056 
0057 - **ActiveMQ:**
0058   - **macOS (using Homebrew):**
0059 
0060     ```bash
0061     brew install activemq
0062     brew services start activemq
0063     ```
0064 
0065   - **Other Systems:** Download and follow the installation instructions from the
0066     [ActiveMQ website](https://activemq.apache.org/components/classic/download/).
0067 
0068 **Usage:**
0069 
0070 The `swf-testbed` CLI also provides commands for managing the testbed services
0071 when they are running locally.
0072 
0073 - `swf-testbed start-local`: Starts the Python agents using Supervisor. It
0074   assumes PostgreSQL and ActiveMQ are already running as system services.
0075 - `swf-testbed stop-local`: Stops the Python agents managed by Supervisor.
0076 - `swf-testbed status-local`: Checks the status of system services (PostgreSQL,
0077   ActiveMQ) and the Python agents managed by Supervisor.
0078 - `python report_system_status.py`: **RECOMMENDED** - Comprehensive system readiness check
0079 
0080 ### Agent Process Management
0081 
0082 The testbed agents are managed by a process manager, which is
0083 responsible for configuring, starting, stopping, and monitoring the agents.
0084 
0085 We use `supervisor` to manage the various Python agent processes. The
0086 configuration is located in `supervisord.conf`. This file is a template and
0087 should be copied to the root of the project during initialization.
0088 
0089 The `swf-testbed init` command will create the `logs` directory and copy the
0090 `supervisord.conf` file for you.
0091 
0092 The `supervisord.conf` file is configured to use the `SWF_HOME` environment
0093 variable to locate the various `swf-*` repositories. This is automatically
0094 configured when you run any `swf-testbed` commands.
0095 
0096 ### Namespace Configuration
0097 
0098 Namespaces isolate workflows and messages, allowing multiple users or projects to share infrastructure without interference.
0099 
0100 **Configure your namespace** in `workflows/testbed.toml`:
0101 
0102 ```toml
0103 [testbed]
0104 namespace = "your-namespace"  # e.g., "epic-fastmon-dev", "team-dec29"
0105 ```
0106 
0107 **Namespace guidelines:**
0108 - Set namespace before running any workflows
0109 - Use descriptive names (project, team, or purpose)
0110 - Multiple users can collaborate in one namespace
0111 - Filter by namespace in the monitor UI to see only your data
0112 
0113 ### Message Broker
0114 
0115 The [ActiveMQ](https://activemq.apache.org/) message broker provides the
0116 messaging backbone for the testbed agents to communicate.
0117 
0118 #### Local Development Broker
0119 
0120 For local development and testing, a standalone broker can be run using Docker.
0121 This is managed by the `docker-compose.yml` file in this repository. To start
0122 the local broker, run:
0123 
0124 ```bash
0125 docker-compose up -d
0126 ```
0127 
0128 When using the local broker, the agents should be configured with the following
0129 environment variables:
0130 
0131 ```bash
0132 export ACTIVEMQ_HOST=localhost
0133 export ACTIVEMQ_PORT=61616
0134 export ACTIVEMQ_USER=admin
0135 export ACTIVEMQ_PASSWORD=admin
0136 ```
0137 
0138 #### System Mode Broker
0139 
0140 In system mode environments, the agents should be configured to use the system-managed
0141 ActiveMQ service. This is done by setting the same environment variables
0142  to point to the system broker's host, port, and credentials.
0143 
0144 Each agent (e.g., `swf-monitor`, `swf-data-agent`) will need to be configured
0145 to read these environment variables and use them to connect to the broker. The
0146 `swf-monitor` application, for example, reads these values from its Django
0147 `settings.py` file, which in turn can be populated from environment variables.
0148 
0149 ## Testing
0150 
0151 The testbed provides a unified, robust test runner to ensure all components are
0152 tested consistently and automatically.
0153 
0154 - To run all tests across all `swf-*` repositories, simply execute:
0155 
0156   ```bash
0157   ./run_all_tests.sh
0158   ```
0159 
0160   This script autodiscovers sibling `swf-*` repositories, runs their test
0161   suites, and reports results with clear separators.
0162 
0163 - To run tests for a specific repository, use its own `run_tests.sh` script
0164   (if present), e.g.:
0165 
0166   ```bash
0167   ./run_tests.sh
0168   ```