Warning, /swf-testbed/docs/development.md is written in an unsupported language. File is not indexed.
0001 # Development Guide
0002
0003 Development workflow and contribution guidelines for the SWF Testbed.
0004
0005 ## Development Workflow
0006
0007 ### Multi-Repository Development Strategy
0008
0009 The SWF testbed consists of multiple coordinated repositories that work together
0010 as an integrated system. Development across these repositories requires careful
0011 coordination to maintain system stability and integration.
0012
0013 #### Repository Structure
0014
0015 The testbed is composed of three core repositories that must be kept as siblings:
0016
0017 - **swf-testbed**: Core testbed infrastructure, CLI, and orchestration
0018 - **swf-monitor**: Django web application for monitoring and data management
0019 - **swf-common-lib**: Shared utilities and common code
0020
0021 Additional repositories will be added as the testbed expands with new agents,
0022 services, and functionality.
0023
0024 #### Branching Strategy
0025
0026 We use a **coordinated infrastructure branching strategy** for cross-repository
0027 development work:
0028
0029 ##### Infrastructure Development (Recommended)
0030
0031 For infrastructure improvements, testing framework enhancements, and foundational
0032 changes that span multiple repositories:
0033
0034 ```bash
0035 # Create synchronized infrastructure branches across all repos
0036 cd swf-testbed && git checkout -b infra/baseline-v1
0037 cd ../swf-monitor && git checkout -b infra/baseline-v1
0038 cd ../swf-common-lib && git checkout -b infra/baseline-v1
0039
0040 # CRITICAL: Push branches to origin immediately to make them available remotely
0041 cd swf-testbed && git push origin infra/baseline-v1
0042 cd ../swf-monitor && git push origin infra/baseline-v1
0043 cd ../swf-common-lib && git push origin infra/baseline-v1
0044
0045 # Work freely across repositories
0046 # Commit frequently with descriptive messages
0047 # Let commit messages document the nature and progression of changes
0048
0049 # When infrastructure phase is complete:
0050 # 1. Test full system integration with ./run_all_tests.sh
0051 # 2. Create coordinated pull requests across all repositories
0052 # 3. Merge to main simultaneously across all repos
0053 # 4. Start next infrastructure iteration (infra/baseline-v2)
0054 ```
0055
0056 ##### Feature Development
0057
0058 For features that primarily affect a single repository:
0059
0060 ```bash
0061 # Create feature branch in the primary repository
0062 git checkout -b feature/your-feature-name
0063
0064 # CRITICAL: Push branch to origin immediately to make it available remotely
0065 git push origin feature/your-feature-name
0066
0067 # Work, commit, and create pull request as normal
0068 # If cross-repo changes are needed, coordinate with infrastructure approach
0069 ```
0070
0071 #### Development Guidelines
0072
0073 1. **Never push directly to main** - Always use branches and pull requests
0074 2. **Push branches to origin immediately** - Always run `git push origin branch-name` right after creating a branch to make it available across all development machines
0075 3. **Coordinate cross-repo changes** - Use matching branch names for related work
0076 4. **Test system integration** - Run `./run_all_tests.sh` before merging infrastructure changes
0077 5. **Maintain test coverage** - As you add functionality, extend the tests to ensure `./run_all_tests.sh` reliably evaluates system integrity
0078 6. **Document through commits** - Use descriptive commit messages to explain the progression of work
0079 7. **Maintain sibling structure** - Keep all `swf-*` repositories as siblings in the same parent directory
0080
0081 #### Pull Request Process
0082
0083 1. **Create descriptive pull requests** with clear titles and descriptions
0084 2. **Reference related PRs** in other repositories when applicable
0085 3. **Ensure tests pass** across all affected repositories
0086 4. **Coordinate merge timing** for cross-repo infrastructure changes
0087 5. **Clean up branches** after successful merges
0088
0089 This workflow ensures that the testbed remains stable and integrated while
0090 allowing for rapid infrastructure development and feature additions.
0091
0092 ### Example Agent Implementations
0093
0094 For developers looking to create new agents or understand how to interact with
0095 the testbed's messaging and API services, standalone examples are provided in
0096 the `example_agents/` directory. These provide a clear, modern blueprint for
0097 agent development.
0098
0099 ## Participants
0100
0101 At present the testbed is a project of the Nuclear and Particle Physics
0102 Software (NPPS) group at BNL; collaborators are welcome.
0103
0104 - Torre Wenaus (lead)
0105 - Maxim Potekhin
0106 - Ejiro Umaka
0107 - Michel Villanueva
0108 - Xin Zhao
0109
0110 ## Agent Development Guidelines
0111
0112 ### Agent Identity and Naming
0113
0114 When developing new agents, follow the established identity management pattern:
0115
0116 **For agents inheriting from BaseAgent:**
0117 ```python
0118 # Agent name automatically assigned via get_next_agent_id()
0119 class MyAgent(BaseAgent):
0120 def __init__(self, debug=False):
0121 super().__init__(agent_type='MYTYPE', subscription_queue='/topic/epictopic', debug=debug)
0122 ```
0123
0124 **For standalone agents (like DAQ simulator):**
0125 ```python
0126 # Implement get_next_agent_id() method following the same pattern as BaseAgent
0127 def get_next_agent_id(self):
0128 """Get the next agent ID from persistent state API."""
0129 # Implementation matches BaseAgent.get_next_agent_id()
0130 ```
0131
0132 **Command Line Interface:**
0133 All agents should support `--debug` flag for verbose heartbeat logging:
0134 ```bash
0135 python my_agent.py --debug # Enable heartbeat logging
0136 python my_agent.py # Quiet mode (default)
0137 ```
0138
0139 ## Glossary
0140
0141 - STF: super time frame. A contiguous set of ~1000 TFs containing about ~0.6s
0142 of ePIC data, corresponding to ~2GB. The STF is the atomic unit of
0143 streaming data processing.
0144 - TF: time frame. Atomic unit of ePIC detector readout ~0.6ms in duration.
0145