Warning, /epic-lfhcal-tbana/tests/smoke/README.md is written in an unsupported language. File is not indexed.
0001 # LFHCal baseline smoke tests
0002
0003 A quick, explicit check that the **already-built** LFHCal stack starts and
0004 performs a few known operations correctly. The same pattern can support small
0005 checks of other components as the software develops. Detailed numerical and
0006 scientific regressions complement these checks with broader validation of
0007 accuracy, detector behavior and analysis results.
0008
0009 ## One command after compilation
0010
0011 Run in the same ROOT/EIC environment used for the build, from the repository root:
0012
0013 ```sh
0014 python3 tests/smoke/run.py --build-dir NewStructure/build
0015 ```
0016
0017 For an in-source CMake build, use `--build-dir NewStructure`. No extra CMake option
0018 is needed. Each command has a 30-second limit; `--timeout 60` changes that limit.
0019 Any required missing artifact, failing command, timeout, or wrong numerical/data
0020 result returns a nonzero exit status. All fixture output is temporary and removed
0021 on success or failure. The suite does not silently skip a required check.
0022
0023 ### Checks in the baseline
0024
0025 1. Ten always-built executables exist and start with explicit `-h`, return zero,
0026 and print usage text: `Convert`, `DataPrep`, `CalibrateData`, `DataAna`,
0027 `Display`, `CompareCalib`, `CompareAna`, `CompareInjection`, `HGCROCStudy`
0028 and `CompareWaveform`. This exercises startup and shared-library discovery,
0029 not data processing.
0030 2. `root-config --version` runs. ROOT then explicitly loads the selected
0031 `libLFHCAL` with its generated dictionary artifacts. No ROOT startup macros
0032 or working-directory autoload are relied upon.
0033 3. The real `Calib` implementation calculates a known low-gain scale:
0034 high-gain scale 120 divided by correlation slope 4 gives 30, with absolute
0035 tolerance `1e-12`. This is a tiny deterministic numerical control, not a MIP fit.
0036 4. ROOT writes a real `Calib` object and a one-entry `TTree` containing a real
0037 `Tile`. A second ROOT process reopens the file without recovery, checks the
0038 object names, entry count and values, and repeats the calibration calculation.
0039
0040 The inputs are generated in code: one channel, one event, no random numbers and
0041 no beam-data dependency. `driver.C` propagates interpreter and payload failures
0042 as nonzero status; the Python runner also requires an explicit success marker.
0043 Checks do not use C++ `assert`, so release builds cannot disable them.
0044
0045 ### Optional decoder-enabled startup checks
0046
0047 When the pinned `NewStructure/h2g_decode` submodule was present during the build,
0048 explicitly request the decoder-enabled set:
0049
0050 ```sh
0051 python3 tests/smoke/run.py --build-dir NewStructure/build --decoder
0052 ```
0053
0054 This additionally requires and starts `HGCROC_Conversion`, `ParseCalibSamples`,
0055 `CompareHGCROCCalib`, `EvaluateRecoEffi` and the standalone decoder executable
0056 `h2g_decode/h2g_run`. Every program is invoked with `-h`; in particular, this
0057 avoids relying on the ineffective no-argument guards in three decoder tools.
0058 Missing requested decoder components fail the run. Without `--decoder`, the
0059 report explicitly says those optional checks were not requested.
0060
0061 ## Philosophy for component smoke tests
0062
0063 A useful smoke test answers a small, concrete question: can this component
0064 perform a representative operation through its real public interface?
0065
0066 - **Use a tiny deterministic fixture.** Generate a few known values or commit a
0067 small, documented input. Keep expected results understandable from the input;
0068 prefer directly specified values to random generation and require no remote data.
0069 - **Exercise the real component.** Call the production executable or public
0070 class interface. Mocks belong in tests of runner mechanics, not in evidence
0071 that the component itself works.
0072 - **Check an explicit result.** Choose one or a few meaningful outputs: a count,
0073 an ID, a calculated value or a round trip. Compare discrete values exactly;
0074 justify numerical tolerances from the calculation and precision involved.
0075 The current `120 / 4 = 30` calibration control is one example.
0076 - **Make failures actionable.** Name the stage and report the failed expectation
0077 with useful observed values. Checks must remain active in Release builds and
0078 propagate a nonzero exit status for wrong results, missing dependencies,
0079 interpreter errors and timeouts. A printed error alone is insufficient.
0080 - **Keep execution bounded and isolated.** Declare required build artifacts,
0081 use short per-command timeouts and temporary output directories, and clean up
0082 on success or failure. Use a fresh process when testing disk state or startup
0083 independently of retained in-memory objects. Running smoke tests should not
0084 install software, rebuild, download data or submit jobs.
0085 - **Keep stages identifiable.** Startup, decoding, calibration and serialization
0086 checks should report their own outcomes. A startup pass establishes readiness
0087 for the next stage; a conversion check must inspect converted results.
0088
0089 For example, a **future HGCROC conversion smoke case** could use a tiny complete
0090 capture with documented provenance, invoke the real conversion interface, then
0091 verify a known event count, event IDs and selected ADC values. This is a proposed
0092 extension, not a case shipped by this baseline. Broader stream coverage and
0093 scientific validation would remain in the detailed regression suite.
0094
0095 ## How to add a small check
0096
0097 Today this is **one explicitly wired baseline**, with no automatic macro or
0098 case discovery. `run.py` names the required executables, optional decoder set,
0099 library artifacts and `payload.C`, then runs the fixed `write` and `read` modes.
0100 `driver.C` loads the selected library and payload and calls the fixed
0101 `lfhcal_smoke_payload()` entry point. Adding a `.C` file to this directory does
0102 not make it run.
0103
0104 1. State the component operation, minimal input, expected result and tolerance
0105 before writing the check. Include the required artifacts and runtime budget.
0106 2. For a small check sharing the current calibration/serialization fixture, add
0107 a readable helper in `payload.C` and call it from the appropriate mode.
0108 Use release-active checks such as `smoke_require` and preserve failure
0109 propagation through the driver.
0110 3. For an independent component, add a clearly named case and explicitly wire
0111 its artifacts, invocation, timeout and success marker into `run.py`. A ROOT
0112 payload must follow the driver's fixed entry-point contract, or the driver
0113 dispatch must be deliberately extended. This requires code changes; there
0114 is no existing generic case registry to populate.
0115 4. Update `test_runner.py` for any new command sequence or failure path, then
0116 run both the runner tests and the real component check. Fake-command tests
0117 establish orchestration behavior; the real run establishes component behavior.
0118 5. Document a copyable run command, prerequisites and expected result here.
0119 Ensure the case is invoked by the existing runner or explicitly register its
0120 command in `.github/workflows/root-smoke.yml`; check that the workflow's path
0121 filters cover its files. Verify the real check in CI as well as locally.
0122
0123 Keep the next addition small enough to understand and diagnose on its own.
0124 A shared fixture is useful when it keeps checks simple; a new test framework is
0125 not a prerequisite for adding another component check.
0126
0127 ## Optional Yall readiness, not a Yall test campaign
0128
0129 If Yall is relevant to this environment, explicitly request only its version:
0130
0131 ```sh
0132 python3 tests/smoke/run.py --build-dir NewStructure/build --yall /path/to/yall-run
0133 ```
0134
0135 This does **not** validate a Yallfile, create/start a campaign, or contact Condor.
0136 Without `--yall`, the report says Yall readiness was not requested. A shell
0137 function is not an executable: if EIC setup exposes only the `yall-run` function,
0138 run `yall-run --version` separately in that activated shell. The runner does not
0139 source environment files or install another copy of Yall.
0140
0141 ## Build-tree versus installed artifacts
0142
0143 The current upstream CMake project has no `install()` rules. Its bootstrap builds
0144 in `NewStructure/build`; that is a post-build check, not proof of a relocatable
0145 installation. This suite does not introduce or claim a new install layout.
0146
0147 For an existing separately deployed layout, point at the actual artifacts rather
0148 than falling back to build-tree binaries:
0149
0150 ```sh
0151 python3 tests/smoke/run.py --bin-dir /deployment/bin \
0152 --library /deployment/lib/libLFHCAL.so
0153 ```
0154
0155 The library's directory must also contain nonempty `libLFHCAL_rdict.pcm` and
0156 `libLFHCAL.rootmap`. Use the test checkout matching those artifacts, because the
0157 ROOT payload includes its `NewStructure` class headers. This is not a standalone
0158 binary-only package or a test that build-tree headers are absent. The runner
0159 prints the selected binary/library/ROOT paths and does not change
0160 `LD_LIBRARY_PATH`. Within ROOT it adds the selected library directory to the
0161 dictionary autoload path. Activate the intended runtime environment first. `--root` and
0162 `--root-config` can select exact matching ROOT executables when needed.
0163
0164 ## Provenance and boundaries
0165
0166 The readiness checks are consolidated from the actual upstream
0167 `tools/bootstrap-yall-integration.sh`: ROOT version, executable presence and
0168 optional Yall availability. Here executable presence is strengthened to startup.
0169 The write/read pattern follows the existing `NewStructure/Tests/dummy.cc` and
0170 `dummyRead.cc` examples, but uses **current production classes**, checks the
0171 results, and does not reuse their committed binaries or old `Dummy.root` file.
0172 The numerical and checked production-object fixture are new coverage.
0173
0174 The complementary `tools/tests` suite exercises bootstrap/environment scripts;
0175 `examples/yall/check_shared_conversions.py` checks campaign graphs. The baseline
0176 runs explicitly after compilation or through CI; bootstrap and installation
0177 behavior are unchanged.
0178
0179 ## Maintainer checks of the runner
0180
0181 ```sh
0182 python3 -m unittest discover -s tests/smoke -p 'test_runner.py' -v
0183 python3 -m unittest discover -s tools/tests -v
0184 ```
0185
0186 The first command uses fake executables to test failure propagation, timeouts,
0187 path quoting, temporary-directory cleanup, optional-Yall behavior and the
0188 validation helper's error-detection guards (22 tests). Passing it is **not**
0189 evidence that the production ROOT smoke checks passed. Run the real one-command
0190 check in a compiled LFHCal environment as well.
0191
0192 ### Optional error-detection validation
0193
0194 Run the normal checks and then deliberately replace ROOT with the external
0195 `false` executable found on `PATH` to verify that a ROOT write failure is detected:
0196
0197 ```bash
0198 tests/smoke/validate.sh --build-dir NewStructure/build --timeout 60
0199 ```
0200
0201 The helper uses an executable path rather than a shell builtin, so the injection
0202 works on Linux and macOS. It labels the normal and deliberately failing phases
0203 clearly, captures only the injected run, and reports
0204 `PASS: expected error detected` only when the runner exits 1 with the exact
0205 expected ROOT-write rejection. A genuine baseline failure stops before injection
0206 and remains visible; unexpected success, another failure stage, a different exit
0207 status, or a timeout prints the injected log and returns nonzero. This offline
0208 injection checks the runner's error detection, not
0209 the behavior of real ROOT.
0210
0211 ## GitHub Actions
0212
0213 `ROOT smoke tests` builds all ten standard and five decoder-enabled targets,
0214 runs both runner modes, the unit tests and real ROOT checks on GitHub-hosted
0215 Ubuntu 24.04. It uses the official
0216 `rootproject/root:6.34.00-ubuntu24.04` image pinned by digest, with read-only
0217 repository permissions and the decoder revision recorded in the submodule.
0218 It runs for relevant pull-request changes, pushes to main, or manual dispatch.
0219 The workflow runs the explicit baseline command above. Contributors adding a
0220 case should keep its invocation and path coverage visible in that workflow.
0221 The separate `Yall examples` workflow checks workflow graphs and environment
0222 scripts; its result complements the ROOT build and smoke result.
0223
0224 ## Legend-lifetime regression
0225
0226 `python3 -m unittest discover -s tests/smoke -p 'test_legend.py' -v`
0227 compiles and runs the ROOT batch-mode regression for upstream PR #94. It checks
0228 that a legend does not retain a destroyed temporary TF1, preserves line style
0229 and label, and survives repaint over ten layers. Live sampled values and Npx
0230 are checked too; the legacy negative control must reject the dangling pointer.
0231
0232 This test depends on the production `AddMipFitLegendEntry` helper from #94.
0233 Until that fix is present, it reports **SKIP: not validated**, not a successful
0234 regression result. With the helper present, missing ROOT/compiler, build errors,
0235 wrong legacy results or repaint failures fail the test. The smoke CI discovers
0236 both runner and legend tests. No production implementation is copied here.
0237 For an integration checkout containing #94, set `LFHCAL_SOURCE_DIR` to that
0238 repository root while running this test; it compiles against that checkout's
0239 header with the current ROOT environment.