Back to home page

EIC code displayed by LXR

 
 

    


Warning, /epic-lfhcal-tbana/tests/smoke/README.md is written in an unsupported language. File is not indexed.

0001 # LFHCal baseline smoke tests
0002 
0003 A quick, explicit check that the **already-built** LFHCal stack starts and
0004 performs a few known operations correctly. The same pattern can support small
0005 checks of other components as the software develops. Detailed numerical and
0006 scientific regressions complement these checks with broader validation of
0007 accuracy, detector behavior and analysis results.
0008 
0009 ## One command after compilation
0010 
0011 Run in the same ROOT/EIC environment used for the build, from the repository root:
0012 
0013 ```sh
0014 python3 tests/smoke/run.py --build-dir NewStructure/build
0015 ```
0016 
0017 For an in-source CMake build, use `--build-dir NewStructure`. No extra CMake option
0018 is needed. Each command has a 30-second limit; `--timeout 60` changes that limit.
0019 Any required missing artifact, failing command, timeout, or wrong numerical/data
0020 result returns a nonzero exit status. All fixture output is temporary and removed
0021 on success or failure. The suite does not silently skip a required check.
0022 
0023 ### Checks in the baseline
0024 
0025 1. Ten always-built executables exist and start with explicit `-h`, return zero,
0026    and print usage text: `Convert`, `DataPrep`, `CalibrateData`, `DataAna`,
0027    `Display`, `CompareCalib`, `CompareAna`, `CompareInjection`, `HGCROCStudy`
0028    and `CompareWaveform`. This exercises startup and shared-library discovery,
0029    not data processing.
0030 2. `root-config --version` runs. ROOT then explicitly loads the selected
0031    `libLFHCAL` with its generated dictionary artifacts. No ROOT startup macros
0032    or working-directory autoload are relied upon.
0033 3. The real `Calib` implementation calculates a known low-gain scale:
0034    high-gain scale 120 divided by correlation slope 4 gives 30, with absolute
0035    tolerance `1e-12`. This is a tiny deterministic numerical control, not a MIP fit.
0036 4. ROOT writes a real `Calib` object and a one-entry `TTree` containing a real
0037    `Tile`. A second ROOT process reopens the file without recovery, checks the
0038    object names, entry count and values, and repeats the calibration calculation.
0039 
0040 The inputs are generated in code: one channel, one event, no random numbers and
0041 no beam-data dependency. `driver.C` propagates interpreter and payload failures
0042 as nonzero status; the Python runner also requires an explicit success marker.
0043 Checks do not use C++ `assert`, so release builds cannot disable them.
0044 
0045 ### Optional decoder-enabled startup checks
0046 
0047 When the pinned `NewStructure/h2g_decode` submodule was present during the build,
0048 explicitly request the decoder-enabled set:
0049 
0050 ```sh
0051 python3 tests/smoke/run.py --build-dir NewStructure/build --decoder
0052 ```
0053 
0054 This additionally requires and starts `HGCROC_Conversion`, `ParseCalibSamples`,
0055 `CompareHGCROCCalib`, `EvaluateRecoEffi` and the standalone decoder executable
0056 `h2g_decode/h2g_run`. Every program is invoked with `-h`; in particular, this
0057 avoids relying on the ineffective no-argument guards in three decoder tools.
0058 Missing requested decoder components fail the run. Without `--decoder`, the
0059 report explicitly says those optional checks were not requested.
0060 
0061 ## Philosophy for component smoke tests
0062 
0063 A useful smoke test answers a small, concrete question: can this component
0064 perform a representative operation through its real public interface?
0065 
0066 - **Use a tiny deterministic fixture.** Generate a few known values or commit a
0067   small, documented input. Keep expected results understandable from the input;
0068   prefer directly specified values to random generation and require no remote data.
0069 - **Exercise the real component.** Call the production executable or public
0070   class interface. Mocks belong in tests of runner mechanics, not in evidence
0071   that the component itself works.
0072 - **Check an explicit result.** Choose one or a few meaningful outputs: a count,
0073   an ID, a calculated value or a round trip. Compare discrete values exactly;
0074   justify numerical tolerances from the calculation and precision involved.
0075   The current `120 / 4 = 30` calibration control is one example.
0076 - **Make failures actionable.** Name the stage and report the failed expectation
0077   with useful observed values. Checks must remain active in Release builds and
0078   propagate a nonzero exit status for wrong results, missing dependencies,
0079   interpreter errors and timeouts. A printed error alone is insufficient.
0080 - **Keep execution bounded and isolated.** Declare required build artifacts,
0081   use short per-command timeouts and temporary output directories, and clean up
0082   on success or failure. Use a fresh process when testing disk state or startup
0083   independently of retained in-memory objects. Running smoke tests should not
0084   install software, rebuild, download data or submit jobs.
0085 - **Keep stages identifiable.** Startup, decoding, calibration and serialization
0086   checks should report their own outcomes. A startup pass establishes readiness
0087   for the next stage; a conversion check must inspect converted results.
0088 
0089 For example, a **future HGCROC conversion smoke case** could use a tiny complete
0090 capture with documented provenance, invoke the real conversion interface, then
0091 verify a known event count, event IDs and selected ADC values. This is a proposed
0092 extension, not a case shipped by this baseline. Broader stream coverage and
0093 scientific validation would remain in the detailed regression suite.
0094 
0095 ## How to add a small check
0096 
0097 Today this is **one explicitly wired baseline**, with no automatic macro or
0098 case discovery. `run.py` names the required executables, optional decoder set,
0099 library artifacts and `payload.C`, then runs the fixed `write` and `read` modes.
0100 `driver.C` loads the selected library and payload and calls the fixed
0101 `lfhcal_smoke_payload()` entry point. Adding a `.C` file to this directory does
0102 not make it run.
0103 
0104 1. State the component operation, minimal input, expected result and tolerance
0105    before writing the check. Include the required artifacts and runtime budget.
0106 2. For a small check sharing the current calibration/serialization fixture, add
0107    a readable helper in `payload.C` and call it from the appropriate mode.
0108    Use release-active checks such as `smoke_require` and preserve failure
0109    propagation through the driver.
0110 3. For an independent component, add a clearly named case and explicitly wire
0111    its artifacts, invocation, timeout and success marker into `run.py`. A ROOT
0112    payload must follow the driver's fixed entry-point contract, or the driver
0113    dispatch must be deliberately extended. This requires code changes; there
0114    is no existing generic case registry to populate.
0115 4. Update `test_runner.py` for any new command sequence or failure path, then
0116    run both the runner tests and the real component check. Fake-command tests
0117    establish orchestration behavior; the real run establishes component behavior.
0118 5. Document a copyable run command, prerequisites and expected result here.
0119    Ensure the case is invoked by the existing runner or explicitly register its
0120    command in `.github/workflows/root-smoke.yml`; check that the workflow's path
0121    filters cover its files. Verify the real check in CI as well as locally.
0122 
0123 Keep the next addition small enough to understand and diagnose on its own.
0124 A shared fixture is useful when it keeps checks simple; a new test framework is
0125 not a prerequisite for adding another component check.
0126 
0127 ## Optional Yall readiness, not a Yall test campaign
0128 
0129 If Yall is relevant to this environment, explicitly request only its version:
0130 
0131 ```sh
0132 python3 tests/smoke/run.py --build-dir NewStructure/build --yall /path/to/yall-run
0133 ```
0134 
0135 This does **not** validate a Yallfile, create/start a campaign, or contact Condor.
0136 Without `--yall`, the report says Yall readiness was not requested. A shell
0137 function is not an executable: if EIC setup exposes only the `yall-run` function,
0138 run `yall-run --version` separately in that activated shell. The runner does not
0139 source environment files or install another copy of Yall.
0140 
0141 ## Build-tree versus installed artifacts
0142 
0143 The current upstream CMake project has no `install()` rules. Its bootstrap builds
0144 in `NewStructure/build`; that is a post-build check, not proof of a relocatable
0145 installation. This suite does not introduce or claim a new install layout.
0146 
0147 For an existing separately deployed layout, point at the actual artifacts rather
0148 than falling back to build-tree binaries:
0149 
0150 ```sh
0151 python3 tests/smoke/run.py --bin-dir /deployment/bin \
0152   --library /deployment/lib/libLFHCAL.so
0153 ```
0154 
0155 The library's directory must also contain nonempty `libLFHCAL_rdict.pcm` and
0156 `libLFHCAL.rootmap`. Use the test checkout matching those artifacts, because the
0157 ROOT payload includes its `NewStructure` class headers. This is not a standalone
0158 binary-only package or a test that build-tree headers are absent. The runner
0159 prints the selected binary/library/ROOT paths and does not change
0160 `LD_LIBRARY_PATH`. Within ROOT it adds the selected library directory to the
0161 dictionary autoload path. Activate the intended runtime environment first. `--root` and
0162 `--root-config` can select exact matching ROOT executables when needed.
0163 
0164 ## Provenance and boundaries
0165 
0166 The readiness checks are consolidated from the actual upstream
0167 `tools/bootstrap-yall-integration.sh`: ROOT version, executable presence and
0168 optional Yall availability. Here executable presence is strengthened to startup.
0169 The write/read pattern follows the existing `NewStructure/Tests/dummy.cc` and
0170 `dummyRead.cc` examples, but uses **current production classes**, checks the
0171 results, and does not reuse their committed binaries or old `Dummy.root` file.
0172 The numerical and checked production-object fixture are new coverage.
0173 
0174 The complementary `tools/tests` suite exercises bootstrap/environment scripts;
0175 `examples/yall/check_shared_conversions.py` checks campaign graphs. The baseline
0176 runs explicitly after compilation or through CI; bootstrap and installation
0177 behavior are unchanged.
0178 
0179 ## Maintainer checks of the runner
0180 
0181 ```sh
0182 python3 -m unittest discover -s tests/smoke -p 'test_runner.py' -v
0183 python3 -m unittest discover -s tools/tests -v
0184 ```
0185 
0186 The first command uses fake executables to test failure propagation, timeouts,
0187 path quoting, temporary-directory cleanup, optional-Yall behavior and the
0188 validation helper's error-detection guards (22 tests). Passing it is **not**
0189 evidence that the production ROOT smoke checks passed. Run the real one-command
0190 check in a compiled LFHCal environment as well.
0191 
0192 ### Optional error-detection validation
0193 
0194 Run the normal checks and then deliberately replace ROOT with the external
0195 `false` executable found on `PATH` to verify that a ROOT write failure is detected:
0196 
0197 ```bash
0198 tests/smoke/validate.sh --build-dir NewStructure/build --timeout 60
0199 ```
0200 
0201 The helper uses an executable path rather than a shell builtin, so the injection
0202 works on Linux and macOS. It labels the normal and deliberately failing phases
0203 clearly, captures only the injected run, and reports
0204 `PASS: expected error detected` only when the runner exits 1 with the exact
0205 expected ROOT-write rejection. A genuine baseline failure stops before injection
0206 and remains visible; unexpected success, another failure stage, a different exit
0207 status, or a timeout prints the injected log and returns nonzero. This offline
0208 injection checks the runner's error detection, not
0209 the behavior of real ROOT.
0210 
0211 ## GitHub Actions
0212 
0213 `ROOT smoke tests` builds all ten standard and five decoder-enabled targets,
0214 runs both runner modes, the unit tests and real ROOT checks on GitHub-hosted
0215 Ubuntu 24.04. It uses the official
0216 `rootproject/root:6.34.00-ubuntu24.04` image pinned by digest, with read-only
0217 repository permissions and the decoder revision recorded in the submodule.
0218 It runs for relevant pull-request changes, pushes to main, or manual dispatch.
0219 The workflow runs the explicit baseline command above. Contributors adding a
0220 case should keep its invocation and path coverage visible in that workflow.
0221 The separate `Yall examples` workflow checks workflow graphs and environment
0222 scripts; its result complements the ROOT build and smoke result.
0223 
0224 ## Legend-lifetime regression
0225 
0226 `python3 -m unittest discover -s tests/smoke -p 'test_legend.py' -v`
0227 compiles and runs the ROOT batch-mode regression for upstream PR #94. It checks
0228 that a legend does not retain a destroyed temporary TF1, preserves line style
0229 and label, and survives repaint over ten layers. Live sampled values and Npx
0230 are checked too; the legacy negative control must reject the dangling pointer.
0231 
0232 This test depends on the production `AddMipFitLegendEntry` helper from #94.
0233 Until that fix is present, it reports **SKIP: not validated**, not a successful
0234 regression result. With the helper present, missing ROOT/compiler, build errors,
0235 wrong legacy results or repaint failures fail the test. The smoke CI discovers
0236 both runner and legend tests. No production implementation is copied here.
0237 For an integration checkout containing #94, set `LFHCAL_SOURCE_DIR` to that
0238 repository root while running this test; it compiles against that checkout's
0239 header with the current ROOT environment.