groups/python/performance_and_plotting.md

0001 @defgroup python_performance_plotting Performance Evaluation and Plotting
0002 @ingroup python_bindings
0003 @brief Extracting efficiency/resolution numbers from a `Sequencer` run, and plotting them.
0004
0005 # Performance writers
0006
0007 ACTS has two output options for the same performance evaluation. Each pair uses the same
0008 collector and produces the same main histogram families. The Python writer returns histograms
0009 through `.histograms()` after `s.run()`; the ROOT writer saves them in a ROOT file. ROOT output
0010 also includes some summary objects and can include matching details.
0011
0012 - **Pattern recognition** (efficiency, fake and duplicate tracks):
0013   `PythonPatternRecognitionPerformanceWriter` for in-memory output, or
0014   `RootPatternRecognitionPerformanceWriter` for a ROOT file.
0015 - **Track parameters** (residuals, pulls, efficiency):
0016   `PythonTrackParameterPerformanceWriter` for in-memory output, or
0017   `RootTrackParameterPerformanceWriter` for a ROOT file.
0018
0019 The Python writers are available from PyPI or a source build. The ROOT writers require a
0020 ROOT-enabled source build.
0021
0022 The pattern-recognition pair takes the same input collections and uses similar configurations.
0023 With a `Sequencer` and truth matcher already configured, add the Python writer like this:
0024
0025 @snippet{trimleft} "../Examples/Scripts/Python/pypi_finding_fitting_demo.py" Python pattern-recognition performance writer
0026
0027 The [PyPI demo test](https://github.com/acts-project/acts/blob/main/Python/Examples/tests/test_examples.py)
0028 executes this example. After `s.run()`, retrieve the results with
0029 `histograms = perfWriterFinder.histograms()`.
0030
0031 For ROOT output, use `acts.examples.root.RootPatternRecognitionPerformanceWriter`, set the same
0032 input collection fields on its `Config`, and set `filePath`. See the
0033 [PyPI finding and fitting demo](https://github.com/acts-project/acts/blob/main/Examples/Scripts/Python/pypi_finding_fitting_demo.py)
0034 for the Python writers and the
0035 [truth-tracking Kalman example](https://github.com/acts-project/acts/blob/main/Examples/Scripts/Python/truth_tracking_kalman.py)
0036 for ROOT output.
0037
0038 ## Resolution fit backend
0039
0040 The track-parameter writers fit residual and pull distributions to extract mean and width
0041 profiles. The ROOT writer uses the ROOT fit backend. For the Python writer, set
0042 `cfg.fitFunction = acts.examples.scipy.makeScipyHistogramFitFunction()` to use SciPy instead;
0043 install `scipy` and `numpy` separately. The ROOT track-parameter writer also supports evaluation
0044 at individual track states and against calibrated measurements. The Python writer currently
0045 evaluates track reference parameters against truth particles.
0046
0047 `TrackTruthMatcher(doubleMatching=True)` is the standard way to produce the
0048 `inputTrackParticleMatching`/`inputParticleTrackMatching` collections these writers need.
0049
0050 ## Available histograms
0051
0052 The writers return efficiency, fake and duplicate track, track-summary, residual, and pull
0053 histograms according to their configuration. Names and dimensions can vary. To see the available
0054 histograms after `s.run()`:
0055
0056 ```python
0057 for name, histogram in sorted(histograms.items()):
0058     print(f"{name:40} {type(histogram).__name__}")
0059 ```
0060
0061 ## Plotting
0062
0063 One-dimensional `Histogram1`, `ProfileHistogram1`, and `Efficiency1` objects support `.plot()`
0064 with matplotlib and mplhep:
0065
0066 ```python
0067 import matplotlib.pyplot as plt
0068
0069 histograms["trackeff_vs_pT"].plot()
0070 plt.savefig("tracking_efficiency.svg")
0071 ```
0072
0073 The following plots illustrate the efficiency and residual views. They use randomly sampled
0074 example counts, not measured ACTS output. The efficiency bars show binomial standard errors;
0075 the residual-count bars show square-root count uncertainties. Regenerate them with
0076 `docs/examples/generate_python_performance_plots.py`.
0077
0078 ![Illustrative tracking efficiency versus transverse momentum.](python/tracking_efficiency.svg){width=450px}
0079 ![Illustrative track-parameter residual distribution.](python/track_residual.svg){width=450px}
0080
0081 ACTS `Histogram1` and `ProfileHistogram1` objects can also be converted to
0082 [boost-histogram](https://boost-histogram.readthedocs.io/) objects for rebinning or other
0083 plotting tools. The converted histogram can also be serialized with Python's `pickle`, so you
0084 can save it and load it in a later analysis. For example, after the pattern-recognition writer
0085 has run:
0086
0087 ```python
0088 import pickle
0089
0090 import boost_histogram as bh
0091
0092 measurements = bh.Histogram(histograms["nMeasurements_vs_eta"])
0093 with open("n_measurements.pkl", "wb") as output:
0094     pickle.dump(measurements, output)
0095 ```
0096
0097 For an `Efficiency1`, convert its `.accepted` and `.total` histograms separately.
0098
0099 For geometry and track visualization (not histogram plotting), see
0100 `acts.examples.visualization.PyVisualization2D` and `TrackVisualizerAlg`, and
0101 `Examples/Scripts/generic_plotter.py` for the YAML-configured plotting tool used by physmon.
0102
0103 > [!note]
0104 > `matplotlib`, `mplhep`, and `boost_histogram` are optional dependencies for this — install
0105 > them alongside `pyacts` if you want to plot.