Warning, /jana2/src/python/externals/pybind11-2.10.3/docs/benchmark.rst is written in an unsupported language. File is not indexed.
0001 Benchmark
0002 =========
0003
0004 The following is the result of a synthetic benchmark comparing both compilation
0005 time and module size of pybind11 against Boost.Python. A detailed report about a
0006 Boost.Python to pybind11 conversion of a real project is available here: [#f1]_.
0007
0008 .. [#f1] http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf
0009
0010 Setup
0011 -----
0012
0013 A python script (see the ``docs/benchmark.py`` file) was used to generate a set
0014 of files with dummy classes whose count increases for each successive benchmark
0015 (between 1 and 2048 classes in powers of two). Each class has four methods with
0016 a randomly generated signature with a return value and four arguments. (There
0017 was no particular reason for this setup other than the desire to generate many
0018 unique function signatures whose count could be controlled in a simple way.)
0019
0020 Here is an example of the binding code for one class:
0021
0022 .. code-block:: cpp
0023
0024 ...
0025 class cl034 {
0026 public:
0027 cl279 *fn_000(cl084 *, cl057 *, cl065 *, cl042 *);
0028 cl025 *fn_001(cl098 *, cl262 *, cl414 *, cl121 *);
0029 cl085 *fn_002(cl445 *, cl297 *, cl145 *, cl421 *);
0030 cl470 *fn_003(cl200 *, cl323 *, cl332 *, cl492 *);
0031 };
0032 ...
0033
0034 PYBIND11_MODULE(example, m) {
0035 ...
0036 py::class_<cl034>(m, "cl034")
0037 .def("fn_000", &cl034::fn_000)
0038 .def("fn_001", &cl034::fn_001)
0039 .def("fn_002", &cl034::fn_002)
0040 .def("fn_003", &cl034::fn_003)
0041 ...
0042 }
0043
0044 The Boost.Python version looks almost identical except that a return value
0045 policy had to be specified as an argument to ``def()``. For both libraries,
0046 compilation was done with
0047
0048 .. code-block:: bash
0049
0050 Apple LLVM version 7.0.2 (clang-700.1.81)
0051
0052 and the following compilation flags
0053
0054 .. code-block:: bash
0055
0056 g++ -Os -shared -rdynamic -undefined dynamic_lookup -fvisibility=hidden -std=c++14
0057
0058 Compilation time
0059 ----------------
0060
0061 The following log-log plot shows how the compilation time grows for an
0062 increasing number of class and function declarations. pybind11 includes many
0063 fewer headers, which initially leads to shorter compilation times, but the
0064 performance is ultimately fairly similar (pybind11 is 19.8 seconds faster for
0065 the largest largest file with 2048 classes and a total of 8192 methods -- a
0066 modest **1.2x** speedup relative to Boost.Python, which required 116.35
0067 seconds).
0068
0069 .. only:: not latex
0070
0071 .. image:: pybind11_vs_boost_python1.svg
0072
0073 .. only:: latex
0074
0075 .. image:: pybind11_vs_boost_python1.png
0076
0077 Module size
0078 -----------
0079
0080 Differences between the two libraries become much more pronounced when
0081 considering the file size of the generated Python plugin: for the largest file,
0082 the binary generated by Boost.Python required 16.8 MiB, which was **2.17
0083 times** / **9.1 megabytes** larger than the output generated by pybind11. For
0084 very small inputs, Boost.Python has an edge in the plot below -- however, note
0085 that it stores many definitions in an external library, whose size was not
0086 included here, hence the comparison is slightly shifted in Boost.Python's
0087 favor.
0088
0089 .. only:: not latex
0090
0091 .. image:: pybind11_vs_boost_python2.svg
0092
0093 .. only:: latex
0094
0095 .. image:: pybind11_vs_boost_python2.png
