Warning, /jana2/src/python/externals/pybind11-2.10.3/docs/advanced/exceptions.rst is written in an unsupported language. File is not indexed.
0001 Exceptions
0002 ##########
0003
0004 Built-in C++ to Python exception translation
0005 ============================================
0006
0007 When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler
0008 that will trap C++ exceptions, translate them to the corresponding Python exception,
0009 and raise them so that Python code can handle them.
0010
0011 pybind11 defines translations for ``std::exception`` and its standard
0012 subclasses, and several special exception classes that translate to specific
0013 Python exceptions. Note that these are not actually Python exceptions, so they
0014 cannot be examined using the Python C API. Instead, they are pure C++ objects
0015 that pybind11 will translate the corresponding Python exception when they arrive
0016 at its exception handler.
0017
0018 .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
0019
0020 +--------------------------------------+--------------------------------------+
0021 | Exception thrown by C++ | Translated to Python exception type |
0022 +======================================+======================================+
0023 | :class:`std::exception` | ``RuntimeError`` |
0024 +--------------------------------------+--------------------------------------+
0025 | :class:`std::bad_alloc` | ``MemoryError`` |
0026 +--------------------------------------+--------------------------------------+
0027 | :class:`std::domain_error` | ``ValueError`` |
0028 +--------------------------------------+--------------------------------------+
0029 | :class:`std::invalid_argument` | ``ValueError`` |
0030 +--------------------------------------+--------------------------------------+
0031 | :class:`std::length_error` | ``ValueError`` |
0032 +--------------------------------------+--------------------------------------+
0033 | :class:`std::out_of_range` | ``IndexError`` |
0034 +--------------------------------------+--------------------------------------+
0035 | :class:`std::range_error` | ``ValueError`` |
0036 +--------------------------------------+--------------------------------------+
0037 | :class:`std::overflow_error` | ``OverflowError`` |
0038 +--------------------------------------+--------------------------------------+
0039 | :class:`pybind11::stop_iteration` | ``StopIteration`` (used to implement |
0040 | | custom iterators) |
0041 +--------------------------------------+--------------------------------------+
0042 | :class:`pybind11::index_error` | ``IndexError`` (used to indicate out |
0043 | | of bounds access in ``__getitem__``, |
0044 | | ``__setitem__``, etc.) |
0045 +--------------------------------------+--------------------------------------+
0046 | :class:`pybind11::key_error` | ``KeyError`` (used to indicate out |
0047 | | of bounds access in ``__getitem__``, |
0048 | | ``__setitem__`` in dict-like |
0049 | | objects, etc.) |
0050 +--------------------------------------+--------------------------------------+
0051 | :class:`pybind11::value_error` | ``ValueError`` (used to indicate |
0052 | | wrong value passed in |
0053 | | ``container.remove(...)``) |
0054 +--------------------------------------+--------------------------------------+
0055 | :class:`pybind11::type_error` | ``TypeError`` |
0056 +--------------------------------------+--------------------------------------+
0057 | :class:`pybind11::buffer_error` | ``BufferError`` |
0058 +--------------------------------------+--------------------------------------+
0059 | :class:`pybind11::import_error` | ``ImportError`` |
0060 +--------------------------------------+--------------------------------------+
0061 | :class:`pybind11::attribute_error` | ``AttributeError`` |
0062 +--------------------------------------+--------------------------------------+
0063 | Any other exception | ``RuntimeError`` |
0064 +--------------------------------------+--------------------------------------+
0065
0066 Exception translation is not bidirectional. That is, *catching* the C++
0067 exceptions defined above will not trap exceptions that originate from
0068 Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below
0069 <handling_python_exceptions_cpp>` for further details.
0070
0071 There is also a special exception :class:`cast_error` that is thrown by
0072 :func:`handle::call` when the input arguments cannot be converted to Python
0073 objects.
0074
0075 Registering custom translators
0076 ==============================
0077
0078 If the default exception conversion policy described above is insufficient,
0079 pybind11 also provides support for registering custom exception translators.
0080 Similar to pybind11 classes, exception translators can be local to the module
0081 they are defined in or global to the entire python session. To register a simple
0082 exception conversion that translates a C++ exception into a new Python exception
0083 using the C++ exception's ``what()`` method, a helper function is available:
0084
0085 .. code-block:: cpp
0086
0087 py::register_exception<CppExp>(module, "PyExp");
0088
0089 This call creates a Python exception class with the name ``PyExp`` in the given
0090 module and automatically converts any encountered exceptions of type ``CppExp``
0091 into Python exceptions of type ``PyExp``.
0092
0093 A matching function is available for registering a local exception translator:
0094
0095 .. code-block:: cpp
0096
0097 py::register_local_exception<CppExp>(module, "PyExp");
0098
0099
0100 It is possible to specify base class for the exception using the third
0101 parameter, a ``handle``:
0102
0103 .. code-block:: cpp
0104
0105 py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
0106 py::register_local_exception<CppExp>(module, "PyExp", PyExc_RuntimeError);
0107
0108 Then ``PyExp`` can be caught both as ``PyExp`` and ``RuntimeError``.
0109
0110 The class objects of the built-in Python exceptions are listed in the Python
0111 documentation on `Standard Exceptions <https://docs.python.org/3/c-api/exceptions.html#standard-exceptions>`_.
0112 The default base class is ``PyExc_Exception``.
0113
0114 When more advanced exception translation is needed, the functions
0115 ``py::register_exception_translator(translator)`` and
0116 ``py::register_local_exception_translator(translator)`` can be used to register
0117 functions that can translate arbitrary exception types (and which may include
0118 additional logic to do so). The functions takes a stateless callable (e.g. a
0119 function pointer or a lambda function without captured variables) with the call
0120 signature ``void(std::exception_ptr)``.
0121
0122 When a C++ exception is thrown, the registered exception translators are tried
0123 in reverse order of registration (i.e. the last registered translator gets the
0124 first shot at handling the exception). All local translators will be tried
0125 before a global translator is tried.
0126
0127 Inside the translator, ``std::rethrow_exception`` should be used within
0128 a try block to re-throw the exception. One or more catch clauses to catch
0129 the appropriate exceptions should then be used with each clause using
0130 ``PyErr_SetString`` to set a Python exception or ``ex(string)`` to set
0131 the python exception to a custom exception type (see below).
0132
0133 To declare a custom Python exception type, declare a ``py::exception`` variable
0134 and use this in the associated exception translator (note: it is often useful
0135 to make this a static declaration when using it inside a lambda expression
0136 without requiring capturing).
0137
0138 The following example demonstrates this for a hypothetical exception classes
0139 ``MyCustomException`` and ``OtherException``: the first is translated to a
0140 custom python exception ``MyCustomError``, while the second is translated to a
0141 standard python RuntimeError:
0142
0143 .. code-block:: cpp
0144
0145 static py::exception<MyCustomException> exc(m, "MyCustomError");
0146 py::register_exception_translator([](std::exception_ptr p) {
0147 try {
0148 if (p) std::rethrow_exception(p);
0149 } catch (const MyCustomException &e) {
0150 exc(e.what());
0151 } catch (const OtherException &e) {
0152 PyErr_SetString(PyExc_RuntimeError, e.what());
0153 }
0154 });
0155
0156 Multiple exceptions can be handled by a single translator, as shown in the
0157 example above. If the exception is not caught by the current translator, the
0158 previously registered one gets a chance.
0159
0160 If none of the registered exception translators is able to handle the
0161 exception, it is handled by the default converter as described in the previous
0162 section.
0163
0164 .. seealso::
0165
0166 The file :file:`tests/test_exceptions.cpp` contains examples
0167 of various custom exception translators and custom exception types.
0168
0169 .. note::
0170
0171 Call either ``PyErr_SetString`` or a custom exception's call
0172 operator (``exc(string)``) for every exception caught in a custom exception
0173 translator. Failure to do so will cause Python to crash with ``SystemError:
0174 error return without exception set``.
0175
0176 Exceptions that you do not plan to handle should simply not be caught, or
0177 may be explicitly (re-)thrown to delegate it to the other,
0178 previously-declared existing exception translators.
0179
0180 Note that ``libc++`` and ``libstdc++`` `behave differently under macOS
0181 <https://stackoverflow.com/questions/19496643/using-clang-fvisibility-hidden-and-typeinfo-and-type-erasure/28827430>`_
0182 with ``-fvisibility=hidden``. Therefore exceptions that are used across ABI
0183 boundaries need to be explicitly exported, as exercised in
0184 ``tests/test_exceptions.h``. See also:
0185 "Problems with C++ exceptions" under `GCC Wiki <https://gcc.gnu.org/wiki/Visibility>`_.
0186
0187
0188 Local vs Global Exception Translators
0189 =====================================
0190
0191 When a global exception translator is registered, it will be applied across all
0192 modules in the reverse order of registration. This can create behavior where the
0193 order of module import influences how exceptions are translated.
0194
0195 If module1 has the following translator:
0196
0197 .. code-block:: cpp
0198
0199 py::register_exception_translator([](std::exception_ptr p) {
0200 try {
0201 if (p) std::rethrow_exception(p);
0202 } catch (const std::invalid_argument &e) {
0203 PyErr_SetString("module1 handled this")
0204 }
0205 }
0206
0207 and module2 has the following similar translator:
0208
0209 .. code-block:: cpp
0210
0211 py::register_exception_translator([](std::exception_ptr p) {
0212 try {
0213 if (p) std::rethrow_exception(p);
0214 } catch (const std::invalid_argument &e) {
0215 PyErr_SetString("module2 handled this")
0216 }
0217 }
0218
0219 then which translator handles the invalid_argument will be determined by the
0220 order that module1 and module2 are imported. Since exception translators are
0221 applied in the reverse order of registration, which ever module was imported
0222 last will "win" and that translator will be applied.
0223
0224 If there are multiple pybind11 modules that share exception types (either
0225 standard built-in or custom) loaded into a single python instance and
0226 consistent error handling behavior is needed, then local translators should be
0227 used.
0228
0229 Changing the previous example to use ``register_local_exception_translator``
0230 would mean that when invalid_argument is thrown in the module2 code, the
0231 module2 translator will always handle it, while in module1, the module1
0232 translator will do the same.
0233
0234 .. _handling_python_exceptions_cpp:
0235
0236 Handling exceptions from Python in C++
0237 ======================================
0238
0239 When C++ calls Python functions, such as in a callback function or when
0240 manipulating Python objects, and Python raises an ``Exception``, pybind11
0241 converts the Python exception into a C++ exception of type
0242 :class:`pybind11::error_already_set` whose payload contains a C++ string textual
0243 summary and the actual Python exception. ``error_already_set`` is used to
0244 propagate Python exception back to Python (or possibly, handle them in C++).
0245
0246 .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
0247
0248 +--------------------------------------+--------------------------------------+
0249 | Exception raised in Python | Thrown as C++ exception type |
0250 +======================================+======================================+
0251 | Any Python ``Exception`` | :class:`pybind11::error_already_set` |
0252 +--------------------------------------+--------------------------------------+
0253
0254 For example:
0255
0256 .. code-block:: cpp
0257
0258 try {
0259 // open("missing.txt", "r")
0260 auto file = py::module_::import("io").attr("open")("missing.txt", "r");
0261 auto text = file.attr("read")();
0262 file.attr("close")();
0263 } catch (py::error_already_set &e) {
0264 if (e.matches(PyExc_FileNotFoundError)) {
0265 py::print("missing.txt not found");
0266 } else if (e.matches(PyExc_PermissionError)) {
0267 py::print("missing.txt found but not accessible");
0268 } else {
0269 throw;
0270 }
0271 }
0272
0273 Note that C++ to Python exception translation does not apply here, since that is
0274 a method for translating C++ exceptions to Python, not vice versa. The error raised
0275 from Python is always ``error_already_set``.
0276
0277 This example illustrates this behavior:
0278
0279 .. code-block:: cpp
0280
0281 try {
0282 py::eval("raise ValueError('The Ring')");
0283 } catch (py::value_error &boromir) {
0284 // Boromir never gets the ring
0285 assert(false);
0286 } catch (py::error_already_set &frodo) {
0287 // Frodo gets the ring
0288 py::print("I will take the ring");
0289 }
0290
0291 try {
0292 // py::value_error is a request for pybind11 to raise a Python exception
0293 throw py::value_error("The ball");
0294 } catch (py::error_already_set &cat) {
0295 // cat won't catch the ball since
0296 // py::value_error is not a Python exception
0297 assert(false);
0298 } catch (py::value_error &dog) {
0299 // dog will catch the ball
0300 py::print("Run Spot run");
0301 throw; // Throw it again (pybind11 will raise ValueError)
0302 }
0303
0304 Handling errors from the Python C API
0305 =====================================
0306
0307 Where possible, use :ref:`pybind11 wrappers <wrappers>` instead of calling
0308 the Python C API directly. When calling the Python C API directly, in
0309 addition to manually managing reference counts, one must follow the pybind11
0310 error protocol, which is outlined here.
0311
0312 After calling the Python C API, if Python returns an error,
0313 ``throw py::error_already_set();``, which allows pybind11 to deal with the
0314 exception and pass it back to the Python interpreter. This includes calls to
0315 the error setting functions such as ``PyErr_SetString``.
0316
0317 .. code-block:: cpp
0318
0319 PyErr_SetString(PyExc_TypeError, "C API type error demo");
0320 throw py::error_already_set();
0321
0322 // But it would be easier to simply...
0323 throw py::type_error("pybind11 wrapper type error");
0324
0325 Alternately, to ignore the error, call `PyErr_Clear
0326 <https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_.
0327
0328 Any Python error must be thrown or cleared, or Python/pybind11 will be left in
0329 an invalid state.
0330
0331 Chaining exceptions ('raise from')
0332 ==================================
0333
0334 Python has a mechanism for indicating that exceptions were caused by other
0335 exceptions:
0336
0337 .. code-block:: py
0338
0339 try:
0340 print(1 / 0)
0341 except Exception as exc:
0342 raise RuntimeError("could not divide by zero") from exc
0343
0344 To do a similar thing in pybind11, you can use the ``py::raise_from`` function. It
0345 sets the current python error indicator, so to continue propagating the exception
0346 you should ``throw py::error_already_set()``.
0347
0348 .. code-block:: cpp
0349
0350 try {
0351 py::eval("print(1 / 0"));
0352 } catch (py::error_already_set &e) {
0353 py::raise_from(e, PyExc_RuntimeError, "could not divide by zero");
0354 throw py::error_already_set();
0355 }
0356
0357 .. versionadded:: 2.8
0358
0359 .. _unraisable_exceptions:
0360
0361 Handling unraisable exceptions
0362 ==============================
0363
0364 If a Python function invoked from a C++ destructor or any function marked
0365 ``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there
0366 is no way to propagate the exception, as such functions may not throw.
0367 Should they throw or fail to catch any exceptions in their call graph,
0368 the C++ runtime calls ``std::terminate()`` to abort immediately.
0369
0370 Similarly, Python exceptions raised in a class's ``__del__`` method do not
0371 propagate, but are logged by Python as an unraisable error. In Python 3.8+, a
0372 `system hook is triggered
0373 <https://docs.python.org/3/library/sys.html#sys.unraisablehook>`_
0374 and an auditing event is logged.
0375
0376 Any noexcept function should have a try-catch block that traps
0377 class:`error_already_set` (or any other exception that can occur). Note that
0378 pybind11 wrappers around Python exceptions such as
0379 :class:`pybind11::value_error` are *not* Python exceptions; they are C++
0380 exceptions that pybind11 catches and converts to Python exceptions. Noexcept
0381 functions cannot propagate these exceptions either. A useful approach is to
0382 convert them to Python exceptions and then ``discard_as_unraisable`` as shown
0383 below.
0384
0385 .. code-block:: cpp
0386
0387 void nonthrowing_func() noexcept(true) {
0388 try {
0389 // ...
0390 } catch (py::error_already_set &eas) {
0391 // Discard the Python error using Python APIs, using the C++ magic
0392 // variable __func__. Python already knows the type and value and of the
0393 // exception object.
0394 eas.discard_as_unraisable(__func__);
0395 } catch (const std::exception &e) {
0396 // Log and discard C++ exceptions.
0397 third_party::log(e);
0398 }
0399 }
0400
0401 .. versionadded:: 2.6