Warning, /jana2/src/python/externals/pybind11-2.10.3/docs/upgrade.rst is written in an unsupported language. File is not indexed.
0001 Upgrade guide
0002 #############
0003
0004 This is a companion guide to the :doc:`changelog`. While the changelog briefly
0005 lists all of the new features, improvements and bug fixes, this upgrade guide
0006 focuses only the subset which directly impacts your experience when upgrading
0007 to a new version. But it goes into more detail. This includes things like
0008 deprecated APIs and their replacements, build system changes, general code
0009 modernization and other useful information.
0010
0011 .. _upgrade-guide-2.9:
0012
0013 v2.9
0014 ====
0015
0016 * Any usage of the recently added ``py::make_simple_namespace`` should be
0017 converted to using ``py::module_::import("types").attr("SimpleNamespace")``
0018 instead.
0019
0020 * The use of ``_`` in custom type casters can now be replaced with the more
0021 readable ``const_name`` instead. The old ``_`` shortcut has been retained
0022 unless it is being used as a macro (like for gettext).
0023
0024
0025 .. _upgrade-guide-2.7:
0026
0027 v2.7
0028 ====
0029
0030 *Before* v2.7, ``py::str`` can hold ``PyUnicodeObject`` or ``PyBytesObject``,
0031 and ``py::isinstance<str>()`` is ``true`` for both ``py::str`` and
0032 ``py::bytes``. Starting with v2.7, ``py::str`` exclusively holds
0033 ``PyUnicodeObject`` (`#2409 <https://github.com/pybind/pybind11/pull/2409>`_),
0034 and ``py::isinstance<str>()`` is ``true`` only for ``py::str``. To help in
0035 the transition of user code, the ``PYBIND11_STR_LEGACY_PERMISSIVE`` macro
0036 is provided as an escape hatch to go back to the legacy behavior. This macro
0037 will be removed in future releases. Two types of required fixes are expected
0038 to be common:
0039
0040 * Accidental use of ``py::str`` instead of ``py::bytes``, masked by the legacy
0041 behavior. These are probably very easy to fix, by changing from
0042 ``py::str`` to ``py::bytes``.
0043
0044 * Reliance on py::isinstance<str>(obj) being ``true`` for
0045 ``py::bytes``. This is likely to be easy to fix in most cases by adding
0046 ``|| py::isinstance<bytes>(obj)``, but a fix may be more involved, e.g. if
0047 ``py::isinstance<T>`` appears in a template. Such situations will require
0048 careful review and custom fixes.
0049
0050
0051 .. _upgrade-guide-2.6:
0052
0053 v2.6
0054 ====
0055
0056 Usage of the ``PYBIND11_OVERLOAD*`` macros and ``get_overload`` function should
0057 be replaced by ``PYBIND11_OVERRIDE*`` and ``get_override``. In the future, the
0058 old macros may be deprecated and removed.
0059
0060 ``py::module`` has been renamed ``py::module_``, but a backward compatible
0061 typedef has been included. This change was to avoid a language change in C++20
0062 that requires unqualified ``module`` not be placed at the start of a logical
0063 line. Qualified usage is unaffected and the typedef will remain unless the
0064 C++ language rules change again.
0065
0066 The public constructors of ``py::module_`` have been deprecated. Use
0067 ``PYBIND11_MODULE`` or ``module_::create_extension_module`` instead.
0068
0069 An error is now thrown when ``__init__`` is forgotten on subclasses. This was
0070 incorrect before, but was not checked. Add a call to ``__init__`` if it is
0071 missing.
0072
0073 A ``py::type_error`` is now thrown when casting to a subclass (like
0074 ``py::bytes`` from ``py::object``) if the conversion is not valid. Make a valid
0075 conversion instead.
0076
0077 The undocumented ``h.get_type()`` method has been deprecated and replaced by
0078 ``py::type::of(h)``.
0079
0080 Enums now have a ``__str__`` method pre-defined; if you want to override it,
0081 the simplest fix is to add the new ``py::prepend()`` tag when defining
0082 ``"__str__"``.
0083
0084 If ``__eq__`` defined but not ``__hash__``, ``__hash__`` is now set to
0085 ``None``, as in normal CPython. You should add ``__hash__`` if you intended the
0086 class to be hashable, possibly using the new ``py::hash`` shortcut.
0087
0088 The constructors for ``py::array`` now always take signed integers for size,
0089 for consistency. This may lead to compiler warnings on some systems. Cast to
0090 ``py::ssize_t`` instead of ``std::size_t``.
0091
0092 The ``tools/clang`` submodule and ``tools/mkdoc.py`` have been moved to a
0093 standalone package, `pybind11-mkdoc`_. If you were using those tools, please
0094 use them via a pip install from the new location.
0095
0096 The ``pybind11`` package on PyPI no longer fills the wheel "headers" slot - if
0097 you were using the headers from this slot, they are available by requesting the
0098 ``global`` extra, that is, ``pip install "pybind11[global]"``. (Most users will
0099 be unaffected, as the ``pybind11/include`` location is reported by ``python -m
0100 pybind11 --includes`` and ``pybind11.get_include()`` is still correct and has
0101 not changed since 2.5).
0102
0103 .. _pybind11-mkdoc: https://github.com/pybind/pybind11-mkdoc
0104
0105 CMake support:
0106 --------------
0107
0108 The minimum required version of CMake is now 3.4. Several details of the CMake
0109 support have been deprecated; warnings will be shown if you need to change
0110 something. The changes are:
0111
0112 * ``PYBIND11_CPP_STANDARD=<platform-flag>`` is deprecated, please use
0113 ``CMAKE_CXX_STANDARD=<number>`` instead, or any other valid CMake CXX or CUDA
0114 standard selection method, like ``target_compile_features``.
0115
0116 * If you do not request a standard, pybind11 targets will compile with the
0117 compiler default, but not less than C++11, instead of forcing C++14 always.
0118 If you depend on the old behavior, please use ``set(CMAKE_CXX_STANDARD 14 CACHE STRING "")``
0119 instead.
0120
0121 * Direct ``pybind11::module`` usage should always be accompanied by at least
0122 ``set(CMAKE_CXX_VISIBILITY_PRESET hidden)`` or similar - it used to try to
0123 manually force this compiler flag (but not correctly on all compilers or with
0124 CUDA).
0125
0126 * ``pybind11_add_module``'s ``SYSTEM`` argument is deprecated and does nothing;
0127 linking now behaves like other imported libraries consistently in both
0128 config and submodule mode, and behaves like a ``SYSTEM`` library by
0129 default.
0130
0131 * If ``PYTHON_EXECUTABLE`` is not set, virtual environments (``venv``,
0132 ``virtualenv``, and ``conda``) are prioritized over the standard search
0133 (similar to the new FindPython mode).
0134
0135 In addition, the following changes may be of interest:
0136
0137 * ``CMAKE_INTERPROCEDURAL_OPTIMIZATION`` will be respected by
0138 ``pybind11_add_module`` if set instead of linking to ``pybind11::lto`` or
0139 ``pybind11::thin_lto``.
0140
0141 * Using ``find_package(Python COMPONENTS Interpreter Development)`` before
0142 pybind11 will cause pybind11 to use the new Python mechanisms instead of its
0143 own custom search, based on a patched version of classic ``FindPythonInterp``
0144 / ``FindPythonLibs``. In the future, this may become the default. A recent
0145 (3.15+ or 3.18.2+) version of CMake is recommended.
0146
0147
0148
0149 v2.5
0150 ====
0151
0152 The Python package now includes the headers as data in the package itself, as
0153 well as in the "headers" wheel slot. ``pybind11 --includes`` and
0154 ``pybind11.get_include()`` report the new location, which is always correct
0155 regardless of how pybind11 was installed, making the old ``user=`` argument
0156 meaningless. If you are not using the function to get the location already, you
0157 are encouraged to switch to the package location.
0158
0159
0160 v2.2
0161 ====
0162
0163 Deprecation of the ``PYBIND11_PLUGIN`` macro
0164 --------------------------------------------
0165
0166 ``PYBIND11_MODULE`` is now the preferred way to create module entry points.
0167 The old macro emits a compile-time deprecation warning.
0168
0169 .. code-block:: cpp
0170
0171 // old
0172 PYBIND11_PLUGIN(example) {
0173 py::module m("example", "documentation string");
0174
0175 m.def("add", [](int a, int b) { return a + b; });
0176
0177 return m.ptr();
0178 }
0179
0180 // new
0181 PYBIND11_MODULE(example, m) {
0182 m.doc() = "documentation string"; // optional
0183
0184 m.def("add", [](int a, int b) { return a + b; });
0185 }
0186
0187
0188 New API for defining custom constructors and pickling functions
0189 ---------------------------------------------------------------
0190
0191 The old placement-new custom constructors have been deprecated. The new approach
0192 uses ``py::init()`` and factory functions to greatly improve type safety.
0193
0194 Placement-new can be called accidentally with an incompatible type (without any
0195 compiler errors or warnings), or it can initialize the same object multiple times
0196 if not careful with the Python-side ``__init__`` calls. The new-style custom
0197 constructors prevent such mistakes. See :ref:`custom_constructors` for details.
0198
0199 .. code-block:: cpp
0200
0201 // old -- deprecated (runtime warning shown only in debug mode)
0202 py::class<Foo>(m, "Foo")
0203 .def("__init__", [](Foo &self, ...) {
0204 new (&self) Foo(...); // uses placement-new
0205 });
0206
0207 // new
0208 py::class<Foo>(m, "Foo")
0209 .def(py::init([](...) { // Note: no `self` argument
0210 return new Foo(...); // return by raw pointer
0211 // or: return std::make_unique<Foo>(...); // return by holder
0212 // or: return Foo(...); // return by value (move constructor)
0213 }));
0214
0215 Mirroring the custom constructor changes, ``py::pickle()`` is now the preferred
0216 way to get and set object state. See :ref:`pickling` for details.
0217
0218 .. code-block:: cpp
0219
0220 // old -- deprecated (runtime warning shown only in debug mode)
0221 py::class<Foo>(m, "Foo")
0222 ...
0223 .def("__getstate__", [](const Foo &self) {
0224 return py::make_tuple(self.value1(), self.value2(), ...);
0225 })
0226 .def("__setstate__", [](Foo &self, py::tuple t) {
0227 new (&self) Foo(t[0].cast<std::string>(), ...);
0228 });
0229
0230 // new
0231 py::class<Foo>(m, "Foo")
0232 ...
0233 .def(py::pickle(
0234 [](const Foo &self) { // __getstate__
0235 return py::make_tuple(self.value1(), self.value2(), ...); // unchanged
0236 },
0237 [](py::tuple t) { // __setstate__, note: no `self` argument
0238 return new Foo(t[0].cast<std::string>(), ...);
0239 // or: return std::make_unique<Foo>(...); // return by holder
0240 // or: return Foo(...); // return by value (move constructor)
0241 }
0242 ));
0243
0244 For both the constructors and pickling, warnings are shown at module
0245 initialization time (on import, not when the functions are called).
0246 They're only visible when compiled in debug mode. Sample warning:
0247
0248 .. code-block:: none
0249
0250 pybind11-bound class 'mymodule.Foo' is using an old-style placement-new '__init__'
0251 which has been deprecated. See the upgrade guide in pybind11's docs.
0252
0253
0254 Stricter enforcement of hidden symbol visibility for pybind11 modules
0255 ---------------------------------------------------------------------
0256
0257 pybind11 now tries to actively enforce hidden symbol visibility for modules.
0258 If you're using either one of pybind11's :doc:`CMake or Python build systems
0259 <compiling>` (the two example repositories) and you haven't been exporting any
0260 symbols, there's nothing to be concerned about. All the changes have been done
0261 transparently in the background. If you were building manually or relied on
0262 specific default visibility, read on.
0263
0264 Setting default symbol visibility to *hidden* has always been recommended for
0265 pybind11 (see :ref:`faq:symhidden`). On Linux and macOS, hidden symbol
0266 visibility (in conjunction with the ``strip`` utility) yields much smaller
0267 module binaries. `CPython's extension docs`_ also recommend hiding symbols
0268 by default, with the goal of avoiding symbol name clashes between modules.
0269 Starting with v2.2, pybind11 enforces this more strictly: (1) by declaring
0270 all symbols inside the ``pybind11`` namespace as hidden and (2) by including
0271 the ``-fvisibility=hidden`` flag on Linux and macOS (only for extension
0272 modules, not for embedding the interpreter).
0273
0274 .. _CPython's extension docs: https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module
0275
0276 The namespace-scope hidden visibility is done automatically in pybind11's
0277 headers and it's generally transparent to users. It ensures that:
0278
0279 * Modules compiled with different pybind11 versions don't clash with each other.
0280
0281 * Some new features, like ``py::module_local`` bindings, can work as intended.
0282
0283 The ``-fvisibility=hidden`` flag applies the same visibility to user bindings
0284 outside of the ``pybind11`` namespace. It's now set automatic by pybind11's
0285 CMake and Python build systems, but this needs to be done manually by users
0286 of other build systems. Adding this flag:
0287
0288 * Minimizes the chances of symbol conflicts between modules. E.g. if two
0289 unrelated modules were statically linked to different (ABI-incompatible)
0290 versions of the same third-party library, a symbol clash would be likely
0291 (and would end with unpredictable results).
0292
0293 * Produces smaller binaries on Linux and macOS, as pointed out previously.
0294
0295 Within pybind11's CMake build system, ``pybind11_add_module`` has always been
0296 setting the ``-fvisibility=hidden`` flag in release mode. From now on, it's
0297 being applied unconditionally, even in debug mode and it can no longer be opted
0298 out of with the ``NO_EXTRAS`` option. The ``pybind11::module`` target now also
0299 adds this flag to its interface. The ``pybind11::embed`` target is unchanged.
0300
0301 The most significant change here is for the ``pybind11::module`` target. If you
0302 were previously relying on default visibility, i.e. if your Python module was
0303 doubling as a shared library with dependents, you'll need to either export
0304 symbols manually (recommended for cross-platform libraries) or factor out the
0305 shared library (and have the Python module link to it like the other
0306 dependents). As a temporary workaround, you can also restore default visibility
0307 using the CMake code below, but this is not recommended in the long run:
0308
0309 .. code-block:: cmake
0310
0311 target_link_libraries(mymodule PRIVATE pybind11::module)
0312
0313 add_library(restore_default_visibility INTERFACE)
0314 target_compile_options(restore_default_visibility INTERFACE -fvisibility=default)
0315 target_link_libraries(mymodule PRIVATE restore_default_visibility)
0316
0317
0318 Local STL container bindings
0319 ----------------------------
0320
0321 Previous pybind11 versions could only bind types globally -- all pybind11
0322 modules, even unrelated ones, would have access to the same exported types.
0323 However, this would also result in a conflict if two modules exported the
0324 same C++ type, which is especially problematic for very common types, e.g.
0325 ``std::vector<int>``. :ref:`module_local` were added to resolve this (see
0326 that section for a complete usage guide).
0327
0328 ``py::class_`` still defaults to global bindings (because these types are
0329 usually unique across modules), however in order to avoid clashes of opaque
0330 types, ``py::bind_vector`` and ``py::bind_map`` will now bind STL containers
0331 as ``py::module_local`` if their elements are: builtins (``int``, ``float``,
0332 etc.), not bound using ``py::class_``, or bound as ``py::module_local``. For
0333 example, this change allows multiple modules to bind ``std::vector<int>``
0334 without causing conflicts. See :ref:`stl_bind` for more details.
0335
0336 When upgrading to this version, if you have multiple modules which depend on
0337 a single global binding of an STL container, note that all modules can still
0338 accept foreign ``py::module_local`` types in the direction of Python-to-C++.
0339 The locality only affects the C++-to-Python direction. If this is needed in
0340 multiple modules, you'll need to either:
0341
0342 * Add a copy of the same STL binding to all of the modules which need it.
0343
0344 * Restore the global status of that single binding by marking it
0345 ``py::module_local(false)``.
0346
0347 The latter is an easy workaround, but in the long run it would be best to
0348 localize all common type bindings in order to avoid conflicts with
0349 third-party modules.
0350
0351
0352 Negative strides for Python buffer objects and numpy arrays
0353 -----------------------------------------------------------
0354
0355 Support for negative strides required changing the integer type from unsigned
0356 to signed in the interfaces of ``py::buffer_info`` and ``py::array``. If you
0357 have compiler warnings enabled, you may notice some new conversion warnings
0358 after upgrading. These can be resolved using ``static_cast``.
0359
0360
0361 Deprecation of some ``py::object`` APIs
0362 ---------------------------------------
0363
0364 To compare ``py::object`` instances by pointer, you should now use
0365 ``obj1.is(obj2)`` which is equivalent to ``obj1 is obj2`` in Python.
0366 Previously, pybind11 used ``operator==`` for this (``obj1 == obj2``), but
0367 that could be confusing and is now deprecated (so that it can eventually
0368 be replaced with proper rich object comparison in a future release).
0369
0370 For classes which inherit from ``py::object``, ``borrowed`` and ``stolen``
0371 were previously available as protected constructor tags. Now the types
0372 should be used directly instead: ``borrowed_t{}`` and ``stolen_t{}``
0373 (`#771 <https://github.com/pybind/pybind11/pull/771>`_).
0374
0375
0376 Stricter compile-time error checking
0377 ------------------------------------
0378
0379 Some error checks have been moved from run time to compile time. Notably,
0380 automatic conversion of ``std::shared_ptr<T>`` is not possible when ``T`` is
0381 not directly registered with ``py::class_<T>`` (e.g. ``std::shared_ptr<int>``
0382 or ``std::shared_ptr<std::vector<T>>`` are not automatically convertible).
0383 Attempting to bind a function with such arguments now results in a compile-time
0384 error instead of waiting to fail at run time.
0385
0386 ``py::init<...>()`` constructor definitions are also stricter and now prevent
0387 bindings which could cause unexpected behavior:
0388
0389 .. code-block:: cpp
0390
0391 struct Example {
0392 Example(int &);
0393 };
0394
0395 py::class_<Example>(m, "Example")
0396 .def(py::init<int &>()); // OK, exact match
0397 // .def(py::init<int>()); // compile-time error, mismatch
0398
0399 A non-``const`` lvalue reference is not allowed to bind to an rvalue. However,
0400 note that a constructor taking ``const T &`` can still be registered using
0401 ``py::init<T>()`` because a ``const`` lvalue reference can bind to an rvalue.
0402
0403 v2.1
0404 ====
0405
0406 Minimum compiler versions are enforced at compile time
0407 ------------------------------------------------------
0408
0409 The minimums also apply to v2.0 but the check is now explicit and a compile-time
0410 error is raised if the compiler does not meet the requirements:
0411
0412 * GCC >= 4.8
0413 * clang >= 3.3 (appleclang >= 5.0)
0414 * MSVC >= 2015u3
0415 * Intel C++ >= 15.0
0416
0417
0418 The ``py::metaclass`` attribute is not required for static properties
0419 ---------------------------------------------------------------------
0420
0421 Binding classes with static properties is now possible by default. The
0422 zero-parameter version of ``py::metaclass()`` is deprecated. However, a new
0423 one-parameter ``py::metaclass(python_type)`` version was added for rare
0424 cases when a custom metaclass is needed to override pybind11's default.
0425
0426 .. code-block:: cpp
0427
0428 // old -- emits a deprecation warning
0429 py::class_<Foo>(m, "Foo", py::metaclass())
0430 .def_property_readonly_static("foo", ...);
0431
0432 // new -- static properties work without the attribute
0433 py::class_<Foo>(m, "Foo")
0434 .def_property_readonly_static("foo", ...);
0435
0436 // new -- advanced feature, override pybind11's default metaclass
0437 py::class_<Bar>(m, "Bar", py::metaclass(custom_python_type))
0438 ...
0439
0440
0441 v2.0
0442 ====
0443
0444 Breaking changes in ``py::class_``
0445 ----------------------------------
0446
0447 These changes were necessary to make type definitions in pybind11
0448 future-proof, to support PyPy via its ``cpyext`` mechanism (`#527
0449 <https://github.com/pybind/pybind11/pull/527>`_), and to improve efficiency
0450 (`rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_).
0451
0452 1. Declarations of types that provide access via the buffer protocol must
0453 now include the ``py::buffer_protocol()`` annotation as an argument to
0454 the ``py::class_`` constructor.
0455
0456 .. code-block:: cpp
0457
0458 py::class_<Matrix>("Matrix", py::buffer_protocol())
0459 .def(py::init<...>())
0460 .def_buffer(...);
0461
0462 2. Classes which include static properties (e.g. ``def_readwrite_static()``)
0463 must now include the ``py::metaclass()`` attribute. Note: this requirement
0464 has since been removed in v2.1. If you're upgrading from 1.x, it's
0465 recommended to skip directly to v2.1 or newer.
0466
0467 3. This version of pybind11 uses a redesigned mechanism for instantiating
0468 trampoline classes that are used to override virtual methods from within
0469 Python. This led to the following user-visible syntax change:
0470
0471 .. code-block:: cpp
0472
0473 // old v1.x syntax
0474 py::class_<TrampolineClass>("MyClass")
0475 .alias<MyClass>()
0476 ...
0477
0478 // new v2.x syntax
0479 py::class_<MyClass, TrampolineClass>("MyClass")
0480 ...
0481
0482 Importantly, both the original and the trampoline class are now specified
0483 as arguments to the ``py::class_`` template, and the ``alias<..>()`` call
0484 is gone. The new scheme has zero overhead in cases when Python doesn't
0485 override any functions of the underlying C++ class.
0486 `rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_.
0487
0488 The class type must be the first template argument given to ``py::class_``
0489 while the trampoline can be mixed in arbitrary order with other arguments
0490 (see the following section).
0491
0492
0493 Deprecation of the ``py::base<T>()`` attribute
0494 ----------------------------------------------
0495
0496 ``py::base<T>()`` was deprecated in favor of specifying ``T`` as a template
0497 argument to ``py::class_``. This new syntax also supports multiple inheritance.
0498 Note that, while the type being exported must be the first argument in the
0499 ``py::class_<Class, ...>`` template, the order of the following types (bases,
0500 holder and/or trampoline) is not important.
0501
0502 .. code-block:: cpp
0503
0504 // old v1.x
0505 py::class_<Derived>("Derived", py::base<Base>());
0506
0507 // new v2.x
0508 py::class_<Derived, Base>("Derived");
0509
0510 // new -- multiple inheritance
0511 py::class_<Derived, Base1, Base2>("Derived");
0512
0513 // new -- apart from `Derived` the argument order can be arbitrary
0514 py::class_<Derived, Base1, Holder, Base2, Trampoline>("Derived");
0515
0516
0517 Out-of-the-box support for ``std::shared_ptr``
0518 ----------------------------------------------
0519
0520 The relevant type caster is now built in, so it's no longer necessary to
0521 include a declaration of the form:
0522
0523 .. code-block:: cpp
0524
0525 PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>)
0526
0527 Continuing to do so won't cause an error or even a deprecation warning,
0528 but it's completely redundant.
0529
0530
0531 Deprecation of a few ``py::object`` APIs
0532 ----------------------------------------
0533
0534 All of the old-style calls emit deprecation warnings.
0535
0536 +---------------------------------------+---------------------------------------------+
0537 | Old syntax | New syntax |
0538 +=======================================+=============================================+
0539 | ``obj.call(args...)`` | ``obj(args...)`` |
0540 +---------------------------------------+---------------------------------------------+
0541 | ``obj.str()`` | ``py::str(obj)`` |
0542 +---------------------------------------+---------------------------------------------+
0543 | ``auto l = py::list(obj); l.check()`` | ``py::isinstance<py::list>(obj)`` |
0544 +---------------------------------------+---------------------------------------------+
0545 | ``py::object(ptr, true)`` | ``py::reinterpret_borrow<py::object>(ptr)`` |
0546 +---------------------------------------+---------------------------------------------+
0547 | ``py::object(ptr, false)`` | ``py::reinterpret_steal<py::object>(ptr)`` |
0548 +---------------------------------------+---------------------------------------------+
0549 | ``if (obj.attr("foo"))`` | ``if (py::hasattr(obj, "foo"))`` |
0550 +---------------------------------------+---------------------------------------------+
0551 | ``if (obj["bar"])`` | ``if (obj.contains("bar"))`` |
0552 +---------------------------------------+---------------------------------------------+