In and after the conference talks that I give about Cython, I often get the question how it compares to other tools like pybind11 and cffi. There are others, but these are definitely the three that are widely used and modern in the sense that they provide an efficient user experience for real-world problems. And as with all tools from the same problem space, there are overlaps and differences. First of all, pybind11 and cffi are pure wrapping tools, whereas Cython is a Python compiler and a complete programming language that is used to implement actual functionality and not just bind to it. So let's focus on the area where the three tools compete: extending Python with native code and libraries.
Using native code from the Python runtime (specifically CPython) has been at the heart of the Python ecosystem since the very early days. Python is a great language for all sorts of programming needs, but it would not have the success and would not be where it is today without its great ecosystem that is heavily based on fast, low-level, native code. And the world of computing is full of such code, often decades old, heavily tuned, well tested and production proven. Looking at indicators like the TIOBE Index suggests that low-level languages like C and C++ keep becoming rather more than less important even in the last years, even decades after their creation.
Today, no-one would attempt to design a (serious/practical) programming language anymore that does not come out of the box with a complete and easy to use way to access all that native code. This ability is often referred to as an FFI, a foreign function interface. Rust is an excellent example for a modern language that was designed with that ability in mind. Even Java and its JVM, which are certainly not known for their ease of code reuse, have provided the JNI (Java Native Interface) from the early days. CPython, being written in C, has made it very easy to interface with C code right from the start, and above all others the whole scientific computing and big data community has made great use of it over the past 25 years.
Over the time, many tools have aimed to simplify the wrapping of external code. The FFI in LuaJIT is an excellent design of an easy to use FFI for the Lua language. The venerable SWIG with its long list of supported target languages is clearly worth mentioning here. Partially a successor to SWIG (and sip), shiboken is a C++ bindings generator used by the PySide project to auto-create wrapper code for the large Qt C++ API.
A general shortcoming of all wrapper generators is that many users eventually reach the limits of their capabilities, be it in terms of performance, feature support, language integration to one side or the other, or whatever. From that point on, users start fighting the tool in order to make it support their use case at hand, and it is not unheard of that projects start over from scratch with a different tool. Therefore, most projects are better off starting directly with a manually written wrapper, at least when the part of the native API that they need to wrap is not prohibitively vast.
Nowadays, three modern tools are widely used in the Python community that support manual wrapping: Cython, cffi and pybind11. These three tools serve three different sides of the need to extend (C)Python with native code.
Cython is Python with native C/C++ data types.
For people coming from a Python background, it is much easier to express their coding needs in Python and then optimising and tuning them, than to rewrite them in a foreign language. Cython allows them to do that by automatically translating their Python code to C, which often avoids the need for an implementation in a low-level language in the first place. Cython uses C type declarations to mix C/C++ operations into Python code freely. There is a very concise Cython syntax that uses special additional keywords (cdef) outside of Python syntax, as well as ways to declare C types in pure Python syntax. The latter allows writing type annotated Python code that gets optimised into fast C level when compiled by Cython, but that remains entirely pure Python code that can be run and analysed by the usual Python tools.
When it comes to wrapping native libraries, Cython has strong support for designing a Python API for them. Being Python, it really keeps the developer focussed on the usage from the Python side and on solving the problem at hand, and takes care of most of the boiler plate code through automatic type conversions and low-level code generation. Its usage is essentially writing C code without having to write C code, but remaining in the wonderful world of the Python language.
pybind11 is modern C++ with Python integration.
pybind11 is the exact opposite of Cython. Coming from C++, it provides a programming API that automatically wraps C++ functions and classes into Python representations. For that, it makes good use of the compile time introspection features that were added to C++11 (hence the name).
For everyone who is comfortable with programming in C++ and wants to make direct use of all C++ features, pybind11 is the easiest way to make the C++ code available to Python.
CFFI is Python with a dynamic runtime interface to native code.
cffi then is the dynamic way to load and bind to external shared libraries. It is similar to the ctypes module in the Python standard library, but generally faster and easier to use. Also, it has very good support for the PyPy Python runtime, still better than what Cython and pybind11 can offer. However, the runtime overhead prevents it from coming any close in performance to the statically compiled code that Cython and pybind11 generate for CPython. And the dependency on a well-defined ABI (binary interface) means that C++ support is mostly lacking.
As long as there is a clear API-to-ABI mapping of the library, cffi can directly load and use a library at runtime, given a header file description of the API. In the more complex cases (e.g. when macros are involved), cffi uses a C compiler to generate a native stub wrapper from the description and uses that to communicate with the library. That raises the dependency bar quite a bit compared to ctypes, but on the other hand also enables wrapping library APIs that are difficult to use with the latter.
This list shows the clear tradeoffs of the three tools. If performance is not important and dynamic runtime access to libraries is an advantage, cffi (or even ctypes) will do the job. Otherwise, it depends on the background of the developer whether Cython or pybind11 are preferable. For users with a strong C++ background, pybind11 will probably be the tool of choice. For users with a Python background (or at least not with a lot of C/C++ experience), Cython will be much easier to learn.