Stefans Welt (Posts about Cython)

Cython is 20!

Stefan Behnel — Mon, 04 Apr 2022 06:52:37 GMT

Today, Cython celebrates its 20th anniversary!

On April 4th, 2002, Greg Ewing published the first release of Pyrex 0.1.

Already at the time, it was invented and designed as a compiler that extended the Python language with C data types to build extension modules for CPython. A design that survived the last 20 years, and that made Pyrex, and then Cython, a major corner stone of the Python data ecosystem. And way beyond that.

Now, on April 4th, 2022, its heir Cython is still very much alive and serves easily hundreds of thousands of developers worldwide, day to day.

I'm very grateful for Greg's ingenious invention at the time. Let's look back at how we got to where we are today.

I came to use Python around the time when Pyrex was first written. Python had already been around for a dozen years, version 2.2 was the latest, and had brought us new style classes which represented a major redesign of the type system, as well as the iterator protocol and even generators, which you had to enable with from __future__ import generators, because of potential conflicts with the new keyword yield. There weren't many times in Python's history when that was necessary. Amongst the greatest Python releases of all times, that was one of them.

Python 2.3 was in the makings, about to bring us another bunch of cool new features like sum(), enumerate() or sets as a data type in the standard library.

CPython already had its reputation of providing a large standard library, and a whole set of third-party packages that added another heap of functionality – although Perl's CPAN would easily cover all of it in its mere shadow. There was no pip install anything at the time (not even easy_install), no virtualenvs, no wheels (and no eggs), some form of binary packages, but you'd mostly python setup.py build your own software, especially on Linux. Those were the days.

Many of the binary third-party packages at the time were hand-written using the bare C-API. There was SWIG for generating C wrappers for lots of languages, including Python. It worked, it was actually great to be able to generate multiple wrappers from a single source. But they all looked mostly the same, and making them work and feel well for each of the language environments was hard to impossible. And few people really needed wrappers for more than the one language they used. So, lots of people used the C-API, and CPython had a reputation of being easily extensible in C – assuming you knew C well enough. And more and more Python users didn't.

In came Greg's idea of writing extension modules in Python. Or in something that looked a lot like Python, with a few C-ish extensions.

I don't know how he came up with the name Pyrex, which is a brand name for thermal-resistant glass (originally invented here in Germany in 1893). But Pyrex clearly hit a nerve at the time and grew very quickly. Within weeks and months, there was support not only for functions but for extension types, and for a growing number of Python language features.

By the time I came to use Pyrex, it was already in a very attractive and feature rich state. From the start, its unique selling point was to allow Python developers to write C code, without having to write C code. And that had made it the basis for large projects like Sage, for which it provided a critical software development infrastructure, being the glue code between heaps of C/C++ math libraries and Python.

In 2004, Martijn Faassen took on a project (and he's good at taking on projects) of making XML in Python actually usable. There was support before, there was minidom, there was PyXML with an extended feature set. But many XML features of the time were missing, the tools were memory hungry and slow, and the DOM-API was far from anything Python users would fall in love with.

There was also a Python interface for libxml2, a C library that covered a large part of the important XML technologies at the time. With the caveat of mapping mostly its C-API to Python 1:1, and thus feeling excessively C-ish and exotic to Python users, making it easy to trigger hard crashes at the same time.

There was another alternative, though: ElementTree, designed by the recently deceased Fredrik Lundh (thanks for all the fish, Fredrik). It was not in the standard library at the time, it only got adopted there in Python 2.5 (together with SQLite), one and a half years later. It was an external package based on the pyexpat parser, and it provided a lovely pythonic API for XML processing. But with even less features than minidom.

So, Martijn decided to bring it all together: the bunch of XML features from libxml2, exposed in the pythonic, and already well established, interface of ElementTree. And being a Python developer, wanting to design the interface from the Python point of view, he chose Pyrex to implement that wrapper, and called it lxml.

I found the lxml project almost a year later, while looking for something that I could use as an extensible XML API. I implemented some features for lxml to turn it into that, and along the way, made enhancements also to Pyrex. Over the Pyrex mailing list, I got in touch with other developers who had their own more or less enhanced versions of Pyrex, including Robert Bradshaw, one of the developers in the Sage project. Eventually, in 2007, we decided to follow the example of the Apache web server in bringing together the scattered bunch of existing Pyrex patches into a new project. William Stein from the Sage project came up with a good name, and the infrastructure to maintain it – github.com wasn't launched yet, and we used the Mercurial DVCS. Thus, the Cython project was born.

It was the beginning of a second, long success story.

In the early years, William Stein was able to provide funding to the Cython project from Sage's resources, given how important Cython was for the development of the Sage Math package. Cython was an integral part of the Sage development sprints called Sage days. We participated in Google's Summer of Code events that brought us in touch with Dag Sverre Seljebotn and Mark Florisson, both of whom moved Cython's integration with NumPy and data processing forward in large steps. And the Sage project also sponsored a workshop in München (where I was living at the time) so that we were able to sit together in person, for the first time, discussing, designing and building many great features in Cython, as well as major advances in the coverage of (then) more recent Python language features, in which Vitja Makarov played an important role.

Over time, the list of contributors to Cython grew longer and longer, from large feature additions to small bug fixes and helpful documentation improvements. In 2008, me and Lisandro Dalcín implemented support for Python 3.0 before it was even released, just like Pyrex and Cython have followed CPython's development ever since they existed, allowing users to easily adapt their extension modules to various C-API changes across (C)Python releases. And in the other direction, some of the optimisations that CPython's own internal code generation tool argument clinic employs for fast Python function argument parsing, were adopted from Cython.

I remember discussions and cooperations with the CPython developers Yury Selivanov on async features and with Victor Stinner, Petr Viktorin or Nick Coghlan on Python C-API topics. Several PyPy developers, including Ronan Lamy and Matti Picus, have helped in word and code to improve the integration and stability between both tools. The exchange with people from large and impactful projects like NumPy, Pandas and the scikit-* family of tools has always helped moving Cython in a user centric direction, while giving me a warm feeling that it truly enables its users to get their work done. And the emergence of complementary tools like pybind11 or Numba has helped to diversify the choices throughout the ecosystem in which Cython resides, while only broadening the field without reducing the impact that the language and compiler has for its users.

Today, after 20 years of development, Cython is a modern programming language, embedded in the Python language rather than the other way round, but still extending it with C/C++ super powers.

We helped our users help their users through many exciting endeavours along the way, taking pictures of black holes, sending robots to Mars, scaling up Django websites to a thousand million users, building climate models, analysing, processing and machine learning of human text, real world images, and other data from countless areas, be it scientific, financial, economic, ecologic or probably any other type of data from small to large scale.

I'm proud and happy to see how far Cython has come from its early beginnings. And I'm excited to continue seeing where it will go from here.

So, from New Zealand, from Europe and the Americas, from Asia, Australia and Africa, to anywhere on Earth, and maybe Mars…

Happy anniversary, Cython!

Should you ship the Cython generated C code or not?

Stefan Behnel — Mon, 22 Jun 2020 09:11:11 GMT

When you use Cython for your Python extensions (not if, when ;-)), there are different opinions on whether you should generate the C code locally and ship it in your sdist source packages on PyPI, or you should make Cython a build-time dependency for your package and let users run it on their side.

Both approaches have their pros and cons, but I personally recommend generating the C code on the maintainer side and then shipping it in sdists. Here is a bit of an explanation to help you with your own judgement.

The C code that Cython generates is deterministic and very intentionally adaptive to where you C-compile it. We work hard to do all environment specific adaptations (Python version, C compiler, …) in the C code and not in the code generator that creates it. It's the holy cow of "generate once, compile everywhere". And that's one of the main selling points of Cython, we write C so you don't have to. But obviously, once the C code is generated, it cannot take as-of-now unknown future environmental changes into account any more, such as changes to the CPython C-API, which we only cover in newer Cython releases.

Because the C code is deterministic, making Cython a build time dependency and then pinning an exact Cython version with it is entirely useless, because you can just generate the exact same C code on your side once and ship it. One dependency less, lots of user side complexity avoided. So, the only case we're talking about here is allowing different (usually newer) Cython versions to build your code.

If you ship the C file, then you know what you get and you don't depend on whatever Cython version users have installed on their side. You avoid the maintenance burden of having to respond to bug reports for seemingly unrelated C code lines or bugs in certain Cython versions (which users will rarely mention in their bug reports).

If, instead, you use a recent Cython version at package build time, then you avoid bit rot in the generated C code, but you risk build failures on user side due to users having a buggy Cython version installed (which may not have existed when you shipped the package, so you couldn't exclude it from the dependency range). Or your code may fail to compile with a freshly released Cython due to incompatible language changes in the new version. However, if those (somewhat exceptional) cases don't happen, then you may end up with a setting in which your code adapts also to newer environments, by using a recent Cython version automatically and generated C code that already knows about the new environment. That is definitely an advantage.

Basically, for maintained packages, I consider shipping the generated C code the right way. Less hassle, easier debugging, better user experience. For unmaintained packages, regenerating the C code at build time can extend the lifetime of the package to newer environments for as long as it does not run into failures due to incompatible Cython compiler changes (so you trade one compatibility level for another one).

The question is whether the point at which a package becomes unmaintained can ever be clear enough to make the switch. Regardless of which way you choose, as with all code out there, at some point in the future someone will have to do something, either to your package/code or to your build setup, in order to prevent fatal bit rot. But using Cython in the first place should at least ease the pain of getting over that point when it occurs.

My responses to a Cython dev interview

Stefan Behnel — Thu, 16 Apr 2020 10:46:20 GMT

I recently received a request for an online interview by Jonathan Ruiz, a CS student in Berlin. He's implementing graph algorithms as part of his final Bachelor thesis, and was evaluating and using Cython to get performance improvements. During his work, he thought it'd be nice to get some comments from a Cython core dev and sent me a couple of questions. Here's what I answered.

First of all, thank you Stefan for your time in this difficult situation.

Thanks for asking me.
How did your interest in programming and then compilers begin?

I have a pretty straight forward background and education in computer science and software development. But I'm not a compiler expert. In fact, I'm not even working on a compiler in the true sense. I'm working on Cython, which is a source code translator and code generator. The actual native code generation is then left to a C compiler. However, we avoid that distinction ourselves in the project because in the end, people use Cython to compile Python down to native code. So the distinction is more of an implementation detail.

I came to Cython through a bit of a diversion. I needed a Python XML library for the proof-of-concept implementation of my doctor's thesis somewhere around 2005. Not so long before that, Martijn Faassen had started writing an ElementTree-like wrapper for the XML library libxml2, called lxml, which had several features that I needed and was easy enough for me to hack on to get the features implemented that I was missing.

lxml was written in a code generator called Pyrex, and I ended up implementing a couple of features in that code generator that helped me in my work on lxml. Not all of these changes were accepted upstream, at least not in a timely fashion, and at some point I found that others had that problem, too, and had ended up with their own long-term forks. Together with Robert Bradshaw and William Stein from the University of Washington in Seattle, USA, we decided to fork Pyrex for good, and start a new official project, which we named Cython. That was in 2007, and I've worked on the Cython project ever since.
What advice would you give to students who want to break into this field?

Read code. Seriously. There is a lot that you can learn at a university about algorithms, about smart ideas that people came up with, about ways to tell and decide what's smart and what isn't, about the way things work (and should work) in general. A CS degree is an excellent way to set a basis for your future software design endeavours.

But there's nothing that comes close to reading other people's code when you're trying to understand how things work in real life and why the tools at hand don't do what you want them to do. And then fixing them to do it.
Which branches of mathematics do you think are important to become a good programmer or what particularly benefited you in Cython optimisation, for example?

I would love to say that my math education at university helped me here and there, but in retrospect, I need to admit that I could have come to the same point where I stand now with just my math lessons at school (although those were pretty decent, I guess). I would claim that statistics are surprisingly important in real life and software development, and are not always taught deeply enough at school (nor in CS studies), IMHO. Even just the understanding that the result of a benchmark run is just a single value in a cloud of scattered results really helps putting those numbers in context in your head.

There are definitely fields in software development in which math is more helpful than in the fields I've touched mostly. Graphics comes to mind, for example. But I think what's much more important than a math education is the ability to read and learn, and to be curious of the work of others. Because these days, 95+% of our software development work is based on what others have already done before us (and for us). Use existing tools, learn how they work and what their limits are, and then extend those limits when you need to.
If I'm not mistaken, since April 2019 you have also been a core developer in CPython: what responsibilities does this position entail?

The main (and most obvious) difference is the ability to click the green merge button on github. :) Seriously, you can do a lot of great work in a project without ever clicking that button. You can create tickets, investigate bugs, write documentation, recommend cool projects to others, help people use them, participate in design discussions, write feature pull requests. You can move a project truly forward without being a "core developer". But once you have the merge right, you are taking over the responsibility for the code that you merge by clicking that button, wherever that code came from. If that code unexpectedly breaks someone's computer at the end of the world, you are the one who has to fix it, somehow. Even just by reverting the merge, but you have to do something. That changes your perspective on that piece of code a lot. Even if the author is you.

Being a core developer in a project is really more of an obligation than an honour. But it can also give you a better standing in a project, because others can see that you are taking responsibility for it. So it comes with a bit of a social status, too.
Cython has been and is a key tool in scientific projects, such as the Event Horizon Telescope. Which scientific libraries are you missing in Cython right now? Are there any special ones that you are working on?

I'm not working on scientific libraries myself, although I know a lot of people from other projects in the field. I'm not missing anything here, personally. :)

OTOH, I like hearing about things that others do with Cython. And I like to help others to make Cython do great things for them.

The really cool thing about OpenSource software development is that I'm creating "eternal" values every day. Whatever I write today may end up helping some person on the other side of the planet, or next door, to invent something cool, to answer the last questions about life, the universe and everything, to save the world or someone else's life. That's their projects, their ideas and their work, but it's the software that I am writing together with lots of other people that helps them get their work done. And that is a great feeling.
Are there any important features that you would like to implement in Cython in the future?

The issue tracker has more than 740 open tickets right now. :) But that answer misses the point. I think the most important goal is to keep helping users getting unstuck when they run into something that they can't really (or easily) solve themselves. Or to help them fix their own problems in a way that more people can benefit from. Cython is a tool for others to use for their own needs. It should continue to achieve that.
How do you see Cython ten years from now?

I never liked that question when interviewing dev candidates, and I'm not going to answer it now. ;-) Ten years is about 10-20% of a human's total lifetime. And it's half of an eternity in Tech. It's a very long time. I like how Niels Bohr (supposedly) put it: "predictions are hard to make, especially about the future".
Thank you very much again for your time, Stefan. Take good care of yourself.

Have fun and stay safe.

Speeding up basic object operations in Cython

Stefan Behnel — Sun, 17 Feb 2019 18:24:48 GMT

Raymond Hettinger published a nice little micro-benchmark script for comparing basic operations like attribute or item access in CPython and comparing the performance across Python versions. Unsurprisingly, Cython performs quite well in comparison to the latest CPython 3.8-pre development version, executing most operations 30-50% faster. But the script allowed me to tune some more performance out of certain less well performing operations. The timings are shown below, first those for CPython 3.8-pre as a baseline, then (for comparison) the Cython timings with all optimisations disabled that can be controlled by C macros (gcc -DCYTHON_...=0), the normal (optimised) Cython timings, and the now improved version at the end.

	CPython 3.8 (pre)	Cython 3.0 (no opt)	Cython 3.0 (pre)	Cython 3.0 (tuned)
Variable and attribute read access:
read_local	5.5 ns	0.2 ns	0.2 ns	0.2 ns
read_nonlocal	6.0 ns	0.2 ns	0.2 ns	0.2 ns
read_global	17.9 ns	13.3 ns	2.2 ns	2.2 ns
read_builtin	21.0 ns	0.2 ns	0.2 ns	0.1 ns
read_classvar_from_class	23.7 ns	16.1 ns	14.1 ns	14.1 ns
read_classvar_from_instance	20.9 ns	11.9 ns	11.2 ns	11.0 ns
read_instancevar	31.7 ns	22.3 ns	20.8 ns	22.0 ns
read_instancevar_slots	25.8 ns	16.5 ns	15.3 ns	17.0 ns
read_namedtuple	23.6 ns	16.2 ns	13.9 ns	13.5 ns
read_boundmethod	32.5 ns	23.4 ns	22.2 ns	21.6 ns
Variable and attribute write access:
write_local	6.4 ns	0.2 ns	0.1 ns	0.1 ns
write_nonlocal	6.8 ns	0.2 ns	0.1 ns	0.1 ns
write_global	22.2 ns	13.2 ns	13.7 ns	13.0 ns
write_classvar	114.2 ns	103.2 ns	113.9 ns	94.7 ns
write_instancevar	49.1 ns	34.9 ns	28.6 ns	29.8 ns
write_instancevar_slots	33.4 ns	22.6 ns	16.7 ns	17.8 ns
Data structure read access:
read_list	23.1 ns	5.5 ns	4.0 ns	4.1 ns
read_deque	24.0 ns	5.7 ns	4.3 ns	4.4 ns
read_dict	28.7 ns	21.2 ns	16.5 ns	16.5 ns
read_strdict	23.3 ns	10.7 ns	10.5 ns	12.0 ns
Data structure write access:
write_list	28.0 ns	8.2 ns	4.3 ns	4.2 ns
write_deque	29.5 ns	8.2 ns	6.3 ns	6.4 ns
write_dict	32.9 ns	24.0 ns	21.7 ns	22.6 ns
write_strdict	29.2 ns	16.4 ns	15.8 ns	16.0 ns
Stack (or queue) operations:
list_append_pop	63.6 ns	67.9 ns	20.6 ns	20.5 ns
deque_append_pop	56.0 ns	81.5 ns	159.3 ns	46.0 ns
deque_append_popleft	58.0 ns	56.2 ns	88.1 ns	36.4 ns
Timing loop overhead:
loop_overhead	0.4 ns	0.2 ns	0.1 ns	0.2 ns

Some things that are worth noting:

There is always a bit of variance across the runs, so don't get excited about a couple of percent difference.
The read/write access to local variables is not reasonably measurable in Cython since it uses local/global C variables, and the C compiler discards any useless access to them. But don't worry, they are really fast.
Builtins (and module global variables in Py3.6+) are cached, which explains the "close to nothing" timings for them above.
Even with several optimisations disabled, Cython code is still visibly faster than CPython.
The write_classvar benchmark revealed a performance problem in CPython that is being worked on.
The deque related benchmarks revealed performance problems in Cython that are now fixed, as you can see in the last column.

What's new in Cython 0.29?

Stefan Behnel — Sun, 14 Oct 2018 16:11:05 GMT

I'm happy to announce the release of Cython 0.29. In case you didn't hear about Cython before, it's the most widely used statically optimising Python compiler out there. It translates Python (2/3) code to C, and makes it as easy as Python itself to tune the code all the way down into fast native code. This time, we added several new features that help with speeding up and parallelising regular Python code to escape from the limitations of the GIL.

So, what exactly makes this another great Cython release?

The contributors

First of all, our contributors. A substantial part of the changes in this release was written by users and non-core developers and contributed via pull requests. A big "Thank You!" to all of our contributors and bug reporters! You really made this a great release.

Above all, Gabriel de Marmiesse has invested a remarkable amount of time into restructuring and rewriting the documentation. It now has a lot less historic smell, and much better, tested (!) code examples. And he obviously found more than one problematic piece of code in the docs that we were able to fix along the way.

Cython 3.0

And this will be the last 0.x release of Cython. The Cython compiler has been in production critical use for years, all over the world, and there is really no good reason for it to have an 0.x version scheme. In fact, the 0.x release series can easily be counted as 1.x, which is one of the reasons why we now decided to skip the 1.x series all together. And, while we're at it, why not the 2.x prefix as well. Shift the decimals of 0.29 a bit to the left, and then the next release will be 3.0. The main reason for that is that we want 3.0 to do two things: a) switch the default language compatibility level from Python 2.x to 3.x and b) break with some backwards compatibility issues that get more in the way than they help. We have started collecting a list of things to rethink and change in our bug tracker.

Turning the language level switch is a tiny code change for us, but a larger change for our users and the millions of source lines in their code bases. In order to avoid any resemblance with the years of effort that went into the Py2/3 switch, we took measures that allow users to choose how much effort they want to invest, from "almost none at all" to "as much as they want".

Cython has a long tradition of helping users adapt their code for both Python 2 and Python 3, ever since we ported it to Python 3.0. We used to joke back in 2008 that Cython was the easiest way to migrate an existing Py2 code base to Python 3, and it was never really meant as a joke. Many annoying details are handled internally in the compiler, such as the range versus xrange renaming, or dict iteration. Cython has supported dict and set comprehensions before they were backported to Py2.7, and has long provided three string types (or four, if you want) instead of two. It distinguishes between bytes, str and unicode (and it knows basestring), where str is the type that changes between Py2's bytes str and Py3's Unicode str. This distinction helps users to be explicit, even at the C level, what kind of character or byte sequence they want, and how it should behave across the Py2/3 boundary.

For Cython 3.0, we plan to switch only the default language level, which users can always change via a command line option or the compiler directive language_level. To be clear, Cython will continue to support the existing language semantics. They will just no longer be the default, and users have to select them explicitly by setting language_level=2. That's the "almost none at all" case. In order to prepare the switch to Python 3 language semantics by default, Cython now issues a warning when no language level is explicitly requested, and thus pushes users into being explicit about what semantics their code requires. We obviously hope that many of our users will take the opportunity and migrate their code to the nicer Python 3 semantics, which Cython has long supported as language_level=3.

But we added something even better, so let's see what the current release has to offer.

A new language-level

Cython 0.29 supports a new setting for the language_level directive, language_level=3str, which will become the new default language level in Cython 3.0. We already added it now, so that users can opt in and benefit from it right away, and already prepare their code for the coming change. It's an "in between" kind of setting, which enables all the nice Python 3 goodies that are not syntax compatible with Python 2.x, but without requiring all unprefixed string literals to become Unicode strings when the compiled code runs in Python 2.x. This was one of the biggest problems in the general Py3 migration. And in the context of Cython's integration with C code, it got in the way of our users even a bit more than it would in Python code. Our goals are to make it easy for new users who come from Python 3 to compile their code with Cython and to allow existing (Cython/Python 2) code bases to make use of the benefits before they can make a 100% switch.

Module initialisation like Python does

One great change under the hood is that we managed to enable the PEP-489 support (again). It was already mostly available in Cython 0.27, but lead to problems that made us back-pedal at the time. Now we believe that we found a way to bring the saner module initialisation of Python 3.5 to our users, without risking the previous breakage. Most importantly, features like subinterpreter support or module reloading are detected and disabled, so that Cython compiled extension modules cannot be mistreated in such environments. Actual support for these little used features will probably come at some point, but will certainly require an opt-in of the users, since it is expected to reduce the overall performance of Python operations quite visibly. The more important features like a correct __file__ path being available at import time, and in fact, extension modules looking and behaving exactly like Python modules during the import, are much more helpful to most users.

Compiling plain Python code with OpenMP and memory views

Another PEP is worth mentioning next, actually two PEPs: 484 and 526, vulgo type annotations. Cython has supported type declarations in Python code for years, has switched to PEP-484/526 compatible typing with release 0.27 (more than one year ago), and has now gained several new features that make static typing in Python code much more widely usable. Users can now declare their statically typed Python functions as not requiring the GIL, and thus call them from a parallel OpenMP loops and parallel Python threads, all without leaving Python code compatibility. Even exceptions can now be raised directly from thread-parallel code, without first having to acquire the GIL explicitly.

And memory views are available in Python typing notation:

import cython
from cython.parallel import prange

@cython.cfunc
@cython.nogil
def compute_one_row(row: cython.double[:]) -> cython.int:
    ...

def process_2d_array(data: cython.double[:,:]):
    i: cython.Py_ssize_t

    for i in prange(data.shape[0], num_threads=16, nogil=True):
        compute_one_row(data[i])

This code will work with NumPy arrays when run in Python, and with any data provider that supports the Python buffer interface when compiled with Cython. As a compiled extension module, it will execute at full C speed, in parallel, with 16 OpenMP threads, as requested by the prange() loop. As a normal Python module, it will support all the great Python tools for code analysis, test coverage reporting, debugging, and what not. Although Cython also has direct support for a couple of those by now. Profiling (with cProfile) and coverage analysis (with coverage.py) have been around for several releases, for example. But debugging a Python module in the interpreter is obviously still much easier than debugging a native extension module, with all the edit-compile-run cycle overhead.

Cython's support for compiling pure Python code combines the best of both worlds: native C speed, and easy Python code development, with full support for all the great Python 3.7 language features, even if you still need your (compiled) code to run in Python 2.7.

More speed

Several improvements make use of the dict versioning that was introduced in CPython 3.6. It allows module global names to be looked up much faster, close to the speed of static C globals. Also, the attribute lookup for calls to cpdef methods (C methods with Python wrappers) can benefit a lot, it can become up to 4x faster.

Constant tuples and slices are now deduplicated and only created once at module init time. Especially with common slices like [1:] or [::-1], this can reduce the amount of one-time initialiation code in the generated extension modules.

The changelog lists several other optimisations and improvements.

Many important bug fixes

We've had a hard time following a change in CPython 3.7 that "broke the world", as Mark Shannon put it. It was meant as a mostly internal change on their side that improved the handling of exceptions inside of generators, but it turned out to break all extension modules out there that were built with Cython, and then some. A minimal fix was already released in Cython 0.28.4, but 0.29 brings complete support for the new generator exception stack in CPython 3.7, which allows exceptions raised or handled by Cython implemented generators to interact correctly with CPython's own generators. Upgrading is therefore warmly recommended for better CPython 3.7 support. As usual with Cython, translating your existing code with the new release will make it benefit from the new features, improvements and fixes.

Stackless Python has not been a big focus for Cython development so far, but the developers noticed a problem with Cython modules earlier this year. Normally, they try to keep Stackless binary compatible with CPython, but there are corner cases where this is not possible (specifically frames), and one of these broke the compatibility with Cython compiled modules. Cython 0.29 now contains a fix that makes it play nicely with Stackless 3.x.

A funny bug that is worth noting is a mysteriously disappearing string multiplier in earlier Cython versions. A constant expression like "x" * 5 results in the string "xxxxx", but "x" * 5 + "y" becomes "xy". Apparently not a common code construct, since no user ever complained about it.

Long-time users of Cython and NumPy will be happy to hear that Cython's memory views are now API-1.7 clean, which means that they can get rid of the annoying "Using deprecated NumPy API" warnings in the C compiler output. Simply append the C macro definition ("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION") to the macro setup of your distutils extensions in setup.py to make them disappear. Note that this does not apply to the old low-level ndarray[...] syntax, which exposes several deprecated internals of the NumPy C-API that are not easy to replace. Memory views are a fast high-level abstraction that does not rely specifically on NumPy and therefore does not suffer from these API constraints.

Less compilation :)

And finally, as if to make a point that static compilation is a great tool but not always a good idea, we decided to reduce the number of modules that Cython compiles of itself from 13 down to 8, thus keeping 5 more modules normally interpreted by Python. This makes the compiler runs about 5-7% slower, but reduces the packaged size and the installed binary size by about half, thus reducing download times in CI builds and virtualenv creations. Python is a very efficient language when it comes to functionality per line of code, and its byte code is similarly high-level and efficient. Compiled native code is a lot larger and more verbose in comparison, and this can easily make a difference of megabytes of shared libraries versus kilobytes of Python modules.

We therefore repeat our recommendation to focus Cython's usage on the major pain points in your application, on the critical code sections that a profiler has pointed you at. The ability to compile those, and to tune them at the C level, is what makes Cython such a great and versatile tool.

Cython, pybind11, cffi – which tool should you choose?

Stefan Behnel — Fri, 14 Sep 2018 22:29:09 GMT

In and after the conference talks that I give about Cython, I often get the question how it compares to other tools like pybind11 and cffi. There are others, but these are definitely the three that are widely used and "modern" in the sense that they provide an efficient user experience for today's real-world problems. And as with all tools from the same problem space, there are overlaps and differences. First of all, pybind11 and cffi are pure wrapping tools, whereas Cython is a Python compiler and a complete programming language that is used to implement actual functionality and not just bind to it. So let's focus on the area where the three tools compete: extending Python with native code and libraries.

Using native code from the Python runtime (specifically CPython) has been at the heart of the Python ecosystem since the very early days. Python is a great language for all sorts of programming needs, but it would not have the success and would not be where it is today without its great ecosystem that is heavily based on fast, low-level, native code. And the world of computing is full of such code, often decades old, heavily tuned, well tested and production proven. Looking at indicators like the TIOBE Index suggests that low-level languages like C and C++ are becoming more and more important again even in the last years, decades after their creation.

Today, no-one would attempt to design a (serious/practical) programming language anymore that does not come out of the box with a complete and easy to use way to access all that native code. This ability is often referred to as an FFI, a foreign function interface. Rust is an excellent example for a modern language that was designed with that ability in mind. The FFI in LuaJIT is a great design of a fast and easy to use FFI for the Lua language. Even Java and its JVM, which are certainly not known for their ease of code reuse, have provided the JNI (Java Native Interface) from the early days. CPython, being written in C, has made it very easy to interface with C code right from the start, and above all others the whole scientific computing and big data community has made great use of that over the past 25 years.

Over time, many tools have aimed to simplify the wrapping of external code. The venerable SWIG with its long list of supported target languages is clearly worth mentioning here. Partially a successor to SWIG (and sip), shiboken is a C++ bindings generator used by the PySide project to auto-create wrapper code for the large Qt C++ API.

A general shortcoming of all wrapper generators is that many users eventually reach the limits of their capabilities, be it in terms of performance, feature support, language integration to one side or the other, or whatever. From that point on, users start fighting the tool in order to make it support their use case at hand, and it is not unheard of that projects start over from scratch with a different tool. Therefore, most projects are better off starting directly with a manually written wrapper, at least when the part of the native API that they need to wrap is not prohibitively vast.

The lxml XML toolkit is an excellent example for that. It wraps libxml2 and libxslt with their huge low-level C-APIs. But if the project had used a wrapper generator to wrap it for Python, mapping this C-API to Python would have made the language integration of the Python-level API close to unusable. In fact, the whole project started because generated Python bindings for both already existed that were like the thrilling embrace of an exotic stranger (Mark Pilgrim). And beautifying the API at the Python level by adding another Python wrapper layer would have countered the advantages of a generated wrapper and also severely limited its performance. Despite the sheer vastness of the C-API that it wraps, the decision for manual wrapping and against a wrapper generator was the foundation of a very fast and highly pythonic tool.

Nowadays, three modern tools are widely used in the Python community that support manual wrapping: Cython, cffi and pybind11. These three tools serve three different sides of the need to extend (C)Python with native code.

Cython is Python with native C/C++ data types.

Cython is a static Python compiler. For people coming from a Python background, it is much easier to express their coding needs in Python and then optimising and tuning them, than to rewrite them in a foreign language. Cython allows them to do that by automatically translating their Python code to C, which often avoids the need for an implementation in a low-level language.

Cython uses C type declarations to mix C/C++ operations into Python code freely, be it the usage of C/C++ data types and containers, or of C/C++ functions and objects defined in external libraries. There is a very concise Cython syntax that uses special additional keywords (cdef) outside of Python syntax, as well as ways to declare C types in pure Python syntax. The latter allows writing type annotated Python code that gets optimised into fast C level when compiled by Cython, but that remains entirely pure Python code that can be run, analysed ad debugged with the usual Python tools.

When it comes to wrapping native libraries, Cython has strong support for designing a Python API for them. Being Python, it really keeps the developer focussed on the usage from the Python side and on solving the problem at hand, and takes care of most of the boilerplate code through automatic type conversions and low-level code generation. Its usage is essentially writing C code without having to write C code, but remaining in the wonderful world of the Python language.
pybind11 is modern C++ with Python integration.

pybind11 is the exact opposite of Cython. Coming from C++, and targeting C++ developers, it provides a C++ API that wraps native functions and classes into Python representations. For that, it makes good use of the compile time introspection features that were added to C++11 (hence the name). Thus, it keeps the user focussed on the C++ side of things and takes care of the boilerplate code for mapping it to a Python API.

For everyone who is comfortable with programming in C++ and wants to make direct use of all C++ features, pybind11 is the easiest way to make the C++ code available to Python.
CFFI is Python with a dynamic runtime interface to native code.

cffi then is the dynamic way to load and bind to external shared libraries from regular Python code. It is similar to the ctypes module in the Python standard library, but generally faster and easier to use. Also, it has very good support for the PyPy Python runtime, still better than what Cython and pybind11 can offer. However, the runtime overhead prevents it from coming any close in performance to the statically compiled code that Cython and pybind11 generate for CPython. And the dependency on a well-defined ABI (binary interface) means that C++ support is mostly lacking.

As long as there is a clear API-to-ABI mapping of a shared library, cffi can directly load and use the library file at runtime, given a header file description of the API. In the more complex cases (e.g. when macros are involved), cffi uses a C compiler to generate a native stub wrapper from the description and uses that to communicate with the library. That raises the runtime dependency bar quite a bit compared to ctypes (and both Cython and pybind11 only need a C compiler at build time, not at runtime), but on the other hand also enables wrapping library APIs that are difficult to use with ctypes.

This list shows the clear tradeoffs of the three tools. If performance is not important, if dynamic runtime access to libraries is an advantage, and if users prefer writing their wrapping code in Python, then cffi (or even ctypes) will do the job, nicely and easily. Otherwise, users with a strong C++ background will probably prefer pybind11 since it allows them to write functionality and wrapper code in C++ without switching between languages. For users with a Python background (or at least not with a preference for C/C++), Cython will be very easy to learn and use since the code remains Python, but gains the ability to do efficient native C/C++ operations at any point.

What CPython could use Cython for

Stefan Behnel — Sun, 09 Sep 2018 20:42:14 GMT

There has been a recent discussion about using Cython for CPython development. I think this is a great opportunity for the CPython project to make more efficient use of its scarcest resource: developer time of its spare time contributors and maintainers.

The entry level for new contributors to the CPython project is often perceived to be quite high. While many tasks are actually beginner friendly, such as helping with the documentation or adding features to the Python modules in the stdlib, such important tasks as fixing bugs in the core interpreter, working on data structures, optimising language constructs, or improving the test coverage of the C-API require a solid understanding of C and the CPython C-API.

Since a large part of CPython is implemented in C, and since it exposes a large C-API to extensions and applications, C level testing is key to providing a correct and reliable native API. There were a couple of cases in the past years where new CPython releases actually broke certain parts of the C-API, and it was not noticed until people complained that their applications broke when trying out the new release. This is because the test coverage of the C-API is much lower than the well tested Python level and standard library tests of the runtime. And the main reason for this is that it is much more difficult to write tests in C than in Python, so people have a high incentive to get around it if they can. Since the C-API is used internally inside of the runtime, it is often assumed to be implicitly tested by the Python tests anyway, which raises the bar for an explicit C test even further. But this implicit coverage is not always given, and it also does not reduce the need for regression tests. Cython could help here by making it easier to write C level tests that integrate nicely with the existing Python unit test framework that the CPython project uses.

Basically, writing a C level test in Cython means writing a Python unittest function and then doing an explicit C operation in it that represents the actual test code. Here is an example for testing the PyList_Append C-API function:

from cpython.object cimport PyObject
from cpython.list cimport PyList_Append

def test_PyList_Append_on_empty_list():
    # setup code
    l = []
    assert len(l) == 0
    value = "abc"
    pyobj_value = <PyObject*> value
    refcount_before = pyobj_value.ob_refcnt

    # conservative test call, translates to the expected C code,
    # although with automatic exception propagation if it returns -1:
    errcode = PyList_Append(l, value)

    # validation
    assert errcode == 0
    assert len(l) == 1
    assert l[0] is value
    assert pyobj_value.ob_refcnt == refcount_before + 1

In the Cython project itself, what we actually do is to write doctests. The functions and classes in a test module are compiled with Cython, and the doctests are then executed in Python, and call the Cython implementations. This provides a very nice and easy way to compare the results of Cython operations with those of Python, and also trivially supports data driven tests, by calling a function multiple times from a doctest, for example:

from cpython.number cimport PyNumber_Add

def test_PyNumber_Add(a, b):
    """
    >>> test_PyNumber_Add('abc', 'def')
    'abcdef'
    >>> test_PyNumber_Add('abc', '')
    'abc'
    >>> test_PyNumber_Add(2, 5)
    7
    >>> -2 + 5
    3
    >>> test_PyNumber_Add(-2, 5)
    3
    """
    # The following is equivalent to writing "return a + b" in Python or Cython.
    return PyNumber_Add(a, b)

This could even trivially be combined with hypothesis and other data driven testing tools.

But Cython's use cases are not limited to testing. Maintenance and feature development would probably benefit even more from a reduced entry level.

Many language optimisations are applied in the AST optimiser these days, and that is implemented in C. However, these tree operations can be fairly complex and are thus non-trivial to implement. Doing that in Python rather than C would be much easier to write and maintain, but since this code is a part of the Python compilation process, there's a chicken-and-egg problem here in addition to the performance problem. Cython could solve both problems and allow for more far-reaching optimisations by keeping the necessary transformation code readable.

Performance is also an issue in other parts of CPython, namely the standard library. Several stdlib modules are compute intensive. Many of them have two implementations: one in Python and a faster one in C, a so-called accelerator module. This means that adding a feature to these modules requires duplicate effort, the proficiency in both Python and C, and a solid understanding of the C-API, reference counting, garbage collection, and what not. On the other hand, many modules that could certainly benefit from native performance lack such an accelerator, e.g. difflib, textwrap, fractions, statistics, argparse, email, urllib.parse and many, many more. The asyncio module is becoming more and more important these days, but its native accelerator only covers a very small part of its large functionality, and it also does not expose a native API that performance hungry async tools could hook into. And even though the native accelerator of the ElementTree module is an almost complete replacement, the somewhat complex serialisation code is still implemented completely in Python, which shows in comparison to the native serialisation in lxml.

Compiling these modules with Cython would speed them up, probably quite visibly. For this use case, it is possible to keep the code entirely in Python, and just add enough type declarations to make it fast when compiled. The typing syntax that PEP-484 and PEP-526 added to Python 3.6 makes this really easy and straight forward. A manually written accelerator module could thus be avoided, and therefore a lot of duplicated functionality and maintenance overhead.

Feature development would also be substantially simplified, especially for new contributors. Since Cython compiles Python code, it would allow people to contribute a Python implementation of a new feature that compiles down to C. And we all know that developing new functionality is much easier in Python than in C. The remaining task is then only to optimise it and not to rewrite it in a different language.

My feeling is that replacing some parts of the CPython C development with Cython has the potential to bring a visible boost for the contributions to the CPython project.

Update 2018-09-12: Jeroen Demeyer reminded me that I should also mention the ease of wrapping external native libraries. While this is not something that is a big priority for the standard library anymore, it is certainly true that modules like sqlite (which wraps sqlite3), ssl (OpenSSL), expat or even the io module (which wraps system I/O capabilities) would have been easier to write and maintain in Cython than in C. Especially I/O related code is often intensive in error handling, which is nicer to do with raise and f-strings than error code passing in C.

A really fast Python web server with Cython

Stefan Behnel — Sun, 15 Jul 2018 19:42:51 GMT

Shortly after I wrote about speeding up Python web frameworks with Cython, Nexedi posted an article about their attempt to build a fast multicore web server for Python that can compete with the performance of compiled coroutines in the Go language.

Their goal is to use Cython to build a web framework around a fast native web server, and to use Cython's concurrency and coroutine support to gain native performance also in the application code, without sacrificing the readability that Python provides.

Their experiments look very promising so far. They managed to process 10K requests per second concurrently, which actually do real processing work. That is worth noting, because many web server benchmarks out there content themselves with the blank response time for a "hello world", thus ignoring any concurrency overhead etc. For that simple static "Hello world!", they even got 400K requests per second, which shows that this is not a very realistic benchmark. Under load, their system seems to scale pretty linearly with the number of threads, also not a given among web frameworks.

I might personally get involved in further improving Cython for this kind of concurrent, async applications. Stay tuned.

Cython for web frameworks

Stefan Behnel — Sun, 24 Jun 2018 10:04:21 GMT

I'm excited to see the Python web community pick up Cython more and more to speed up their web frameworks.

uvloop as a fast drop-in replacement for asyncio has been around for a while now, and it's mostly written in Cython as a wrapper around libuv. The Falcon web framework optionally compiles itself with Cython, while keeping up support for PyPy as a plain Python package. New projects like Vibora show that it pays off to design a framework for both (Flask-like) simplicity and (native) speed from the ground up to leverage Cython for the critical parts. Quote of the day:

"300.000 req/sec is a number comparable to Go's built-in web server (I'm saying this based on a rough test I made some years ago). Given that Go is designed to do exactly that, this is really impressive. My kudos to your choice to use Cython." – Reddit user 'beertown'.

Alex Orlov gave a talk at the PyCon-US in 2017 about using Cython for more efficient code, in which he mentioned the possibility to speed up the Django URL dispatcher by 3x, simply by compiling the module as it is.

Especially in async frameworks, minimising the time spent in processing (i.e. outside of the I/O-Loop) is critical for the overall responsiveness and performance. Anton Caceres and I presented fast async code with Cython at EuroPython 2016, showing how to speed up async coroutines by compiling and optimising them.

In order to minimise the processing time on the server, many template engines use native accelerators in one way or another, and writing those in Cython (instead of C/C++) is a huge boost in terms of maintenance (and probably also speed). But several engines also generate Python code from a templating language, and those templates tend to be way more static than not (they are rarely runtime generated themselves). Therefore, compiling the generated template code, or even better, directly targeting Cython with the code generation instead of just plain Python has the potential to speed up the template processing a lot. For example, Cython has very fast support for PEP-498 f-strings and even transforms some '%'-formatting patterns into them to speed them up (also in code that requires backwards compatibility with older Python versions). That can easily make a difference, but also the faster function and method calls or looping code that it generates.

I'm sure there's way more to come and I'm happily looking forward to all those cool developments in the web area that we are only just starting to see appear.

Update 2018-07-15: Nexedi posted an article about their attempts to build a fast web server using Cython, both for the framework layer and the processing at the application layer. Worth keeping an eye on.

What's new in Cython 0.28?

Stefan Behnel — Tue, 13 Mar 2018 22:05:16 GMT

The freshly released Cython 0.28 is another major step in the development of the Cython compiler. It comes with several big new features, some of them long awaited, as well as various little optimisations. It also improves the integration with recent Python features (including the upcoming Python 3.7) and the Pythran compiler for NumPy expressions. The Changelog has the long list of relevant changes. As always, recompiling with the latest version will make your code adapt automatically to new Python releases and take advantage of the new optimisations.

The most long requested change in this release, however, is the support for read-only memory views. You can now simply declare a memory view as const, e.g. const double[:,:], and Cython will request a read-only buffer for it. This allows interaction with bytes objects and other non-writable buffer providers. Note that this makes the item type of the memory view const, i.e. non-writable, and not just the view itself. If the item type is not just a simple numeric type, this might require minor changes to the data types used in the code reading from the view. This feature was an open issue essentially ever since memory views were first introduced into Cython, back in 2009, but during all that time, no-one stepped forward to implement it. Alongside with this improvement, users can now write view[i][j] instead of view[i,j] if they want to, without the previous slow-down due to sub-view creation.

The second very long requested feature is the support for copying C code verbatimly into the generated files. The background is that some constructs, especially C hacks and macros, but also some adaptations to C specifics used by external libraries, can really only be done in plain C in order to use them from Cython. Previously, users had to create an external C header file to implement these things and then use a cdef extern from ... block to include the file from Cython code. Cython 0.28 now allows docstrings on these cdef extern blocks (with or without a specific header file name) that can contain arbitrary C/C++ code, for example:

cdef extern from *:
    """
    #define add1(i) ((i) + 1)
    """
    cdef int add1(int x) nogil

print(add1(x=2))

This is definitely considered an expert feature. Since the code is copied verbatimly into the generated C code file, Cython has no way to apply any validation or safety checks. Use at your own risk.

Another big new feature is the support for multiple inheritance from Python classes by extension types (a.k.a. cdef classes). Previously, extension types could only inherit from other natively implemented types, inlcuding builtins. While cdef classes still cannot inherit only from Python classes, and also cannot inherit from multiple cdef classes, it is now possible to use normal Python classes as additional base classes, following an extension type as primary base. This enables use cases like Python mixins, while still keeping up the efficient memory layout and the fast C-level access to attributes and cdef methods that cdef classes provide.

A bit of work has been done to start reducing the shared library size of Cython generated extension modules. In general, Cython aims to optimise its operations (especially Python operations) for speed and extensively uses C function inlining, optimistic code branches and type specialisations for that. However, the code in the module init function is really only executed once and rarely contains any loops, certainly not time critical ones. Therefore, Cython has now started to avoid certain code intensive optimisations inside of the module init code and also uses GCC pragmas to make the C compiler optimise this specific function for smaller size instead of speed. Without making the import visibly slower, this results in a certain reduction of the overall library size, but probably still leaves some space for future improvements.

Several new optimisations for Python builtins were implemented and often contributed by users. This includes faster operations and iteration for sets and bytearrays, from which existing code can benefit through simple recompilation. We are always happy to receive these contributions, and several tickets in the bug tracker are now marked as beginner friendly "first issues".

Cython has long supported f-strings, and the new release brings another set of little performance improvements for them. More interestingly, however, several common cases of unicode string %-formatting are now mapped to the f-string builder, as long as the argument side is a literal tuple. If the template string uses no unsupported formats, Cython applies this transformation automatically, which leads to visibly faster string formatting and avoids the intermediate creation of Python number objects and the value tuple. Existing code that makes use of %-formatting, including code in compiled Python .py files that needs to stay compatible with Python 2.x, can therefore benefit directly without rewriting all the template strings. Further coverage of formatting features for this transformation is certainly possible, and contributions are welcome.

Finally, a last minute change improves the handling of string literals that are being passed into C++ functions as std::string& references. Previously, the generated code always unpacked a Python byte string and made a fresh copy of it, whereas now Cython detects const arguments and passes the string literal directly. Also in the non-const case, Cython does not follow C++ in outright rejecting the literal argument at compile time, but instead just creates a writable copy and passes it into the function. This avoids special casing in user code and leads to working code by default, as expected in Python, and Cython.