What's new in Cython 0.23
It was really nice to hear Cython being mentioned in pretty much every second talk at this year's EuroSciPy in Cambridge, either directly or by promoting tools that were themselves written in Cython. It seems to have become a key cornerstone in Python's ecosystem.
For me, this has been an intense conference summer overall, so I only just noticed that I hadn't written up anything about the long list of changes in Cython 0.23 yet. Well, apart from the changelog :). So here it goes.
Cython 0.23 is a major feature release. It implements the new language features of the new Python 3.5, but with the usual backports to make them available in Python 2.6 and later. My personal feature favourites are PEP 492 (async/await) and inlined generator expressions, but also the new support for Cython code coverage analysis with the coverage.py tool, that I blogged about already. Note that some Py3.5 features, such as the PEP 465 matrix multiplication operator, were already released early this year in Cython 0.22.
One of the greatest changes in Python 3.5, from a Cython perspective,
is the shift from generators and coroutines as builtin types to
making them ABCs, i.e. abstract base classes in the
collections.abc
module. This enables full interoperability of
Cython's own implementations with Python's builtin types. To make
this available in older Python versions, I created the backports_abc package for
applications to use. Basically, you would install it and then say
import backports_abc backports_abc.patch()
to get the Coroutine
and Generator
classes added to the
collections.abc
module if they are missing (i.e. in CPython
versions before 3.5). This allows Cython to register its internal
types with them and from that point on, any code using isinstance(x,
Generator)
or isinstance(x, Coroutine)
will be able to support
Cython's fast builtin types. If you find software packages that do
not support this yet because they do type checks instead of ABC
instance checks, please file a bug report with their projects to get
them adapted. Twisted comes to mind, for example, or Tornado.
[Update 2015-09-15]: The next Tornado release will have support for
Cython coroutines.
CPython 3.5 itself has already been fixed up, thanks to Yury Selivanov, to natively support Cython implemented generators and coroutines. This allows asynchronous code (currently, but not limited to, asyncio) to take advantage of Cython's speed, which can easily be several times faster than Python coroutines in practice.
What else is there in Cython 0.23? Well, inlined generator
expressions are back. This means that when you call builtin
functions like any()
or sorted()
on a generator expression,
Cython will inline its own (partial) implementation of that function
into the generator implementation and avoid calling back and forth
between the generator and its consumer. Instead, it will calculate
the final result inside of the generator and only return it in one
step at the very end. This can speed up the overall execution
considerably.
Another feature that I invested some time into is faster numeric
operations on Python's long
(Py3 int
) and float
types.
Arithmetic or shifting operations that involves integer or float
constants are now faster because they avoid double unpacking of the
operands and inline their constant parts.
On the C++ integration front, several minor issues have seen improvements and new standard declarations were added, which makes Cython 0.23 an even nicer and rounder tool for wrapping C++ code bases, or for making use of STL containers in Cython code. More C++ improvements are planned for the upcoming 0.24 release.
As usual, the complete list of features and fixes is much longer for such a large release. Hope you like it.