It was really nice to hear Cython being mentioned in pretty much every
second talk at this year's EuroSciPy in Cambridge, either directly or
by promoting tools that were themselves written in Cython. It seems
to have become a key cornerstone in Python's ecosystem.
For me, this has been an intense conference summer overall, so I only
just noticed that I hadn't written up anything about the long list of
changes in Cython 0.23 yet. Well, apart from the changelog :). So
here it goes.
Cython 0.23 is a major feature release. It
implements the new language features of the new Python 3.5, but with
the usual backports to make them available in Python 2.6 and later.
My personal feature favourites are PEP 492 (async/await) and inlined generator
expressions, but also the new support for Cython code coverage
analysis with the
coverage.py tool, that I
blogged about already. Note that some Py3.5 features, such as the
PEP 465 matrix multiplication operator, were already released
early this year in Cython 0.22.
One of the greatest changes in Python 3.5, from a Cython perspective,
is the shift from generators and coroutines as builtin types to
making them ABCs, i.e. abstract base classes in the
collections.abc module. This enables full interoperability of
Cython's own implementations with Python's builtin types. To make
this available in older Python versions, I created the backports_abc package for
applications to use. Basically, you would install it and then say
to get the Coroutine and Generator classes added to the
collections.abc module if they are missing (i.e. in CPython
versions before 3.5). This allows Cython to register its internal
types with them and from that point on, any code using isinstance(x,
Generator) or isinstance(x, Coroutine) will be able to support
Cython's fast builtin types. If you find software packages that do
not support this yet because they do type checks instead of ABC
instance checks, please file a bug report with their projects to get
them adapted. Twisted comes to mind, for example, or Tornado.
[Update 2015-09-15]: The next Tornado release will have support for
CPython 3.5 itself has already been fixed up, thanks to Yury
Selivanov, to natively support Cython implemented generators and
coroutines. This allows asynchronous code (currently, but not limited
to, asyncio) to
take advantage of Cython's speed, which can easily be several times
faster than Python coroutines in practice.
What else is there in Cython 0.23? Well, inlined generator
expressions are back. This means that when you call builtin
functions like any() or sorted() on a generator expression,
Cython will inline its own (partial) implementation of that function
into the generator implementation and avoid calling back and forth
between the generator and its consumer. Instead, it will calculate
the final result inside of the generator and only return it in one
step at the very end. This can speed up the overall execution
Another feature that I invested some time into is faster numeric
operations on Python's long (Py3 int) and float types.
Arithmetic or shifting operations that involves integer or float
constants are now faster because they avoid double unpacking of the
operands and inline their constant parts.
On the C++ integration front, several minor issues have seen
improvements and new standard declarations were added, which makes
Cython 0.23 an even nicer and rounder tool for wrapping C++ code
bases, or for making use of STL containers in Cython code. More C++
improvements are planned for the upcoming 0.24 release.
As usual, the complete list of features and fixes is much
longer for such a large release. Hope you like it.