Stackless-Python

Author:Richard Tew (richard.m.tew@gmail.com)
Release:3.5.6
Date:September 17, 2018

Stackless-Python is an enhanced version of thePython® programming language[1].It allows programmers to reap the benefits of thread-based programming withoutthe performance and complexity problems associated with conventional threads.The microthreads that Stackless adds to thePython® programming language are a cheap and lightweightconvenience, which if used properly, can not only serve as a way to structurean application or framework, but by doing so improve program structure andfacilitate more readable code.

If you are reading this text as part of a version of aPython® interpreter you have installed,then you have installedStackless-Python rather than standardPython®[2].

Overview / API

Unless actual use is made of the enhanced functionality that Stackless addsto standardPython®, Stackless will behave exactly the same as standardPython® would and is usedin exactly the same way. This functionality is exposed as a frameworkthrough thestackless module.

What you need to know

Stackless-Python provides a minimal framework and it is not accompanied by anysupport functionality, that would otherwise address the common needs that mayarise when building a more targeted framework around it.

Blocking operations

When operations are invoked that block thePython® interpreter, the user needsto be aware that this inherently blocks all running tasklets. Until the taskletthat engaged that operation is complete, thePython® interpreter and thereforescheduler is blocked on that operation and in that tasklet. Operations thatblock the interpreter are often related to synchronous IO (file readingand writing, socket operations, interprocess communication and more), althoughtime.sleep() should also be kept in mind. The user is advised to chooseasynchronous versions of IO functionality.

Some third-party modules are available that replace standard libraryfunctionality with Stackless-compatible versions. The advantage of thisapproach is that other modules which use that standard functionality,also work with Stackless when the replacement is installed. TheStackless socketmodule is the most commonly used replacement module.

Exceptions

Certain exceptions that may occur within tasklets, are expected to reach allthe way up the call stack to the scheduler. This means that naively usingtheexcept statement may result in hard to track down problems.

A description of the problem with bareexcept statements can be read inthe documentation for theTaskletExit exception.

Debugging

The Stackless scheduling mechanism changes the way thePython® debugging hookswork so that debugging hooks are set per-tasklet rather than per-thread.However, very few debuggers, certainly none of those in the standard librarytake this into account. As a result of this, debugging is unlikely to workwith special handling being worked into your use of Stackless.

A description of this problem can be read in the Stacklessdebuggingdocumentation.

External resources

There are a range of resources available outside of this document. However most of them arenowadays quite dated:

History

Continuations are a feature that require the programming language they arepart of, to be implemented in a way conducive to their presence. In order toadd them to thePython® programming language, Christian Tismer modified itextensively. The primary way in which it was modified, was to make it Stackless.And so, his fork of standardPython® was namedStackless-Python.

Now, storing data on the stack locks execution on an operating system thread tothe current executing functionality, until that functionality completes and thestack usage is released piece by piece. In order to add continuations to standardPython®,that data needed to be stored on the heap instead, therefore decoupling theexecuting functionality from the stack. With the extensive changes this requiredin place, Christianreleased Stackless Python.

Maintaining the fork of a programming language is a lot of work, and when theprogramming language changes in ways that are incompatible with the changes inthe fork, then that work is sigificantly increased. Over time it becameobvious that the amount of changes to standardPython® were too much weight to carry,and Christian contemplated a rewrite of Stackless. It became obvious that asimpler approach couldbe taken, where Stackless was no longer stackless and no longer had continuations.

Following the rewrite, a framework was designed and addedinspired by coming fromCSP and theLimbo programminglanguage. From this point on, Stackless was in a state where it contained theminimum functionality to give the benefits it aimed to provide, with theminimum amount of work required to keep it maintained.

A few years later in 2004, while sprinting on Stackless in Berlin, Christian andArmin Rigocame up with a way totake the core functionality of Stackless and build an extension module that providedit. This was the creation of greenlets, which are very likely a more populartool than Stackless itself today. The greenlet source code in practice can beused as the base for green threading functionality not just in thePython® programming language, but inother programming languages and projects.

WithStackless-Python a solid product, Christian’s focus moved onto otherprojects,PyPy among them. One of his interests in PyPy wasa proper implementation of the Stackless functionality, where it could beintegrated as a natural part of anyPython® interpreter built.

For a while,Stackless-Python languished, with no new versions to match thereleases of standardPython® itself. Then in 2006, CCP sent Kristján Valur Jonsson andRichard Tew to PyCon wherethey sprintedwith the aid of Christian Tismer. The result was an up to date release ofStackless-Python.From this point in time, maintaining and releasingStackless-Pythonhas been undertaken by Richard and Kristján. A few years later Anselm Kruisjoined the team.

[1]“Python” and the Python logos are trademarks or registered trademarks of thePython Software Foundation, used byStackless-Python with permission from the Foundation.Seehttp://www.python.org/psf/trademarks/ for details.
[2]With the term “standardPython®” we refer to the reference implementation of thePython® programming languagethat is released by thePython Software Foundation onhttp://www.python.org.