Python Object Sharing (POSH)

16.12.2002

Abstract

Language support for threads was introduced at a late stage in the Python development process. To accomodate that the majority of Python code was not thread-safe, an approach using a global interpreter lock was introduced. This lock has remained a major scalability bottleneck when executing multi-threaded Python programs on multi-processor architectures. A common workaround to achieve parallelism on such architectures is to use multiple processes instead of multiple threads. Typically, these types of applications use ad-hoc shared memory or messaging APIs for interprocess communication.

This thesis presents POSH, an extension to Python that combines the advantages of the shared memory programming model in a multi-threaded Python program with the scalability of a multi-process Python program.

POSH allows Python objects to be transparently shared across multiple processes using shared memory. Shared objects are indistinguishable from regular objects, as seen by Python code. The POSH system is implemented as an extension module and requires no changes to the Python runtime.