You may ask why should I want to run python code in Delphi threads. The main reason is that you want to keep the main thread of your application running and serving the user, instead of blocking while waiting for python code to finish. This is very important in GUI applications.

Up until Python 12, python has supported thread concurrency but not thread parallelism. This means that you could not run two threads executing python code in parallel, even if your computer had multiple CPU cores. At any point in time only one thread could execute python code. Python 12 introduces a way to run python threads in parallel. ThisP4D discussion provides more details about this new feature of python 12, explains its limitations and shows how to use it with P4D.

The reason for the lack of parallelism is the infamousGlobal Interpreter Lock (aka GIL). You need to get hold of the GIL to run python code or call any of the python API. The P4DTPythonThread class encapsulates all the complexities related to the management of GIL.

When PythonEngine loads the python DLL, the main thread holds the GIL. Before you can run python code in threads you must release the GIL. TPythonThread has two class methods that allow you to release the GIL and then acquire back.

classprocedurePy_Begin_Allow_Threads;classprocedurePy_End_Allow_Threads;

Py_Begin_Allow_Threads saves the thread state and releases the GIL, whilstPy_End_Allow_Threads tries to get hold of GIL and restore the previous thread state. You should only callPy_Begin_Allow_Threads if you alread hold the GIL. Otherwise a deadlock will occur.

You should callPy_Begin_Allow_Threads in your main thread, after the python DLL was loaded, for example in your FormCreate event handler. Also you can callPy_End_Allow_Threads when you are done executing python in threads.

After releasing the GIL in the main thread, the steps required to run python code in threads (including the main thread) are:

1. Acquire the GIL
2. Run python code
3. Release the GIL

TPythonThread, a subclass of Delphi'sTThread, encpsulates the complexity related to the above steps. You need to subclassTPythonThread and overwrite the abstract methodExecuteWithPython. Then you create and run instances of this subclass in the same way you run other Delphi threads. Demo 33, provides an example of how useTPythonThread.

The main property you need to know about isThreadExecMode. It can take one of the following values (enumeration):

1. emNewState (default)
2. emNewInterpreter
3. emNewInterpreterOwnGIL (only available with Python 12)

In most cases what you want is the default value. emNewInterpreter creates a so called subinterpreter, which is an isolated environnment for running python code. Global variables and module imports from other threads are not available in the subinterpreter. Finally, emNewInterpreterOwnGIL supports running python code in parallel, but it is only available in python 12 or later and has certain limitations discussedhere.

Instead of subclassingTPythonThread and overwriting itsExecuteWithPython method, you can use the newly introducedThreadPythonExec function, which is a wrapper aroundTPythonThread that takes anonymous methods as arguments. Its signature is:

procedureThreadPythonExec(ExecuteProc : TProc; TerminateProc : TProc =nil;  WaitToFinish: Boolean = False; ThreadExecMode : TThreadExecMode = emNewState);

Here is an example of how to use it from your main thread:

ThreadPythonExec(procedurebegin  GetPythonEngine.ExecString(SomePythonScript);end);

The optionalTerminateProc, if provided it is executed in the main thread when the thread finishes usingTThread.Queue. This is useful, if say you want to update your GUI when the thread exits. TheThreadExecMode parameter has been explained above. IfWaitToFinish is True, blocks the main thread until the python thread finishes. This beats the objective of running python code in threads though.

If you are using the System.Threading module you may want to run python code inside a task. Also you may want to run python code in threads other than those descending fromTPythonThread. You still have to go through the steps mentioned above:

1. Acquire the GIL
2. Run python code
3. Release the GIL

The newly introduced functionSafePyEngine simplifies the above steps. This is an example of how to run python code inside a threading Task.

Task := TTask.Create(procedurevar  Py: IPyEngineAndGIL;begin  Py := SafePyEngine;  Py.Engine.ExecString(SomePythonScript);end);Task.Start;

SafePyEngine acquires the GIL and returns an Interface. Upon destruction, the Interface releases the GIL.

Python threads created using python's threading module are managed by python. Python switches execution between them at regular time intervals. This is not the case for Delphi threads running python code. Acquiring the GIL is a blocking operation. It blocks until the GIL becomes available. So, it is important that you release the GIL ASAP giving the opportunity to other threads to run python code. Here is an example of how to do that.

var Thread := TThread.CreateAnonymousThread(procedurevar  Py: IPyEngineAndGIL;begin  Py := SafePyEngine;// Gets the GIL which is automatically released at the end of the procedure.whilenot TThread.Current.Terminateddobegin// Run some python code    TPythonThread.Py_Begin_Allow_Threads;// Release the GIL// Issue an HTTP Request and wait for the response// Other threads can run python code    TPythonThread.Py_End_Allow_Threads;// Acquire the GIL// Run some more python code    TPythonThread.Py_Begin_Allow_Threads;// Release the GIL// Call TThread.Synchronize to update the GUI// Other threads can run python code    TPythonThread.Py_End_Allow_Threads;// Acquire the GILend;end);Thread.Start;