
Posted on • Originally published atgeekpython.in
Multi-Threaded Programs in Python Using threading Module
You may have heard the terms "parallelization" or "concurrency", which refer to scheduling tasks to run parallelly or concurrently (at the same time) to save time and resources. This is a common practice in asynchronous programming, wherecoroutines are used to execute tasks concurrently.
Threading in Python is used to run multiple tasks at the same time, hence saving time and resources and increasing efficiency.
Although multi-threading can save time and resources by executing multiple tasks at the same time, using it in code can lead to safety and reliability issues.
In this article, you'll learn what is threading in Python and how you can use it to make multiple tasks run concurrently.
What is Threading?
Threading, as previously stated, refers to the concurrent execution of multiple tasks in a single process. This is accomplished by utilizing Python'sthreading
module.
Threads are smaller units of the program that run concurrently and share the same memory space.
How to Create Threads and Execute Concurrently
Python provides a module calledthreading
that provides a high-level threading interface to create and manage threads in Python programs.
Create and Start Thread
A thread can be created using theThread
class provided by thethreading
module. Using this class, you can create an instance of theThread
and then start it using the.start()
method.
importthreading# Creating Target Functiondefnum_gen(num):forninrange(num):print("Thread: ",n)# Main Code of the Programif__name__=="__main__":print("Statement: Creating and Starting a Thread.")thread=threading.Thread(target=num_gen,args=(3,))thread.start()print("Statement: Thread Execution Finished.")
A thread is created by instantiating theThread
class with atarget
parameter that takes a callable object in this case, thenum_gen
function, and anargs
parameter that accepts a list or tuple of arguments, in this case,3
.
This means that you are tellingThread
to run thenum_gen()
function and pass3
as an argument.
If you run the code, you'll get the following output:
Statement: Creating and Starting a Thread.Statement: Thread Execution Finished.Thread: 0Thread: 1Thread: 2
You can notice that theStatement section of the code has finished before theThread
did.Why does this happen?
The thread starts executing concurrently with the main program and the main program does not wait for the thread to finish before continuing its execution. That's why the above code resulted in executing theprint
statement before the thread was finished.
To understand this, you need to understand the execution flow of the program:
First, the
"Statement: Creating and Starting a Thread."
print statement is executed.Then the thread is created and started using
thread.start()
.The thread starts executing concurrently with the main program.
The
"Statement: Thread Execution Finished."
print statement is executed by the main program.The thread continues and prints the output.
The thread and the main program run independently that's why their execution order is not fixed.
join() Method - The Saviour
Seeing the above situation, you might have thought then how to suspend the execution of the main program until the thread is finished executing.
Well, thejoin()
method is used in that situation, it doesn't let e*xecute the code further until the current thread terminates*.
importthreading# Creating Target Functiondefnum_gen(num):forninrange(num):print("Thread: ",n)# Main Code of the Programif__name__=="__main__":print("Statement: Creating and Starting a Thread.")thread=threading.Thread(target=num_gen,args=(3,))thread.start()thread.join()print("Statement: Thread Execution Finished.")
After creating and starting a thread, thejoin()
method is called on theThread
instance (thread
). Now run the code, and you'll get the following output.
Statement: Creating and Starting a Thread.Thread: 0Thread: 1Thread: 2Statement: Thread Execution Finished.
As can be seen, the"Statement: Thread Execution Finished."
print statement is executed after the thread terminates.
Daemon Threads
Daemon threads run in the background and terminate immediately whether they completed the work or not when the main program exits.
You can make a daemon thread by passing thedaemon
parameter when instantiating theThread
class. You can pass a boolean value to indicate whether the thread is a daemon (True
) or not (False
).
importthreadingimporttimedefdaemon_thread():whileTrue:print("Daemon thread is running.")time.sleep(1)print("Daemon thread finished executing.")if__name__=="__main__":thread1=threading.Thread(target=daemon_thread,daemon=True)thread1.start()print("Main program exiting.")
A thread is created by instantiating theThread
class passing thedaemon_thread
function inside it and to mark it as adaemon thread, thedaemon
parameter is set toTrue
.
Thedaemon_thread()
function is an infinite loop that prints a statement, sleeps for one second, and then again prints a statement.
Now when you run the above code, you'll get the following output.
Daemon thread is running.Main program exiting.
You can see that as soon as the main program exits, the daemon thread terminates.
At the time when thedaemon_thread()
function enters the loop, the concurrently running main program exits, and thedaemon_thread()
function never reaches the nextprint
statement as can be seen in the output.
threading.Lock - Avoiding Race Conditions
Threads, as you know, run concurrently in a program. If your program has multiple threads, they may share the same resources or the critical section of the code at the same time, this type of condition is calledrace conditions.
This is where theLock
comes into play, it acts like a synchronization barrier that prevents multiple threads from accessing the particular code or resources simultaneously.
The thread calls theacquire()
method to acquire theLock
and therelease()
method to release theLock
.
importthreading# Creating Lock instancelock=threading.Lock()data=""defread_file():globaldatawithopen("sample.txt","r")asfile:forinfoinfile:data+="\n"+infodeflock_task():lock.acquire()read_file()lock.release()if__name__=="__main__":thread1=threading.Thread(target=lock_task)thread2=threading.Thread(target=lock_task)thread1.start()thread2.start()thread1.join()thread2.join()# Printing the data read from the fileprint(f"Data:{data}")
First, aLock
is created using thethreading.Lock()
and store it inside thelock
variable.
An empty string is created (data
) for storing the information from both threads concurrently.
Theread_file()
function is created that reads the information from thesample.txt
file and adds it to the data.
Thelock_task()
function is created and when it is called, the following events occur:
The
lock.acquire()
method will acquire the Lock immediately when thelock_task()
function is called.If the
Lock
is available, the program will execute theread_file()
function.After the
read_file()
function finished executing, thelock.release()
method will release theLock
to make it available again for other threads.
Within theif __name__ == "__main__"
block, two threads are createdthread1
andthread2
that both runs thelock_task()
function.
Both threads run concurrently and attempt to access and execute theread_file()
function at the same time but only one thread can access and enter theread_file()
at a time due to theLock
.
The main program waits for both threads to execute completely because ofthread1.join()
andthread2.join()
.
Then using theprint
statement, the information present in the file is printed.
Data: Hello there! Welcome to GeekPython.Hello there! Welcome to GeekPython.
As can be seen in the output, one thread at a time reads the file. However, there were two threads that's why the file was read two times, first bythread1
and then bythread2
.
Semaphore Objects in Threading
Semaphore allows you to limit the number of threads that you want to access the shared resources simultaneously. Semaphore has two methods:
acquire()
: Thread can acquire the semaphore if it is available.When a thread acquires a semaphore,the semaphore's count decrement if it is greater than zero. If the count is zero, the thread waits until the semaphore is available.release()
: After using the resources, thethread releases the semaphore that results in an increment in the count. This means that shared resources are available.
Semaphore is used to limit access to shared resources, preventing resource exhaustion and ensuring controlled access to resources with limited capacity.
importthreading# Creating a semaphoresem=threading.Semaphore(2)defthread_task(num):print(f"Thread{num}: Waiting")# Acquire the semaphoresem.acquire()print(f"Thread{num}: Acquired the semaphore")# Simulate some workfor_inrange(5):print(f"Thread{num}: In process")# Release the semaphore when donesem.release()print(f"Thread{num}: Released the semaphore.")if__name__=="__main__":thread1=threading.Thread(target=thread_task,args=(1,))thread2=threading.Thread(target=thread_task,args=(2,))thread3=threading.Thread(target=thread_task,args=(3,))thread1.start()thread2.start()thread3.start()thread1.join()thread2.join()thread3.join()print("All threads have finished.")
In the above code,Semaphore
is instantiated with the integer value of2
which means two threads are allowed to run at the same time.
Three threads are created and all of them use thethread_task()
function. But only two threads are allowed to run at the same time, so two threads will access and enter thethread_task()
function at the same time, and when any of the threads releases the semaphore, the third thread will acquire the semaphore.
Thread 1: WaitingThread 1: Acquired the semaphoreThread 1: In processThread 1: In processThread 1: In processThread 1: In processThread 1: In processThread 2: WaitingThread 2: Acquired the semaphoreThread 1: Released the semaphore.Thread 2: In processThread 2: In processThread 3: WaitingThread 2: In processThread 3: Acquired the semaphoreThread 3: In processThread 2: In processThread 2: In processThread 2: Released the semaphore.Thread 3: In processThread 3: In processThread 3: In processThread 3: In processThread 3: Released the semaphore.All threads have finished.
Using ThreadPoolExecutor to Execute Tasks from a Pool of Worker Threads
TheThreadPoolExecutor
is a part ofconcurrent.features
module that is used to execute multiple tasks concurrently. UsingThreadPoolExecutor
, you can run multiple tasks or functions concurrently without having to manually create and manage threads.
fromconcurrent.futuresimportThreadPoolExecutor# Creating pool of 4 threadsexecutor=ThreadPoolExecutor(max_workers=4)# Function to evaluate square numberdefsquare_num(num):print(f"Square of{num}:{num*num}.")task1=executor.submit(square_num,5)task2=executor.submit(square_num,2)task3=executor.submit(square_num,55)task5=executor.submit(square_num,4)# Wait for tasks to complete and then shutdownexecutor.shutdown()
The above code creates aThreadPoolExecutor
with a maximum of4
worker threads which means the thread pool can have a maximum of 4 worker threads executing the tasks concurrently.
Four tasks are submitted to theThreadPoolExecutor
using thesubmit
method with thesquare_num()
function and various arguments. This will execute the function with specified arguments and prints the output.
In the end, theshutdown
method is called, so thatThreadPoolExecutor
shutdowns after the tasks are completed and resources are freed.
You don't have to explicitly call theshutdown
method if you createThreadPoolExecutor
using thewith statement.
fromconcurrent.futuresimportThreadPoolExecutor# Taskdefsquare_num(num):print(f"Square of{num}:{num*num}.")# Using ThreadPoolExecutor as context managerwithThreadPoolExecutor(max_workers=4)asexecutor:task1=executor.submit(square_num,5)task2=executor.submit(square_num,2)task3=executor.submit(square_num,55)task5=executor.submit(square_num,4)
In the above code, theThreadPoolExecutor
is used with thewith
statement. When thewith
block is exited, theThreadPoolExecutor
is automatically shut down and its resources are released.
Both codes will produce the same result.
Square of 5: 25.Square of 2: 4.Square of 55: 3025.Square of 4: 16.
Common Function in Threading
Thethreading
module provides numerous functions and some of them are explained below.
Getting Main and Current Thread
Thethreading
module has amain_thread()
and acurrent_thread()
function which is used to get the main thread and the currently running thread respectively.
importthreadingdeftask():for_inrange(2):# Getting the current thread nameprint(f"Current Thread:{threading.current_thread().name} is running.")# Getting the main thread nameprint(f"Main thread :{threading.main_thread().name} started.")thread1=threading.Thread(target=task)thread2=threading.Thread(target=task)thread1.start()thread2.start()thread1.join()thread2.join()print(f"Main thread :{threading.main_thread().name} finished.")
Because themain_thread()
andcurrent_thread()
functions return aThread
object,threading.main_thread().name
is used toget the name of the main thread andthreading.current_thread().name
is used toget the name of the current thread.
Main thread : MainThread started.Current Thread: Thread-1(task) is running.Current Thread: Thread-1(task) is running.Current Thread: Thread-2(task) is running.Current Thread: Thread-2(task) is running.Main thread : MainThread finished.
Monitoring Currently Active Threads
Thethreading.enumerate()
function is used to return the list ofThread
objects that are currently running. This includes the main thread even if it is terminated and excludes terminated threads and threads that have not started yet.
If you want to get the number ofThread
objects that are currently alive, you can utilize thethreading.active_count()
function.
importthreadingdeftask():print(f"Current Thread :{threading.current_thread().name} is running.")# Getting the main thread nameprint(f"Main thread :{threading.main_thread().name} started.")threads_list=[]for_inrange(5):thread=threading.Thread(target=task)thread.start()threads_list.append(thread)# Getting the active thread countprint(f"\nActive Thread Count:{threading.active_count()}")forthreadinthreads_list:thread.join()print(f"Main thread :{threading.main_thread().name} finished.")# Getting the active thread countprint(f"Active Thread Count:{threading.active_count()}")# Getting the list of active threadsforactiveinthreading.enumerate():print(f"Active Thread List:{active.name}")
Output
Main thread : MainThread started.Current Thread : Thread-1(task) is running.Active Thread Count: 2Current Thread : Thread-2(task) is running.Active Thread Count: 2Current Thread : Thread-3(task) is running.Active Thread Count: 2Current Thread : Thread-4(task) is running.Active Thread Count: 2Current Thread : Thread-5(task) is running.Active Thread Count: 1Main thread : MainThread finished.Active Thread Count: 1Active Thread List: MainThread
Getting Thread Id
importthreadingimporttimedeftask():print(f"Thread{threading.get_ident()} is running.")time.sleep(1)print(f"Thread{threading.get_ident()} is terminated.")print(f"Main thread started.")threads_list=[]for_inrange(5):thread=threading.Thread(target=task)thread.start()threads_list.append(thread)forthreadinthreads_list:thread.join()print(f"Main thread finished.")
Every thread running in a process is assigned an identifier and thethreading.get_ident()
function is used to retrieve the identifier of the currently running thread.
Main thread started.Thread 9824 is running.Thread 7188 is running.Thread 4616 is running.Thread 3264 is running.Thread 7716 is running.Thread 7716 is terminated.Thread 9824 is terminated.Thread 7188 is terminated.Thread 4616 is terminated.Thread 3264 is terminated.Main thread finished.
Conclusion
A thread is a smaller unit in the program that is created using thethreading
module in Python. Threads are tasks or functions that you can use multiple times in your program to execute concurrently to save time and resources.
In this article, you've learned:
What is threading and how do you create and start a thread
Why
join()
method is usedWhat aredaemon threads and how to create one
How to Lock threads to avoid race conditions
Howsemaphore is used to limit the number of threads that can access the shared resources at the same time.
How you can execute a group of tasks using the
ThreadPoolExecutor
without having to create threads.Some common functions provided by the
threading
module.
🏆Other articles you might be interested in if you liked this one
✅Comparing the accuracy of 4 pre-trained deep learning models?
✅What are coroutines in Python and how do use them in asynchronous programming?
✅Async/Await in Python using the asyncio module?
✅How to structure a Flask app using Flask Blueprint?
✅Upload and display images on the frontend using Flask in Python.
✅How to connect the SQLite database with the Flask app using Python?
That's all for now
Keep Coding✌✌
Top comments(0)
For further actions, you may consider blocking this person and/orreporting abuse