Python Tutorial

This tutorial will be focused on one of important Python's topics, GIL. We will also cover how the GIL impacts the Python programs' performance with the code implementation. Before diving into this topic, let's have a basic idea of the GIL.

GIL or Global Interpreter Lock

Python Global Interpreter Lock or GIL is an important part of multithreading programming. It is a type of process lock used when working with multiple processes. It gives the control to only one thread. Generally, Python uses a single thread to run a single process. We get the same performance result of the single-threaded and multi-threaded processes using the GIL. It restricts achieving multithreading in Python because it prevents the threads and works as a single thread.

Note - Python doesn't support multithreading because threading packages couldn't let us use the multiple CPU cores.

Why Python Developers Uses GIL?

Python provides the unique reference counter feature, which is used for memory management. The reference counter counts the total number of references made internally in Python to assign a value to a data object. When the reference counts reach zero, the assigned memory of the object is released. Let's see the below example.

Example -

import sys
a = []
b = a
sys.getrefcount(a)

The main concern with the reference count variable is that it can be affected when two or three threads trying to increase or decrease its value simultaneously. It is known as race condition. If this condition occurs, it can be caused leaked memory that is never released. It may crash or bugs in the Python program.

GIL helps us remove such a situation by using the locks to all shared data structures across threads so that they are not changed inconsistently. Python provides an easy way to implement the GIL as it deals with thread-safe memory management. GIL requires offering a single lock to a thread for processing in Python. It increases the performance of a single-threaded program as only one lock requires to be handled. It also helps to make any CPU-bound program and prevents the deadlocks condition.

The Impact on Multi-threaded Python Programs

There is a difference between CPU-bounds in their performance and I/O bound for a typical Python program or any computer program. CPU-bound programs are generally pushed the CPU to its limits. These programs are generally used for mathematical computation such as matrix multiplications, searing, image processing, etc.

I/O bound programs are those programs that spend time to get input/output that can be generated by the user, file, database, network, etc. Such programs have to wait for some significant amount of time until the source provides the input. On the other hand, the source also has its own processing time. For example - a user is thinking about what to enter as an input.

Let's understand the following example.

Example -

import time
from threading import Thread
COUNT = 100000000

def countdown(num):
    while num>0:
        num -= 1

start_time = time.time()
countdown(COUNT)
end_time = time.time()

print('Time taken in seconds -', end_time - start_time)

Output:

Time taken in seconds - 7.422671556472778

Now we modify the above code by running the two threads.

Example - 2:

import time
from threading import Thread

COUNT = 100000000

def countdown(num):
    while num>0:
        num -= 1

thread1 = Thread(target=countdown, args=(COUNT//2,))
thread2 = Thread(target=countdown, args=(COUNT//2,))

start_time = time.time()
thread1.start()
thread2.start()
thread1.join()
thread2.join()
end_time = time.time()
print('Time taken in seconds -', end_time - start_time)

Output:

Time taken in seconds - 6.90830135345459

As we can see that both codes took the same time to finish. GIL prevented the CPU-bound threads from executing in parallel in the second code.

Why Hasn't the GIL Been Removed Yet?

Many programmers have a complaint regarding this, but Python cannot bring the changes as significant as the removal of GIL. Another reason is that GIL is not improved as of now. If it changes in Python 3, it will create some serious issues. Instead of removing GIL, the GIL concept can improve. According to Guido van Rossom -

"I'd welcome a set of patches into Py3k only if the performance for a single-threaded program (and for a multi-threaded but I/O-bound program) does not decrease".

There are also many methods available that solve the same problem solved by the GIL, but there are hard to implement.

How to Deal with Python's GIL

Using multiprocessing is the most suitable way to prevent the program from GIL. Python offers various interpreters for each process to run, so in that scenario, the single thread is provided to each process in multiprocessing. Let's understand the following example.

Example -

from multiprocessing import Pool
import time

COUNT = 50000000
def countdown(num):
    while num>0:
        num -= 1

if __name__ == '__main__':
    pool = Pool(processes=2)
    start_time = time.time()
    r1 = pool.apply_async(countdown, [COUNT//2])
    r2 = pool.apply_async(countdown, [COUNT//2])
    pool.close()
    pool.join()
    end_time = time.time()
    print('Time taken in seconds -', end_time - start_time)

Output:

Time taken in seconds - 3.3707828521728516

It may seem a decent performance is increased but process management has its own overheads and multiple processes are heavier than multiple threads.

Conclusion

In this tutorial, we have discussed the GIL and how we can use it. It gives the control to single thread to execute at time. This tutorial also covered why GIL is important for Python programmers.

Next TopicBasic Python for Java Developers

← prev next →