Python Has A Major Scalability Flaw! - System Design

Scaling applications is a necessity in today’s tech world.

An application’s use can expand exponentially and hence applications should be developed with the aim to handle more and more requests per second as this happens.

Before talking about scalability, we need to be familiar with a few computer science concepts.

Let’s talk about them!

What is a Process? : A Process is the running instance of a program/ application. A program can have multiple instances of it (Processes) running at the same time.

What is a Thread? - A Thread represents the actual processor instructions that are being executed in the context of a Process. Each Process has at least one thread executing its instructions.

What is Multi-threading? - In Multi-Threading, multiple threads work together, executing instructions within the context of a process in parallel. Threads within a process share state information, memory, and many other attributes.

Multi-Threaded Application

In a multi-threaded program, the application process can utilize different cores in the processor of the computer’s CPU and run multiple threads in parallel.

This is done to improve computational efficiency / reduce execution time. This forms the basis of Vertical scaling and the next step can be to add in more and faster processors / executing cores. But this has limitations. You cannot have ever-increasing processing power to upgrade to and this becomes expensive!

To know more about scaling: Checkout this blog on Horizontal vs Vertical Scaling - https://www.thegeekyminds.com/post/all-about-scaling-a-system-horizontal-scaling-vs-vertical-scaling-system-design

Cost of Horizontal vs. Vertical Scaling

Multi-Threading in Python Applications

Let’s write a function called countdown and execute it

This function takes 1.946 seconds to finish its execution.

Let’s use multi-threading and execute this function again.

One would expect this to run twice as fast because we have used two threads to execute this function.

Surprisingly, this does not work the way we expect it to!

Do you know why?

The reason for this is the Python GIL (Global Interpreter Lock).

Let’s talk about where it comes from.

CPython

CPython is the default and most widely used reference implementation of Python.