Student Login

"Python is not good at ..."

Suddenly, without warning, a clever reader asked me:

"I just overheard a colleague say: Python is good at many things... but what it is not good at is parallel processing. Is this true?"

The short answer: No, it's not true.

But the longer answer is complicated.

What makes it complicated is the GIL. The global interpreter lock.

There's a lot to say about this. But one key insight is understanding the difference between Python threads, and OS threads.

When you write a program using threads in a language like C, that's using operating-system level threads. That gives real parallelism, provided the operating system actually supports that.

But Python's runtime - thanks to the GIL - only lets your Python program run one thread at a time. I call these "Python threads", and they're certainly different than OS threads.

Does that matter?

If your app is doing something IO- or network-bound, like scraping web pages, probably not. Since each thread would just be waiting most of the time anyway.

For anything CPU bound, though, it matters a lot. And that's probably what that colleague was thinking of.

But:

Python provides ways to get around the GIL. Like the multiprocessing module. Or, for the toughest cases, C extensions.

In fact, when you learn about all of Python's tools for concurrency, and how to use them...

The GIL turns out to not be a significant limitation, for many engineering domains.

So that's the longer answer.

For Teams Bootcamp