Performance

Every Python developer eventually hits a wall where their code needs to do more than one thing “at once,” and then discovers that Python offers not one but three completely different answers: threading, multiprocessing, and asyncio. They all live in the standard library, they all let you write code that overlaps in time, and their APIs even look superficially similar. But they solve different problems, and picking the wrong one doesn’t just underperform, it can silently fail to help at all. ...

I still remember the first time a generator saved a script from getting OOM-killed. I was processing a multi-gigabyte log file, and the “obvious” version of the code read every line into a list before filtering it. It worked fine on my laptop with a 10MB sample file and then fell over the moment it hit the real dataset in production. Swapping a list comprehension for a generator expression fixed it in about thirty seconds. That asymmetry, huge payoff for a tiny syntax change, is why generators are one of the first things I reach for when data gets big or infinite. ...