Most of the Python code that I write isn’t part of an exotic framework or huge application. Instead, it’s usually related to a mundane data analysis task involving a CSV file. It isn’t glamorous, but Python is an effective tool at getting the job done without too much fuss. When working on such problems, I prefer to not worry too much about low-level details (I just want the final answer). However, if you use Python for manipulating a lot of data, you may find that your scripts use a large amount of memory. In this article, I’m going to peek under the covers of how memory gets used in a Python program and explore options for using it more efficiently. I’ll also look at some techniques for exploring and measuring the memory use of your programs. Disclosure: Python 3 is assumed for all of the examples, but the underlying principles apply equally to Python 2.
