Python and Memory

I found myself thinking about an interesting problem I ran into a few years ago. I was wondering why an open source metrics collection system seemed to have a relatively low performance ceiling when relaying metrics.

After much troubleshooting, I found that the performance issue resided in attempts at splitting a list: when it had thousands of messages, it would pull off some messages from the front of a list, then split the list, and the way it was doing this was inefficient. In the end I found some improvement in using a deque, but the problem has left me with continuing questions about some of these oddities in how Python does its memory management.