How to delete objects to increase free memory in python?

Question

My python application reads data from files and stores these data in dictionaries during start up (dictionaries are properties of data reader classes). Once the application starts and the read data is used, these data in the dictionaries are no longer needed. However, they consume large amount of memory. How do I delete these dictionaries to free the memory?

For example:

class DataReader():
    def __init__(self, data_file):
        self.data_file = data_file

    def read_data_file_and_store_data_in_dictionary():
        self.data_dictionary = {}
        for [data_name, data] in self.data_file:
             self.data_dictionary[data_name] = data

class Application():
    def __init__(self, data_file):
        self.data_reader = DataReader()
        self.data_reader.read()

    def start_app(self):
        self.use_read_data()

After application is started, self.data_dictionary is no longer needed. How do I delete self.data_dictionary permanently?

Answer 1

Use the del statement

del self.data_dictionary  # or del obj.data_dictionary

Note this will only delete this reference to the dictionary. If any other references still exist for the dictionary (say if you had done d = data_reader.data_dictionary and d still references data_dictionary ) then the dictionary will not be freed from memory. This also includes any references to d.keys(), d.values(), d.items() .

Only when all references have been removed will the dictionary finally be released.

Answer 2

Using Python, you should not care about memory management.

Python has an excellent garbage collector , which counts for each object the reference in the code.
If there are no references, the object will be unallocated.

In your case, if the memory is not free after you're done using it, it means that the object can be still used in your program. If you delete it and then try to call it, you will get a ReferenceError .

Someone in other answers is suggesting to use del , but it will only delete the variable name.

My suggestion is to ensure that your code does not actually call the object anymore, and if it does, manipulate your data accordingly (use a lightweight db , save them on local hard drive, ...) and retrieve them when needed. If your big dictionaries are class parameters of a class which is still used, but doesn't need the dicts anymore, you should take those dicts outside the class (maybe referencing a new class, which only manages the dicts). In this Q&A you will find useful tips for optimizing memory usage.

You can read this article to have a really interesting dive into the Python's garbage collector

Answer 3

How about having the data in a smaller scope?

class Application():
    def __init__(self, data_file):
        self.use_read_data(DataReader(data_file).read())

After application is started, self.data_dictionary is no longer needed

If you do not need the data for the whole lifetime of the application then you shouldn't be storing it in an instance attribute.

Choose the right scope and you won't need to care about deleting variables.

Answer 4

del will delete the reference to the object, however it could still be on memory. In that case, the garbage collector ( gc.collect([generation]) ) will free the memory:

https://docs.python.org/2.7/library/gc.html

import gc

[...]
# Delete reference
del object
# Garbage collector
gc.collect()
[...]