繁体 English 中英

Flask没有在Centos7中释放内存

[英]Flask is not releasing memory in Centos7

原文 2019-11-26 08:22:10 7 0 python/ flask/ memory-leaks/ centos7/ joblib

我已经创建了一个简单flask方法，其装载和卸载的字典对象（含有LogisticRegression和TfidfVectorizer使用了解到sklearn从使用该磁盘） joblib 。 这是函数：

def load_unload_model():
    a_model_path = 'model.bin'
    print('Memory before loading model:', get_memory_usage())
    model_obj = joblib.load(a_model_path)
    print('Memory after loading model:', get_memory_usage())
    del model_obj
    gc.collect()
    print('Memory after unloading model:', get_memory_usage())

当我使用Flask方法迭代调用此函数时，进程的内存随着每个请求而不断增加。 如果我从Main运行相同的功能，则加载之前和卸载之后的内存将保持不变。 在Windows上不会发生此问题。

这是完整的代码：

import os, gc, psutil, sys
from flask import Flask
import joblib
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

app = Flask(__name__)
global_request_counter = 0


def get_memory_usage():
    a_process = psutil.Process(os.getpid())
    return str(round(a_process.memory_info().rss/(1000*1000))) + 'MB'

def load_unload_model():
    a_model_path = 'model.bin'
    a_model_path2 = 'model2.bin'
    print('Memory before loading model:', get_memory_usage())
    model_obj = joblib.load(a_model_path)
    model_obj2 = joblib.load(a_model_path2)
    print('Memory after loading model:', get_memory_usage())
    del model_obj
    del model_obj2
    gc.collect()
    print('Memory after unloading model:', get_memory_usage())


@app.route('/prediction_service/load_model', methods=['GET'])
def load_model_api():
    global global_request_counter
    print('Request:', global_request_counter+1, 'Started')
    load_unload_model()
    print('Request:', global_request_counter+1, 'Ended')
    print('-'*50)
    global_request_counter += 1
    return "Execution Successful"


if __name__ == "__main__":
    run_flask = 1

    if run_flask == 1:
        app.run(host="0.0.0.0", port="8060")
    else:
        for i in range(15):
            print('Iteration:', i+1, 'Started')
            load_unload_model()
            print('Iteration:', i+1, 'Completed')
            print('-'*50)

这是使用Flask在Centos7上的输出：

请求：1开始
加载前内存：71MB
加载型号后的内存：134MB
卸载后内存型号：83MB
请求：1结束