Gui(pyqt5) lags a lot, and then python crashing

Question

I'm trying to do GUI / control centre for my program. And it works fine, but not for a long time. So, I have got two lcd numbers, and I want to refresh widgets as fast as possible.

Code has got 3 parts:

1. Main
2. Functions to access realtime data
3. Gui made in Qt Designer

First:

import sys  
from PyQt5 import QtWidgets, QtCore
import untitled  
import tweepy
from configuration import *
import datetime
import pandas as pd
import time
from bitmex import bitmex
from configuration import *
from strategy import Strategy
from trader import Trader
import json
import operator
import sys
from xbt import get_data


class ExampleApp(QtWidgets.QMainWindow, untitled.Ui_MainWindow):

    def __init__(self):
        # 
        super().__init__()
        self.setupUi(self)  # initial UI
        self.pushButton.clicked.connect(self.tweet)  # tweet if pressed

    def tweet(self):

        auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
        auth.set_access_token(access_token, access_token_secret)

        api = tweepy.API(auth)

        # If the authentication was successful, you should
        # see the name of the account print out
        print(api.me().name)

        # If the application settings are set for "Read and Write" then
        # this line should tweet out the message to your account's
        # timeline. The "Read and Write" setting is on https://dev.twitter.com/apps
        api.update_status(status='Tweet about last order')

Here is the main part whats for my opinion causing it. I think it's not good to use timer in main, but i found that solution on StackOverflow. !

def main():

    app = QtWidgets.QApplication(sys.argv)  
    window = ExampleApp()  
    window.setupUi(window)

    def update_label():
        current_time = str(datetime.datetime.now().time())
        window.label_3.setText(current_time)
    def update_lcd_delta():
        price = str(get_data.get_data_gui())
        window.lcdNumber.display(price)

    def actual_price_lcd():
        actual_price_xbt = str(get_data.price_xbt())
        window.lcdNumber_2.display(actual_price_xbt)

    timer = QtCore.QTimer()
    timer.timeout.connect(update_label) #update_label with current time
    timer.start(10000)  # every 10,000 milliseconds
    timer2 = QtCore.QTimer() #update lcd number with another number
    timer2.timeout.connect(update_lcd_delta)  
    timer2.start(10000)

    timer3 = QtCore.QTimer()
    timer3.timeout.connect(actual_price_lcd)  #update 
    timer3.start(100)

    window.show()  
    app.exec_()  


if __name__ == '__main__':  
    main()  

2nd part for getting data (it works fine):

import pandas as pd

from bitmex import bitmex
from configuration import *


class get_data():

    def get_data_gui():

        client = bitmex(test=TEST_EXCHANGE, api_key=API_KEY, api_secret=API_SECRET)


        prices_now = pd.DataFrame(client.Trade.Trade_getBucketed(  # get 24 candles with unfinished one
                binSize='1h',
                partial=True,
                symbol='XBTUSD',
                count=24,
                reverse=True
            ).result()[0])
        prices_now.set_index(['close'], inplace=True)
        delta = (prices_now['high'].max() - prices_now['low'].min()) / prices_now['low'].min()

        return delta

    def price_xbt():

        client = bitmex(test=TEST_EXCHANGE, api_key=API_KEY, api_secret=API_SECRET)

        order_book = pd.DataFrame(client.OrderBook.OrderBook_getL2(  # get orderbook
                symbol='XBTUSD',
                depth=2
                # 0 is full depth, using 200-300 we can adjust speed from 0.5 seconds to 0.1 (depends from bitmex)
            ).result()[0])
        print(order_book)
        order_book.set_index(['price', 'size', 'side'], inplace=False)
        price_first_bid = order_book.loc[order_book[order_book.side == 'Buy'].index[0], 'price']
        print(price_first_bid)
        return price_first_bid

As I understood, I have memory leak. How can I fix that?

This is the crash info:

    REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Accelerate framework               128K        1 
Activity Tracing                   256K        1 
CG backing stores                 5944K        3 
CG image                            16K        2 
CoreGraphics                         8K        1 
CoreImage                           24K        2 
CoreUI image data                 1372K        7 
CoreUI image file                  404K        4 
Dispatch continuations            8192K        1 
Kernel Alloc Once                    8K        1 
MALLOC                           208.3M       85 
MALLOC guard page                   32K        7 
MALLOC_LARGE (reserved)            128K        1         reserved VM address space (unallocated)
Memory Tag 242                      12K        1 
STACK GUARD                         20K        5 
Stack                             18.0M        5 
VM_ALLOCATE                        108K       10 
VM_ALLOCATE (reserved)            32.0M        1         reserved VM address space (unallocated)
__DATA                            30.4M      397 
__FONT_DATA                          4K        1 
__LINKEDIT                       238.3M      116 
__TEXT                           276.3M      362 
__UNICODE                          564K        1 
mapped file                       57.8M       22 
shared memory                      632K       10 
===========                     =======  ======= 
TOTAL                            878.6M     1047 
TOTAL, minus reserved VM space   846.5M     1047 

---

I want to update this lcd numbers very fast. How to do it properly?

Answer 1

I don't know so much about it, but i think the main leak cause is that you are making a client connection 10 times a second with timer3 . That's a lot of attempts to get info. As far as i know, you might even be better of with making a single global client , i think the leak is the result of making too many (10 a second + 2 every 10 seconds) clients. Making one when you start probably would do, i think something doesn't get cleared when the client falls out of scope.

Another reason for lag is that making the updates and querying the database is slow, and i think QTimers just run callbacks the main thread, meaning waiting for those updates will also slow down the GUI. You need to do the heavy lifting in a thread, and then just use timers to get information you already have, to the GUI. It might also be this that's causing the memory leak, but i don't know why in that case.

Either way, i think you should fix both problems:

• Reuse your client variable, don't remake it every call.
• The functions the timers call need to do things fast, let database stuff happen in a thread, and use timers to update the GUI with the thread gathered information. The main problem here is timer3 which is making 10 calls a seconds to that database.