从 python 脚本运行多个终端并执行命令 (Ubuntu)

Question

What I have is a text file containing all items that need to be deleted from an online app. Every item that needs to be deleted has to be sent 1 at a time. To make deletion process faster, I divide the items in text file in multiple text files and run the script in multiple terminals (~130 for deletion time to be under 30 minutes for ~7000 items).

This is the code of the deletion script:

from fileinput import filename
from WitApiClient import WitApiClient
import os


dirname = os.path.dirname(__file__)
parent_dirname = os.path.dirname(dirname)
token = input("Enter the token")
file_name = os.path.join(parent_dirname, 'data/deletion_pair.txt')

with open(file_name, encoding="utf-8") as file:
        templates = [line.strip() for line in file.readlines()]

for template in templates:
    entity, keyword = template.split(", ")
    print(entity, keyword)
    resp = WitApiClient(token).delete_keyword(entity, keyword)
    print(resp)

So, I divide the items in deletion_pair.txt and run this script multiple times in new terminals (~130 terminals). Is there a way to automate this process or do in more efficient manner?

Answer 1

I used threading to run multiple functions simultaneously:

from fileinput import filename
from WitApiClient import WitApiClient
import os
from threading import Thread

dirname = os.path.dirname(__file__)
parent_dirname = os.path.dirname(dirname)
token = input("Enter the token")
file_name = os.path.join(parent_dirname, 'data/deletion_pair.txt')

with open(file_name, encoding="utf-8") as file:
    templates = [line.strip() for line in file.readlines()]

batch_size = 20
chunks = [templates[i: i + batch_size] for i in range(0, len(templates), batch_size)]

def delete_function(templates, token):
    for template in templates:
        entity, keyword = template.split(", ")
        print(entity, keyword)
        resp = WitApiClient(token).delete_keyword(entity, keyword)
        print(resp)

for chunk in chunks:
    thread = Thread(target=delete_function, args=(chunk, token))
    thread.start()

It worked, Any one has any other solution. please post or if the same code can be written more efficiently then please do tell. Thanks.