Python-每个任务都从.txt文件读取每个名称（多处理）

Question

So I have been working abit with multiprocessing and finally got it to work with multiprocessing - Basically now I got it to run so every tasks runs now depending on how many tasks I enter.

def main():


    user_input = 0
    while True:
        try:
            user_input = int(input(Fore.WHITE + 'How many tasks do you wanna run? [NUMBERS] \n' + Fore.RESET))
        except ValueError:
            print(Fore.RED + "Stop being stupid" + Fore.RESET)
            continue
        else:
            HowManyThread = user_input
            print()
            i = 0
            jobs = []
            for i in range(HowManyThread):
                p = multiprocessing.Process(target=info, args=(str(i),))
                jobs.append(p)
                time.sleep(.5)
                p.start()

            for p in jobs:
                p.join()

            sys.exit()

however I was looking through other stackoverflow threads and found

with open(fname) as f:
    content = f.readlines()
# you may also want to remove whitespace characters like `\n` at the end of each line
content = [x.strip() for x in content] 

where I have all my names in fname etc:

Barry Alex Sven Mila Jenny etc etc...

However what the multiprocessing is doing though is that it actually reads the same for each tasks and I can't really get a grip what to do now...

Basically what I want to do is that etc...

tasks 1 to read first line tasks 2 to read second line tasks 3 to read third line etc etc...

What could be the best solution for this?

Answer 1

You cannot do it. On multiprocessing, every process has own non-shared status.

So every process will ask the operating system to open the file. Every process will start reading the file from the beginning. You can force every process to process only some lines, but every process needs to read all lines, to find where they start and where they end, to count lines.

You example is not something that should be done in multiprocess, but with server (which serve lines) and every multiprocess ask the server for a new line. But this is much more server-client than multiprocess algorithm. Or 'async' or with threads (not sure that read is atomic, so it could make things more complex.)

Answer 2

You can do this with a process which distribute lines to subprocess. If you are on a UNIX system, you should look for os.fork() and os.pipe() .