简体   繁体   English

通过多个线程使用ifstream读取同一文件(pthread)

[英]Reading same file with ifstream by multiple threads (pthread)

I have a function that is executed simultaneously by multiple threads created in my program and which recursively creates more threads to execute the same function again. 我有一个函数,该函数由我的程序中创建的多个线程同时执行,并且递归地创建更多线程以再次执行同一函数。 I have to process a large file in this function. 我必须在此功能中处理一个大文件。 As there would be multiple threads processing the same file, I guess I'll have to seek to the start of the file for each thread. 由于将有多个线程处理同一个文件,我想我必须为每个线程寻找文件的开头。 Will this move the other file streams also to the start of the file? 这会将其他文件流也移动到文件的开头吗? Will there be any issues? 有什么问题吗?

void *myFunc(){
    string lin;
    ifstream ifs ("input.txt");
    if(ifs){
        ifs.seekg(0,ifs.beg);
        while(getline(ifs,lin)){
            ...
            do something
            ...
        }
        ifs.close();
    }
    pthread_t ptds[100];
    int cc = 0;
    if(some condition based on the above code){
        for(int i=0;i<100;i++){
            int rc = pthread_create(&ptds[cc++], NULL, myFunc, NULL);
            if (rc){
                cout << "Error:unable to create thread," << rc << endl;
                exit(-1);
            }
            else{
                cout << "Thread created" << endl;
            }
        }
        void* status;
        int rc;
        for(int i=0; i < 100; i++ ){
            rc = pthread_join(ptds[i], &status);
            if (rc){
                cout << "Error:unable to join," << rc << endl;
                exit(-1);
            }
        }
    }
}

I am getting some ambiguous results. 我得到一些模棱两可的结果。 The results change every time I run the code. 每次我运行代码,结果都会改变。 I'm assuming this has something to do with file I/O synchronization. 我假设这与文件I / O同步有关。 How to resolve this? 如何解决呢?

If you try to read the same file object in multiple threads, they'll clobber each other's state. 如果您尝试在多个线程中读取同一文件对象,它们将破坏彼此的状态。 I suggest reading the file into memory once (or memory-mapping it) and having each thread read the data from memory, which they can do wait-free. 我建议一次将文件读入内存(或对其进行内存映射),然后让每个线程从内存中读取数据,他们可以无等待地执行此操作。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM