简体   繁体   English

对 unix 文件执行阻塞写操作

[英]Having a Blocking write operation to a unix file

Is there some way to open a file on unix for writting that is blocking (that actually wait for the data to be in the file before completing the "write" operation?是否有某种方法可以在 unix 上打开文件以进行阻塞写入(实际上是在完成“写入”操作之前等待数据进入文件中?

i'm having a program that takes result lines from a list and writes them in a file.我有一个程序从列表中获取结果行并将它们写入文件。

My current implementation is as follows (i removed the processing done on each line)我目前的实现如下(我删除了每一行的处理)

    ArrayList<String> lines = new ArrayList<>();
    BufferedReader in = new BufferedReader(new FileReader(inFile));
    String line;
    while((line = in.readLine()) != null){
      lines.add(line);
    }
    int written = 0;
    PrintWriter out = new PrintWriter(new FileWriter(outFile));
    for(String l : lines){
      written++;
      out.println(l);
      if(written % 10000 == 0)
        Message.info("lines written "+written);
    }
    out.close();

The current behaviour is the following:当前行为如下:
[17:12:43 INFO ] lines written 10000 [17:12:43 INFO] 行写入 10000
[17:12:43 INFO ] lines written 20000 [17:12:43 INFO] 行写入 20000
[17:12:43 INFO ] lines written 30000 [17:12:43 INFO] 行写入 30000
[17:12:44 INFO ] lines written 40000 [17:12:44 INFO] 行写入 40000
[17:12:44 INFO ] lines written 50000 [17:12:44 INFO ] 行写入 50000
[17:12:44 INFO ] lines written 60000 [17:12:44 INFO ] 行写入 60000
[17:12:45 INFO ] lines written 70000 [17:12:45 INFO] 行写入 70000
[17:12:45 INFO ] lines written 80000 [17:12:45 INFO] 行写入 80000
[17:12:45 INFO ] lines written 90000 [17:12:45 INFO] 行写入 90000
[17:12:46 INFO ] lines written 100000 [17:12:46 INFO ] 行写入 100000

Where the program runs very fast, but at the end, waits (30-40secondes) for all the data (4Go) to be written to the file before ending.程序运行速度非常快,但最后会等待(30-40 秒)所有数据(4Go)在结束前写入文件。 (i can see the file growing on the disk during this time) (我可以在这段时间看到磁盘上的文件增长)
When what i want, is for my INFO message to be displayed only when the data are really in the file: the program will seem slower, but will end immediatly after the last message.当我想要的是仅当数据确实在文件中时才显示我的 INFO 消息:程序看起来会更慢,但会在最后一条消息之后立即结束。

I tried using BufferedWriter (from new BufferedWriter() and Files.newBufferedWriter()), FileOutputStream, OutputStream, but all of those seem to propose only non-blocking IO operation我尝试使用 BufferedWriter(来自 new BufferedWriter() 和 Files.newBufferedWriter())、FileOutputStream、OutputStream,但所有这些似乎都建议仅使用非阻塞 IO 操作

The program is run on Ubuntu (I read that the implementation is filesystem dependant)该程序在 Ubuntu 上运行(我读到该实现依赖于文件系统)

So, is there some way to wait for a println()/write() operation to be complete before execution the next line of code?那么,有没有办法在执行下一行代码之前等待 println()/write() 操作完成?

I "feel" that java as delegated the actual writing to the OS, and doesn't wait for the data to be on the disk to go on, but since java waits for the last write to be complete before exiting, there must be a way the wait after each line我“感觉到” java将实际写作委派给了OS,并且不会等待数据在Z34D1F91F91FB2E514B8576FAB1FAB1A75A89A6BZ上的磁盘上,但Z9A.6BZ YOR14333333333333FF.193FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FF73FFF73FF73FF73FFF73FF73FF73FFF。每行之后的等待方式

The dirty workaround : Every 10,000 lines, I close the file and open it again using StanddardOpenOption.APPEND .肮脏的解决方法:每 10,000 行,我关闭文件并使用StanddardOpenOption.APPEND再次打开它。
This has the expected behaviour with only an acceptable time overhead.这具有预期的行为,只有可接受的时间开销。

The clean solution : open the file with `StanddardOpenOption.SYNC.干净的解决方案:使用 `StanddardOpenOption.SYNC 打开文件。
This has the expected behaviour with a STRONG time overhead这具有预期的行为,时间开销很大

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM