How to get the total number of rows before reading in a large file in PHP

Question

I am already using this example of how to read large data files in PHP line by line

Now, what it'd like to do, is obtain the total number of rows in the file so that I may display a percentage complete or at least what the total number of rows are so I can provide some idea of how much processing is left to be done.

Is there a way to get the total number of rows without reading in the entire file twice? (once to count the rows and once to do the processing)

Answer 1

Poor mans answer:

No, but you can estimate. Calc a simple average reading (use the first 250 lines) and go with that.

estNumOfLines = sizeOfFile / avgLineSize

You could store off the number of lines in the file when you are creating the file...

Alternatively, you could display the number of KB processed, and that would be perfectly accurate.

Answer 2

You can determine the size of the file, then guage your progress through it by adding up the size of your reads:

$fname = 'foofile.txt';
$fsize = filesize($fname);
$count = 0;
$handle = fopen($fname, "r") or die("Couldn't get handle");
if ($handle) {
  while (!feof($handle)) {
    $buffer = fgets($handle, 4096);
    // Process buffer here..
    $count++;
    echo ($count * 4096)/$fsize . " percent read.";
  }
  fclose($handle);
}

Note: code adapted from referenced answer

Answer 3

Is there any reason you need to count rows and not bytes? If all you want to know is "percent done", just track it the by number bytes read/total bytes.

Answer 4

使用linux命令wc -l filename.txt这将输出文件中的行数。

Answer 5

How would you know the number of pages in a book, without counting them?
You would measure the width of a page and the width of the book and divide one by the other.

Same here, calculate the average line length from the first few lines, then do the same math with the file size...