C++ much faster than Bash script writing to text file

Question

I wanted to test the performance of writing to a file in a bash script vs a C++ program.

Here is the bash script:

#!/bin/bash

while true; do
        echo "something" >> bash.txt
done

This added about 2-3 KB to the text file per second.

Here is the C++ code:

#include <iostream>
#include <fstream>

using namespace std;

int main() {
    ofstream myfile;
    myfile.open("cpp.txt");

    while (true) {
        myfile << "Writing this to a file Writing this to a file \n";
    }

    myfile.close();
}

This created a ~6 GB text file in less than 10 seconds.

What makes this C++ code so much faster, and/or this bash script so much slower?

Answer 1

There are several reasons to it.

First off, interpreted execution environments (like bash , perl alongside with non-JITed lua and python etc.) are generally much slower than even poorly written compiled programs ( C , C++ , etc.).

Secondly, note how fragmented your bash code is - it just writes a line to a file, then it writes one more, and so on. Your C++ program, on the other side, performs buffered write - even without your direct efforts to it. You might see how slower will it run if you substitute

myfile << "Writing this to a file Writing this to a file \n";

with

myfile << "Writing this to a file Writing this to a file" << endl;

for more information about how streams are implemented in C++, and why \\n is different from endl , see any reference documentation on C++.

Thirdly, as comments prove, your bash script performs open/close of the target file for each line. This implies a significant performance overhead in itself - imagine myfile.open and myfile.close moved inside your loop body!

Answer 2

As others have already pointed out, this is because you are currently opening and closing the file with each line you write in your script (and shell scripts are interpreted while C++ is compiled). You might batch the writes instead and write once, for example

MSG="something"
logfile="test.txt"
(
for i in {1..10000}; do
        echo $MSG
done
) >> $logfile

Which will write the message 10k times but only open the log once.

Answer 3

Compiled vs. Interpreted Languages

Bash is interpreted while C++ is compiled. Just that makes it a lot faster