C++ istream、getline、二进制文件、正则表达式和字符串的意外行为

Question

I'm working with a file that combines text and binary formats (and sometimes just plain text). So I decided to open the file as binary and give it a try. However, I'm getting unexpected behaviors when I later use regular expressions (the kind of issues that indicate memory corruption):

(Edited to have a minimum example)

#include <string>
#include <iostream>
#include <fstream>
#include <regex>
#include <ios>

struct FortranFormat {
    std::string itemsPerRow;
    std::string type;
    std::string numberOfCharacters;
};

class XXXParserException: virtual public std::runtime_error {
    using runtime_error::runtime_error;
};


std::string parseSection(const std::string &line) {
    return  line.substr(16, std::string::npos );
}


FortranFormat parse(const std::string& expression) {
    const std::regex getItemsExpr("\\(([0-9]+)([A|a|I|i|F|f|E|e])([0-9]+)\\)");

    std::cout << "expression: " << expression << std::endl;

    std::smatch elements;
    if (std::regex_match(expression, elements, getItemsExpr)) {
        
        return {elements[1].str(),elements[2].str(),elements[3].str()};
    } else {
        throw XXXParserException("The expression " + expression + " is not a recognized Fortran Format.");
    }
}

void main() {

    std::ifstream fb;
    fb.open("example.txt", std::ios::binary); // remove the binary flag, and it works
    std::string line;
    getline(fb, line);
    std::cout << "line: " << line << std::endl;
    std::string formula = parseSection(line);
  
    auto format = parse(formula);
    
    std::cout << "format: " << format.type << std::endl;
}

The prints have the right information:

line: *VALUES        6(5E16.8)
expression: (5E16.8)

(Even the exception text is broken and only has the last portion: " is not a recognized Fortran Format.")

So, more out of curiosity than nothing: Am I doing something fundamentally wrong that is breaking something internally? Is this something that could be attributed to the compiler (VS2015)?

Just FYI, I will try a "jump between formats approach" to solve the issue (save the current position, close and open as text or binary as needed, restore position), but I just want to understand what might be wrong with my current approach.

Answer 1

There are two things to consider:

In text mode, \n is handled as the native EOL combination (so it would be \r\n on Windows). In binary mode no such thing is done, so \n is always the newline feed character and nothing else. You ask to read as much text as possible until \n , which on Windows leaves you with \r at the end of the string.

Then, std::regex_match requires the whole string to match regex. Your regex doesn't allow for extra whitespace at the end of string, so it doesn't match. std::regex_search would return true on that input, because a substring without the last character matches the pattern.

---

Protip: Raw string literals make regexes much easier, because you don't have to escape literals (easy to debug in regex101 now:):

const std::regex getItemsExpr(R"eos(\(([0-9]+)([A|a|I|i|F|f|E|e])([0-9]+)\))eos");