在g ++和boost中的regex_replace（）替换字符串中对'\\'的不同处理

Question

after upgrading gcc from version 4.8.5 to version 5.3.1 I thought I could get rid of boost's regex-implementation (boost version 1.54.0) and use the one provided by gcc (it didn't work with gcc before version 4.9 AFAIK). However, this turned out to be a problem because those two implementations behave differently:

#include <regex>
#include <boost/regex.hpp>
#include <iostream>
#include <string>

int main() {
    std::string s="\\needs_another_backslash";
    std::string reg("^(\\\\)(needs)(.+)");
    std::string rep("\\\\got$3");
    std::regex sr(reg);
    boost::regex br(reg);
    std::cout<<"string before replacement:\n"<<s<<std::endl<<
        "regular expression:\n"<<reg<<std::endl<<
        "replacement string:\n"<<rep<<std::endl<<
        "std::regex_replace:\n"<<std::regex_replace(s,sr,rep)<<std::endl<<
        "boost::regex_replace:\n"<<boost::regex_replace(s,br,rep)<<std::endl;
    return 0;
}

This gives the following output:

string before replacement: \\needs_another_backslash regular expression: ^(\\\\)(needs)(.+) replacement string: \\\\got$3 std::regex_replace: \\\\got_another_backslash boost::regex_replace: \\got_another_backslash

It seems as if boost treats the '\\' in a replacement string specially whereas gcc does not. Since the magical character for backreference in the replacement string for std::regex_replace is '$' (which it also is in boost as the example proofs), I tend to think that gcc is right. However, in many other programs (like vim eg) it is '\\'. Therefore, boost might have a point in treating '\\' specially. So who is right?

Answer 1

First, the std example actually is not a matter of gcc, but of C++ standard, to which gcc (in this case) is compliant. The standard states in 28.5.2:

When a regular expression match is to be replaced by a new string, the new string shall be constructed using the rules used by the ECMAScript replace function in ECMA-262, part 15.5.4.11 String.prototype.replace. In addition, during search and replace operations all non-overlapping occurrences of the regular expression shall be located and replaced, and sections of the input that did not match the expression shall be copied unchanged to the output string.

And ECMA states:

Otherwise, let newstring denote the result of converting replaceValue to a String. The result is a String value derived from the original input String by replacing each matched substring with a String derived from newstring by replacing characters in newstring by replacement text as specified in Table 22. These $ replacements are done left-to-right, and, once such a replacement is performed, the new replacement text is not subject to further replacements. For example, "$1,$2".replace(/(\\$(\\d))/g, "$$1-$1$2") returns "$1-$11,$1-$22". A $ in newstring that does not match any of the forms below is left as is.

(If part: replaceValue is a function.)

Nothing mentioned about escape sequences being replaced. Tried out with firefox:

var test = "\\needs_another_backslash";
test = test.replace(/^(\\)(needs)(.+)/, "\\\\got$3");
alert(test);

Result: \\\\got_another_backslash .

boost documentation states:

Effects: If fmt is either a null-terminated string, or a container of char_type's, then copies the character sequence [fmt.begin(), fmt.end()) to OutputIterator out. For each format specifier or escape sequence in fmt, replace that sequence with either the character(s) it represents, or the sequence of characters within *this to which it refers. The bitmasks specified in flags determines what format specifiers or escape sequences are recognized, by default this is the format used by ECMA-262, ECMAScript Language Specification, Chapter 15 part 5.4.11 String.prototype.replace.

Additionally, it states for match_type_flags :

Specifies that when a regular expression match is to be replaced by a new string, that the new string is constructed using the rules used by the ECMAScript replace function in ECMA-262, ECMAScript Language Specification, Chapter 15 part 5.4.11 String.prototype.replace. (FWD.1).

This is functionally identical to the Perl format string rules.

[...]

Tried with perl 5.18.2 on linux:

my $test = "\\needs_another_backslash";
$test =~ s/^(\\)(needs)(.+)/\\\\got$3/;
print "$test\n";

Resulted in \\\\got_another_backslash .

With std::string reg("^(\\\\\\\\)(needs)(.+)"); , as a string literal is passed, reg holds a string ^(\\\\)(needs)(.+) (not a literal, so left out the quotes!), and with std::string rep("\\\\\\\\got$3"); , rep holds \\\\got$3 .

But there is obviously a difference in interpretation. Assume we had for both std and boost one and the same ECMAScript engine.

Then, what both std and boost yet do consistently, is compiling the reg string as regular expression:

sprintf(b, "/%s/", reg);
sr /* br, respectively */ = ECMAScriptEngine::compileFromSource(b);

I think this is reflected quite nicely by creating an instance of std/boost::regex class.

Then comes the difference, however: std passes s , sr and rep to the ECMAScript engine such that it calls directly s.(String.prototype.replace)(sr, rep); (of course there is no such function for s in reality – just lets assume we could do it this way).

boost lets compile the rep string, too (side note: I haven't installed boost, so I did not verify this behaviour myself...):

sprintf(b, "'%s'", rep); // note: '', not //!
ecma_rep = ECMAScriptEngine::compileFromSource(b);

and then makes the engine call s.(String.prototype.replace)(sr, ecma_rep); .

Interestingly, boost does not compile the source string s, where it again agrees with std...

In the end, I think, however, the standard implementation reflects closer what we actually want to do:

s.replace(regex, string);
s.replace(/reg/, rep);
(std::string).replace(std::regex(std::string), std::string);
std::regex_replace(s, std::regex(reg), rep);

vs

s.replace(regex, string);
s.replace(/reg/, "rep");
(std::string).replace(boost::regex(std::string), boost::???(std::string));
boost::regex_replace(s, boost::regex(reg), rep); // not boost::???(rep)!

Not sure if this is sufficient to say one is right and the other wrong, however (would mean that the wrong one is buggy!). Possibly we even have to remain with a third option: Both approaches are valid (so both are right and none is wrong) and, unfortunately, they are incompatible...