批处理Windows中的正则表达式

Question

I am currently working on a batch job(and I am new to this as well) to get a list of maven dependencies . Been working to fetch those dependencies using regex but to no avail. I have tested my regex on this site and it seems it's working.

Checked the needed escape characters inside a FOR loop from this stackoverflow question but again, to not avail thus I have resorted to ask help in here.

See below for my code snippet esp. the regex part, the outer loop is fine.

for /f "tokens=*" %%i in ('findstr "+- \\-"  dependency-list.txt') do (
    for /f "tokens=*" %%j in ('findstr /i /r "^^(.+^)\[:\]^(.+^)^(:jar:^)^(.+^)^(:compile^)" %%i') do (
        echo %%j
    )
)

And this is the sample data fetched from my outer loop:

[INFO] +- org.owasp.esapi:esapi:jar:2.0.1:compile
[INFO] |  +- commons-configuration:commons-configuration:jar:1.5:compile
[INFO] |  |  +- commons-lang:commons-lang:jar:2.3:compile

I wanted to get these part only using regex:

commons-configuration:commons-configuration:jar:1.5:compile
org.owasp.esapi:esapi:jar:2.0.1:compile
commons-lang:commons-lang:jar:2.3:compile

These are the errors I'm getting when trying to execute my batch script.

FINDSTR: Cannot open org.owasp.esapi:esapi:jar:2.0.1:compile
'+-' is not recognized as an internal or external command,
operable program or batch file.
| was unexpected at this time.

Any help would be greatly appreciated.

NOTE: Once I am able to execute my regex successfully, the outer loop can be removed.

Answer 1

The regular expression capabilities of FINDSTR are very limited as it can be read on opening a command prompt window and running findstr /? to get output its help. The regular expression capabilities of FINDSTR are far away from Perl compatible regular expressions or Boost.Regex or other regular expression implementations in Perl syntax.

There is no option which can be used to get output by FINDSTR only the string matched by a regular expression. There are several options which controls the output of FINDSTR , but none for getting output just the found string.

So it is necessary to remove everything left to data of interest on processing lines containing +- or \\- in file dependency-list.txt with a different method. Windows command processor offers just two commands for string reformatting tasks: FOR and SET .

The batch file below demonstrates both of them taking into account that delayed expansion is needed on accessing value of an environment variable defined/modified in same command block.

@echo off
setlocal EnableExtensions DisableDelayedExpansion
cls
set "DeleteExample="
if exist dependency-list.txt goto GetData
(
    set "DeleteExample=1"
    echo First line with no data of interest.
    echo [INFO] +- org.owasp.esapi:esapi:jar:2.0.1:compile
    echo [INFO] ^|  +- commons-configuration:commons-configuration:jar:1.5:compile
    echo [INFO] ^|  ^|  +- commons-lang:commons-lang:jar:2.3:compile
    echo ;[INFO]^|  \- commons-lang:commons-lang:jar:4.5:compile
    echo Line trailing space at end +- 
    echo Line with no space at end  +-
    echo Line with just a hyphen character - somewhere on line.
    echo [INFO] ^|  ^|  +- !commons-configuration!:commons-configuration!:jar:5.2:compile
    echo Last line with no data of interest.
)>dependency-list.txt

:GetData
echo First solution
echo ==============
echo/
setlocal EnableDelayedExpansion
for /F "tokens=*" %%I in ('%SystemRoot%\System32\findstr.exe /R "+- \\-" dependency-list.txt 2^>nul') do (
    echo Line read: "%%I"
    set "Line=%%I"
    set "Line=!Line:*- =!"
    if defined Line echo Line work: "!Line!"
    rem More commands working with environment variable Line referenced with
    rem exclamation marks for delayed expansion on execution of the command line.
)
endlocal

echo/
echo Second solution
echo ===============
echo/
for /F "tokens=1* delims=-" %%I in ('%SystemRoot%\System32\findstr.exe /R "+- \\-" dependency-list.txt') do (
    if not "%%J" == "" (
        echo Line read: "%%J"
        set "Line=%%J"
        setlocal EnableDelayedExpansion
        call set "Line=%%Line:~1%%"
        if defined Line call echo Line work: "%%Line%%"
        rem More commands working with environment variable Line referenced
        rem with two percent signs on both side and with using command CALL.
        endlocal
    )
)

echo/
echo Third solution
echo ==============
echo/
for /F "tokens=1* delims=-" %%I in ('%SystemRoot%\System32\findstr.exe /R "+- \\-" dependency-list.txt 2^>nul') do (
    if not "%%J" == "" (
        echo Line read: "%%J"
        for /F "tokens=*" %%L in ("%%J") do if not "%%L" == "" (
            echo Line work: "%%L"
            rem More commands working with loop variable L.
        )
    )
)

echo/
echo Fourth solution
echo ===============
echo/
for /F delims^=^ eol^= %%I in ('%SystemRoot%\System32\findstr.exe /R "+- \\-" dependency-list.txt 2^>nul') do (
    echo Line read: "%%I"
    set "Line=%%I"
    setlocal EnableDelayedExpansion
    set "Line=!Line:*- =!"
    if defined Line echo Line work: "!Line!"
    rem More commands working with environment variable Line referenced with
    rem exclamation marks for delayed expansion on execution of the command line.
    endlocal
)

echo/
echo Fifth solution
echo ==============
echo/
for /F delims^=^ eol^= %%I in ('%SystemRoot%\System32\findstr.exe /L /C:+- /C:"\\-" dependency-list.txt') do (
    echo Line read: "%%I"
    set "Line=%%I"
    call :ProcessLine
)
goto EndDemo

:ProcessLine
set "Line=%Line:*- =%"
if defined Line echo Line work: "%Line%"
rem More commands working with environment variable Line referenced with
rem percent signs for expansion before execution of the command line.
goto :EOF

:EndDemo
if defined DeleteExample del dependency-list.txt
echo/
endlocal
pause

The output of this batch file is:

First solution
==============

Line read: "[INFO] +- org.owasp.esapi:esapi:jar:2.0.1:compile"
Line work: "org.owasp.esapi:esapi:jar:2.0.1:compile"
Line read: "[INFO] |  +- commons-configuration:commons-configuration:jar:1.5:compile"
Line work: "commons-configuration:commons-configuration:jar:1.5:compile"
Line read: "[INFO] |  |  +- commons-lang:commons-lang:jar:2.3:compile"
Line work: "commons-lang:commons-lang:jar:2.3:compile"
Line read: "Line trailing space at end +- "
Line read: "Line with no space at end  +-"
Line work: "Line with no space at end  +-"
Line read: "[INFO] |  |  +- :commons-configuration:jar:5.2:compile"
Line work: ":commons-configuration:jar:5.2:compile"

Second solution
===============

Line read: " org.owasp.esapi:esapi:jar:2.0.1:compile"
Line work: "org.owasp.esapi:esapi:jar:2.0.1:compile"
Line read: " commons-configuration:commons-configuration:jar:1.5:compile"
Line work: "commons-configuration:commons-configuration:jar:1.5:compile"
Line read: " commons-lang:commons-lang:jar:2.3:compile"
Line work: "commons-lang:commons-lang:jar:2.3:compile"
Line read: " "
Line read: " !commons-configuration!:commons-configuration!:jar:5.2:compile"
Line work: "!commons-configuration!:commons-configuration!:jar:5.2:compile"

Third solution
==============

Line read: " org.owasp.esapi:esapi:jar:2.0.1:compile"
Line work: "org.owasp.esapi:esapi:jar:2.0.1:compile"
Line read: " commons-configuration:commons-configuration:jar:1.5:compile"
Line work: "commons-configuration:commons-configuration:jar:1.5:compile"
Line read: " commons-lang:commons-lang:jar:2.3:compile"
Line work: "commons-lang:commons-lang:jar:2.3:compile"
Line read: " "
Line read: " !commons-configuration!:commons-configuration!:jar:5.2:compile"
Line work: "!commons-configuration!:commons-configuration!:jar:5.2:compile"

Fourth solution
===============

Line read: "[INFO] +- org.owasp.esapi:esapi:jar:2.0.1:compile"
Line work: "org.owasp.esapi:esapi:jar:2.0.1:compile"
Line read: "[INFO] |  +- commons-configuration:commons-configuration:jar:1.5:compile"
Line work: "commons-configuration:commons-configuration:jar:1.5:compile"
Line read: "[INFO] |  |  +- commons-lang:commons-lang:jar:2.3:compile"
Line work: "commons-lang:commons-lang:jar:2.3:compile"
Line read: ";[INFO]|  \- commons-lang:commons-lang:jar:4.5:compile"
Line work: "commons-lang:commons-lang:jar:4.5:compile"
Line read: "Line trailing space at end +- "
Line read: "Line with no space at end  +-"
Line work: "Line with no space at end  +-"
Line read: "[INFO] |  |  +- !commons-configuration!:commons-configuration!:jar:5.2:compile"
Line work: "!commons-configuration!:commons-configuration!:jar:5.2:compile"

Fifth solution
==============

Line read: "[INFO] +- org.owasp.esapi:esapi:jar:2.0.1:compile"
Line work: "org.owasp.esapi:esapi:jar:2.0.1:compile"
Line read: "[INFO] |  +- commons-configuration:commons-configuration:jar:1.5:compile"
Line work: "commons-configuration:commons-configuration:jar:1.5:compile"
Line read: "[INFO] |  |  +- commons-lang:commons-lang:jar:2.3:compile"
Line work: "commons-lang:commons-lang:jar:2.3:compile"
Line read: ";[INFO]|  \- commons-lang:commons-lang:jar:4.5:compile"
Line work: "commons-lang:commons-lang:jar:4.5:compile"
Line read: "Line trailing space at end +- "
Line read: "Line with no space at end  +-"
Line work: "Line with no space at end  +-"
Line read: "[INFO] |  |  +- !commons-configuration!:commons-configuration!:jar:5.2:compile"
Line work: "!commons-configuration!:commons-configuration!:jar:5.2:compile"

The first four solutions run in a separate command process started with cmd.exe /C in background the command line:

C:\Windows\System32\findstr.exe /R "+- \\-" dependency-list.txt

This command line results in searching in file dependency-list.txt in current directory which can be different to directory of the batch file with a case-sensitive regular expression for lines containing either +- or \\- .

The fifth solution runs in separate command process the command line:

C:\Windows\System32\findstr.exe /L /C:+- /C:"\\-" dependency-list.txt

This command line finds the same lines, but with using a case-sensitive literal search for finding lines containing either +- or \\- .

FINDSTR outputs nothing to STDOUT and also nothing to STDERR on not finding any line containing either +- or \\- .

FOR captures the output to STDOUT of separate command process and processes the output line by line.

Empty lines are ignored by FOR as by default also lines starting with a semicolon because of eol=; is the default for end of line character option. FOR splits up by default the line into substrings (tokens) using normal space and horizontal tab character as string delimiters and assigns just first space/tab delimited string to the specified loop variable.

First solution

The first solution modifies line processing behavior of FOR by using option tokens=* which results in removing all leading spaces/tabs from a line and if there is something left assigned the rest of the line with including spaces/tabs to the specified loop variable I . The lines in output starting with Line read: shows the string assigned to loop variable I after line processing by FOR .

This string is assigned to environment variable Line . Then everything up to first occurrence of a string consisting of a hyphen and a space is removed from the line using a string substitution done by command SET with using delayed environment variables expansion as required on referencing value of an environment variable defined/modified inside a command block starting with ( and ending with matching ) .

If there is still something left after the string substitution resulting in environment variable Line being still defined, this remaining part of the line is output with Line work: to see what was done by the string substitution.

This solution is fast, but have the disadvantages that lines starting with a semicolon are ignored by FOR and permanently enabled delayed environment variable expansion results in getting a line containing one or more exclamation marks not processed correct. However, this is the best solution on file dependency-list.txt contains never lines of interest starting with ; and containing one or more ! .

Second solution

The second solution uses command FOR to split up each non-empty line not starting with a semicolon into two substrings. The first substring up to first occurrence of one or more - is assigned to specified loop variable I and the rest of the line to next loop variable J according to ASCII table.

So for the example lines J contains the data of interest with a leading space on not being an empty string. The leading space is removed on using again command SET to get assigned to environment variable Line finally only the string from second character with character index 1 up to end of string.

But instead of using delayed expansion a different technique is used by referencing the environment variable value just set before in the command block with two percent signs on both side and use command CALL to force Windows command processor to parse the two lines before execution of the commands SET or ECHO a second time. So on parsing the entire command block before executing FOR the two lines with CALL became:

call set "Line=%Line:~1%"
if defined Line call echo Line work: %Line%

Those two lines are parsed on each iteration of the loop a second time because of command CALL resulting in replacing %Line:~1% and %Line% by current string without first character respectively completely before executing set and echo .

The lines of interest containing an exclamation mark are processed correct by this solution because of avoiding usage of delayed expansion.

Third solution

The third solution is similar to second solution. It uses a second FOR to remove all leading spaces/tabs from string assigned to loop variable J by the outer FOR and assigns the remaining string to loop variable L which can be even an empty string.

This solution is definitely better than the second solution because of being faster.

Fourth solution

The four solution is written to assign to loop variable I always the entire non-empty line from captured output independent on which character is the first character. Line splitting behavior is disabled with definition of an empty list of delimiters and ignoring lines starting with a specific character is disabled with definition of no end of line character.

Removal of string left to first occurrence of hyphen and space is done like in first solution. The difference is that delayed expansion is disabled on assigning the line to environment variable Line and just enabled next temporarily for processing the line string.

The disadvantage of this solution is that the commands setlocal EnableDelayedExpansion and endlocal inside the loop do much more than just enabling and disabling delayed expansion. See this answer for details about the commands SETLOCAL and ENDLOCAL . So this solution is definitely not the fastest one in set of provided solutions.

Fifth solution

The fifth solution is similar to fourth solution with the difference to show the common technique for avoiding the usage of delayed expansion on processing string values inside a FOR loop by using a subroutine called from within the FOR loop.

This solution is also very slow, but has the big advantage for easier debugging the batch file on running it with @echo ON at top from within a batch file.

For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.

• call /?
• cls /?
• del /?
• echo /?
• endlocal /?
• findstr /?
• for /?
• goto /?
• if /?
• pause /?
• rem /?
• set /?
• setlocal /?