从unix bash脚本转换为Windows批处理脚本

Question

我正在尝试将Linux bash脚本转换为其Windows计数器部分。
这是实际的bash脚本：

#!/bin/bash
# Usage parse_log.sh caffe.log
# It creates the following two text files, each containing a table:
#     caffe.log.test (columns: '#Iters Seconds TestAccuracy TestLoss')
#     caffe.log.train (columns: '#Iters Seconds TrainingLoss LearningRate')


# get the dirname of the script
DIR="$( cd "$(dirname "$0")" ; pwd -P )"

if [ "$#" -lt 1 ]
then
echo "Usage parse_log.sh /path/to/your.log"
exit
fi
LOG=`basename $1`
sed -n '/Iteration .* Testing net/,/Iteration *. loss/p' $1 > aux.txt
sed -i '/Waiting for data/d' aux.txt
sed -i '/prefetch queue empty/d' aux.txt
sed -i '/Iteration .* loss/d' aux.txt
sed -i '/Iteration .* lr/d' aux.txt
sed -i '/Train net/d' aux.txt
grep 'Iteration ' aux.txt | sed  's/.*Iteration \([[:digit:]]*\).*/\1/g' > aux0.txt
grep 'Test net output #0' aux.txt | awk '{print $11}' > aux1.txt
grep 'Test net output #1' aux.txt | awk '{print $11}' > aux2.txt

# Extracting elapsed seconds
# For extraction of time since this line contains the start time
grep '] Solving ' $1 > aux3.txt
grep 'Testing net' $1 >> aux3.txt
$DIR/extract_seconds.py aux3.txt aux4.txt

# Generating
echo '#Iters Seconds TestAccuracy TestLoss'> $LOG.test
paste aux0.txt aux4.txt aux1.txt aux2.txt | column -t >> $LOG.test
rm aux.txt aux0.txt aux1.txt aux2.txt aux3.txt aux4.txt

# For extraction of time since this line contains the start time
grep '] Solving ' $1 > aux.txt
grep ', loss = ' $1 >> aux.txt
grep 'Iteration ' aux.txt | sed  's/.*Iteration \([[:digit:]]*\).*/\1/g' > aux0.txt
grep ', loss = ' $1 | awk '{print $9}' > aux1.txt
grep ', lr = ' $1 | awk '{print $9}' > aux2.txt

# Extracting elapsed seconds
$DIR/extract_seconds.py aux.txt aux3.txt

# Generating
echo '#Iters Seconds TrainingLoss LearningRate'> $LOG.train
paste aux0.txt aux3.txt aux1.txt aux2.txt | column -t >> $LOG.train
rm aux.txt aux0.txt aux1.txt aux2.txt  aux3.txt

这是我的转换版本：

ECHO OFF
REM Usage parse_log.sh caffe.log
REM It creates the following two text files, each containing a table:
REM     caffe.log.test (columns: '#Iters Seconds TestAccuracy TestLoss')
REM     caffe.log.train (columns: '#Iters Seconds TrainingLoss LearningRate')

SET sed="tools/3rdparty/bin/sed.exe"
SET awk="tools/3rdparty/bin/awk.exe"
SET grep="tools/3rdparty/bin/grep.exe"
SET rm="tools/3rdparty/bin/rm.exe"
SET paste="tools/3rdparty/bin/paste.exe"

REM get the dirname of the script
SET DIR=%~dp0

IF [%1]==[] (
ECHO "Usage parse_log.bat path\to\your.log"
EXIT
)

SET LOG="basename %~n1"
ECHO "Dir = %DIR%"
ECHO "LOG = %LOG%"

%sed% -n "/Iteration .* Testing net/,/Iteration *. loss/p" %1 > aux.txt
%sed% -i "/Waiting for data/d" aux.txt
%sed% -i "/prefetch queue empty/d" aux.txt
%sed% -i "/Iteration .* loss/d" aux.txt
%sed% -i "/Iteration .* lr/d" aux.txt
%sed% -i "/Train net/d" aux.txt
%grep% "Iteration " aux.txt | %sed%  "s/.*Iteration \([[:digit:]]*\).*/\1/g" > aux0.txt
%grep% "Test net output #0" aux.txt | %awk% "{print %11}" > aux1.txt
%grep% "Test net output #1" aux.txt | %awk% "{print %11}" > aux2.txt

REM Extracting elapsed seconds
REM For extraction of time since this line contains the start time
%grep% "] Solving " %1 > aux3.txt
%grep% "Testing net" %1 >> aux3.txt
%DIR%/extract_seconds.py aux3.txt aux4.txt

REM Generating
ECHO "#Iters Seconds TestAccuracy TestLoss"> %LOG%.test
%paste% aux0.txt aux4.txt aux1.txt aux2.txt | column -t >> %LOG%.test
%rm% aux.txt aux0.txt aux1.txt aux2.txt aux3.txt aux4.txt

REM For extraction of time since this line contains the start time
%grep% "] Solving " %1 > aux.txt
%grep% ", loss = " %1 >> aux.txt
%grep% "Iteration " aux.txt | %sed%  "s/.*Iteration \([[:digit:]]*\).*/\1/g" > aux0.txt
%grep% ", loss = " %1 | %awk% "{print %9}" > aux1.txt
%grep% ", lr = " %1 | %awk% "{print %9}" > aux2.txt

REM Extracting elapsed seconds
%DIR%/extract_seconds.py aux.txt aux3.txt

REM Generating
ECHO "#Iters Seconds TrainingLoss LearningRate"> %LOG%.train
%paste% aux0.txt aux3.txt aux1.txt aux2.txt | column -t >> %LOG%.train
%rm% aux.txt aux0.txt aux1.txt aux2.txt  aux3.txt

问题首先是我不知道这是否是正确的转换，并且确实完成了bash脚本的功能。 第二，当我运行脚本时，当光标到达ECHO "LOG = %LOG%"部分时，光标开始闪烁（等待），很长一段时间后，它会打印以下内容：

tools/3rdparty/bin/sed.exe: couldn't edit aux.txt: is a terminal
tools/3rdparty/bin/sed.exe: couldn't edit aux.txt: is a terminal
tools/3rdparty/bin/sed.exe: couldn't edit aux.txt: is a terminal
tools/3rdparty/bin/sed.exe: couldn't edit aux.txt: is a terminal
tools/3rdparty/bin/sed.exe: couldn't edit aux.txt: is a terminal

它又等了！

这有什么不对？

---

UPDATE

将aux.txt更改为auxx.txt后，将解决第一个错误。 但现在我得到了这些错误：

awk: {print examples\cifar10\caffe.log1}
awk:                ^ backslash not last character on line
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
awk: {print examples\cifar10\caffe.log1}
awk:                ^ backslash not last character on line
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
tools/3rdparty/bin/grep.exe: writing output: Invalid argument
Usage: ./extract_seconds input_file output_file
'column' is not recognized as an internal or external command,
operable program or batch file.

Answer 1

AUX是保留字，请参阅命名文件，路径和命名空间 ：

不要将以下保留名称用于文件名：

CON，PRN，AUX，NUL，COM1，COM2，COM3，COM4，COM5，COM6，COM7，COM8，COM9，LPT1，LPT2，LPT3，LPT4，LPT5，LPT6，LPT7，LPT8和LPT9。 同时避免使用这些名称后立即进行扩展; 例如，不建议使用NUL.txt 。 有关更多信息，请参阅命名空间 。

请改用另一个不同的名称，例如auxx.txt 。

Answer 2

请阅读有关如何使用空格设置环境变量的答案？ 为什么在命令行上使用'set var = text'后，没有字符串输出'echo％var％'？ 因为您很可能不知道如何定义具有正确空格的变量值。

不要使用IF [%1]==[]因为方括号在IF上的Windows命令脚本中没有特殊含义，因此批处理可能因语法错误而退出，其中左值包含1个或多个空格。 请改用IF "%~1"==""并在命令提示符窗口中执行call /? 解释%~1 （删除了周围双引号的第一个参数）。

建议始终使用%~1 ， %~2 ，...将这些参数引用显式嵌入双引号中。

在Windows上，目录分隔符是反斜杠\\字符，而不是正斜杠/字符。 Windows命令处理器会自动修正这个常犯的错误在那里检测到它，但它不能检测和校正/在所有情况下自动完成。

awk来自* nix，并且反斜杠与Windows命令处理器具有完全不同的含义。 我没有使用awk的经验，但我认为最好在Windows上指定带有作为参数传递给awk的路径的文件名，并使用正斜杠作为目录分隔符而不是反斜杠。 因此我建议更换线路：

SET awk="tools/3rdparty/bin/awk.exe"
SET grep="tools/3rdparty/bin/grep.exe"

rem Other lines of your batch file.

%grep% "Test net output #0" auxx.txt | %awk% "{print %11}" > aux1.txt
%grep% "Test net output #1" auxx.txt | %awk% "{print %11}" > aux2.txt

通过

set "awk=tools\3rdparty\bin\awk.exe"
set "grep=tools\3rdparty\bin\grep.exe"

rem Other lines of your batch file.

set "LogFileName=%~11"
set "LogFileName=%LogFileName:\=/%"
"%grep%" "Test net output #0" auxx.txt | "%awk%" "{print %LogFileName%}" > aux1.txt
"%grep%" "Test net output #1" auxx.txt | "%awk%" "{print %LogFileName%}" > aux2.txt

在Windows上使用* nix工具需要考虑Windows本机命令行工具（如cmd.exe ）解释的awk.exe以及grep.exe和grep.exe等* nix工具解释的awk.exe ，以便在参数上使用正确的语法。

set "LogFileName=%LogFileName:\\=/%"将日志文件名中的正斜杠替换为相对路径的所有反斜杠。 在命令提示符窗口中运行set /? 并阅读所有输出帮助页面，以获取有关环境变量值中字符串替换/替换的详细信息。

并在线

%paste% aux0.txt aux3.txt aux1.txt aux2.txt | column -t >> %LOG%.train

Windows命令处理器（ cmd.exe ）找不到具有在当前目录中的环境变量PATHEXT中定义的文件扩展名的应用程序column或在环境变量PATH定义的任何目录。 此应用程序还必须使用文件扩展名和（相对）路径指定。