[英]Shell Script loop is executing multiple times
I have a log file. 我有一个日志文件。 I'm doing tail -f and grep options whenever new logs are coming.
每当有新日志出现时,我都会执行tail -f和grep选项。 I'm facing loop issue, It is executing multiple times.
我面临循环问题,它正在执行多次。 here is my script,
这是我的剧本,
AuditTypeID=$""
QueryResult=$""
tail -n 0 -F hive-server2.log \
| while read LINE
do
if [ `echo $LINE | grep -c "select *" ` -gt 0 ]
then
AuditTypeID=15
QueryResult=$(
awk '
BEGIN{ print "" }
/Executing command\(queryId/{ sub(/.*queryId=[^[:space:]]+: /,""); q=$0 }
/s3:\/\//{ print "," q }
' OFS=',' hive-server2.log \
| sed -n \$p
)
elif [ `echo $LINE | grep -c 'select count' ` -gt 0 ]
then
AuditTypeID=22
QueryResult="$(
grep -oE 'select count\(.\) from [a-zA-Z][a-zA-Z0-9]*' hive-server2.log \
| sed -n \$p
)"
fi
user=$(
cat hive-server2.log \
| grep user \
| awk -F "[. ]" '{print "," $(NF-1)}' \
| tr -d ',' \
| tr -d 'UTC'
)
Additional_Info=$(
echo -e "{\"user\":\"""${user}""\", \"query\":\"""${QueryResult}""\",\"s3Path\":\"""${s3}""\"}"
)
echo -e "$Additional_Info" > op.json
for file in /var/log/hive/op.json
do
boto-rsync $file s3://hive-log/log/script/$file.$current_time
done
done
It will filter the operations based on the keyword. 它将根据关键字过滤操作。 For some reason it is executing multiple times.
由于某种原因,它执行了多次。 I need to save the output for only one instance and any help to simplify the logic is appreciated.
我只需要保存一个实例的输出,就可以帮助您简化逻辑。
First thing I see in your script is that in the first awk scriptlet inside the if
statement you seem to be reparsing the whole of hive-server2.log
(which is probably racy/bad because you are tailing to your script, and hive-server.log
is growing?)... and this reparsing of the log seems to be a common theme in the script -- I think this is the root cause of the issue. 我在脚本中看到的第一件事是,在
if
语句内的第一个awk脚本中,您似乎正在重新解析整个hive-server2.log
(这可能很不好/很坏,因为您要跟踪脚本和hive-server.log
正在增长?)...,日志的重新解析似乎是脚本中的常见主题-我认为这是问题的根本原因。
One simplification ;) readily apparent is removal of the elif
code -- it will never run because /select count/
will be matched by the if
statement's /select */
. 一个简单的简化;)显而易见的是删除了
elif
代码-它永远不会运行,因为/select count/
将与if
语句的/select */
匹配。
To truly take a stab at simplifying this, my strategy would be to rewrite the whole of this in awk . 为了真正简化这个过程,我的策略是用awk重写整个过程。 There is nothing that you are doing here that is beyond awk 's built-in capabilities -- and awk can fire off external shell commands as easily as sh .
您在这里所做的任何事情都无法超越awk的内置功能-而且awk可以像sh一样轻松地触发外部shell命令。 The awk implementation will also likely be much faster.
awk的实现也可能会快得多。
I started trying to do this translation, but with the way you are specifying multiple reparsing of hive-server2.log
, I frankly got lost. 我开始尝试进行此转换,但是用您指定
hive-server2.log
多次重新解析的方式,坦率地说,我迷路了。 Having a bit of input and intended output would help here... Please post hive-server2.log
and your expected output. 有一点输入和预期的输出将在这里有所帮助...请发布
hive-server2.log
和您的预期输出。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.