When trying to get the YARN logs for a Spark application the logs seems to be compressed to gzip format, how can the same be decompressed. LogType:stdout.gz
LogType:stdout.gz
Log Upload Time:Tue Jan 21 03:38:17 +0000 2020
LogLength:1074
Log Contents:
řϲ£8Ưy
½ þ! ߶⌔kRك²[µ'1Î
pv歷±ɄA²]ö!券ߏ£²¢ڶ蕷eź6f÷-нOϯ«¿^��zùǪ¼¿Cû¯UQ
The logs are fetched using the yarn logs -applicationId <applicationId> > file_name.log
is there something that needs to be append to the command to decompress the files while outputing the results.
Kindly look into this approach:
gunzip -c stdout.gz | myScript.pl -o myoutfile.txt -i -
or you can go with:
zcat stdout.gz > yarnstdout
Source: https://askubuntu.com/questions/418771/uncompress-and-pipe-the-output-to-script
Similar questions: https://unix.stackexchange.com/questions/156261/unzipping-a-gz-file-without-removing-the-gzipped-file/156324#156324 how to pipe contents of large tar.gz file to STDOUT?
Cheers!
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.