在bash中分解文本文件

Question

I have a text file in the following format:

variableStep chrom=chr1 span=10
10161   1
10171   1
10181   2
10191   2
10201   2
10211   2
10221   2
10231   2
10241   2
10251   1
variableStep chrom=chr10 span=10
70711   1
70721   2
70731   2
70741   2
70751   2
70761   2
70771   2
70781   2
70791   1
71161   1
71171   1
71181   1
variableStep chrom=chr11 span=10
104731  1
104741  1
104751  1
104761  1
104771  1
104781  1
104791  1
104801  1
128711  1
128721  1
128731  1

I need a way to break this down into several files named for example "chr1.txt", "chr10.txt and "chr11.txt". How would I go about doing this?

I about the the following way:

cat file.txt | \
while IFS=$'\t' read  -r -a rowArray; do
    echo -e "${rowArray[0]}\t${rowArray[1]}\t${rowArray[2]}"
done > $file.mod.txt

That reads line by line and then saves line by line. However, I need something a little more elaborate that spans rows. "chr1.txt" would include everything from the row 10161 1 to row 10251 1, "chr10.txt" would include everything from the row 70711 1 to row 71181 1, etc. It's also specific in that I have to read in the actual chr# from each line as well, and save that as the file name.

The help is really appreciated.

Answer 1

awk -F'[ =]' '
  $1 == "variableStep" {file = $3 ".txt"; next}
  file != "" {print > file}' < input.txt

Answer 2

This worked for me:

IFS=$'\n'
curfile=""
content=($(< file.txt))
for ((idx = 0; idx < ${#content[@]}; idx++)); do
    if [[ ${content[idx]} =~ ^.*chrom=(\\b.*?\\b)\ .*$ ]]; then
        curfile="${BASH_REMATCH[1]}.txt"
        rm -rf ${curfile}
    elif [ -n "${curfile}" ]; then
        echo ${content[idx]} >> ${curfile}
    fi
done

Answer 3

Awk is appropriate for this problem domain because the text file is already (more or less) organized into columns. Here's what I would use:

awk 'NF == 3 && index($2, "=") { filename = substr($2, index($2, "=") + 1) }
     NF == 2 && filename { print $0 > (filename ".txt") }' < input.txt

Explanation:

Think of the lines starting with variableStep as "three columns" and the other lines as "two columns". The above script says, "Parse the text file line-by-line; if a line has three columns and the second column contains an '=' character, assign 'all of the characters in the second column that occur after the '=' character' to a variable called filename . If a line has two columns and the filename variable's been assigned, write the entire line to the file that's constructed by concatenating the string in the filename variable with '.txt'".

Notes:

• NF is a built-in variable in Awk that represents the "number of fields", where a "field" (in this case) can be thought of as a column of data.
• $0 and $2 are built-in variables that represent the entire line and the second column of data, respectively. ( $1 represents the first column, $3 represents the third column, etc...)
• substr and index are built-in functions described here: http://www.gnu.org/software/gawk/manual/gawk.html#String-Functions The redirection operator (>) acts differently in Awk than it does in a shell script; subsequent writes to the same file are appended.
• String concatenation is performed simply by writing expressions next to each other. The parenthesis ensure the concatenation happens before the file gets written to.

More details can be found here: http://www.gnu.org/software/gawk/manual/gawk.html#Two-Rules

Answer 4

i used sed to filter ....

code part :

Kaizen ~/so_test $ cat zsplit.sh

cntr=1;
prev=1;
for curr in `cat ztmpfile2.txt | nl | grep variableStep | tr -s " " | cut -d" " -f2 | sed -n 's/variableStep//p'`
do
sed -n "$prev,$(( ${curr} - 1))p" ztmpfile2.txt > zchap$cntr.txt ;
#echo "displaying : : zchap$cntr.txt " ;
#cat zchap$cntr.txt ;
prev=$curr; cntr=$(( $cntr + 1 ));
done

 sed -n "$prev,$ p" ztmpfile2.txt > zchap$cntr.txt ;
 #echo "displaying : : zchap$cntr.txt " ;
 #cat zchap$cntr.txt ;

output :

Kaizen ~/so_test $  ./zsplit.sh
+ ./zsplit.sh
zchap1.txt :: 1 :: 1
displaying : : zchap1.txt
variableStep chrom=chr1 span=10
zchap2.txt :: 1 :: 12
displaying : : zchap2.txt
variableStep chrom=chr1 span=10
10161   1
10171   1
10181   2
10191   2
10201   2
10211   2
10221   2
10231   2
10241   2
10251   1
zchap3.txt :: 12 :: 25
displaying : : zchap3.txt
 variableStep chrom=chr10 span=10
70711   1
70721   2
70731   2
70741   2
70751   2
70761   2
70771   2
70781   2
70791   1
71161   1
71171   1
71181   1
displaying : : zchap4.txt
variableStep chrom=chr11 span=10
104731  1
104741  1
104751  1
104761  1
104771  1
104781  1
104791  1
104801  1
128711  1
128721  1
128731  1

from the result zchap* files , iff you want you can remove the line : variableStep chrom=chr11 span=10 by using sed -- sed -i '/variableStep/d' zchap*

does this help ?