Bash提取某些标签之间的所有行

Question

I'm trying to get a command to extract several strings depending on amount of tags in xml file. I have such file structure:

<task id="0">
some stuff
</task>

<task id="1">
some other stuff
</task>

1. How can I get all the text between opening and closing tag? I've tried awk and sed but with no success.
   
2. Will I be able to create multiple strings depending on amount of <task> tags? I mean, when I take id="0" for start, will it end on correct </task> tag or the last one in file?

Answer 1

I advise against handling xml content using line oriented tools such as grep / sed / awk etc. Xml is not a line oriented format; therefore the specific distribution of xml elements across lines when expressed textually is incidental. (You could have your example written in one single line and still it would be equally correct xml formatting.)

My suggestion for parsing well formed xml content in shell scripts is the xmlstarlet tool. It's sort of a swiss army knife for dealing with xml in a scriptable way.

First, make sure your xml content is well formed . The following is a well formed xml containing the data of your example:

<?xml version="1.0" encoding="UTF-8"?>
<tasks>
<task id="0">some stuff</task>
<task id="1">some other stuff</task>
<task id="2">yet another stuff</task>
</tasks>

(The "well formedness" of a xml file can be checked with xmlstarlet val .)

For extracting content from the xml, use xmlstarlet sel . This tool requires XPath expressions that it uses for filtering what content must be selected. (In most ways, xmlstarlet sel and Xpath are for xml what grep and regular expressions are for line oriented content.)

Examples using the above xml sample saved in file tasks.xml :

Extract content of all tasks

$ xmlstarlet sel -T -t -m '/tasks/task' -v '.' -n tasks.xml 
some stuff
some other stuff
yet another stuff

Get all task ids

$ xmlstarlet sel -T -t -m '/tasks/task' -v '@id' -n tasks.xml 
0
1
2

Extract content of task 0

$ xmlstarlet sel -T -t -m '/tasks/task[@id="0"]' -v '.' -n tasks.xml 
some stuff

Extract content of all tasks whose id is greater than or equal to 1

$ xmlstarlet sel -T -t -m '/tasks/task[@id>="1"]' -v '.' -n tasks.xml
some other stuff
yet another stuff

Naive conversion to cvs format

$ xmlstarlet sel -T -t -m '/tasks/task' -v '@id' -o ',' -v '.' -n tasks.xml 
0,some stuff
1,some other stuff
2,yet another stuff

Answer 2

On GNU sed:

sed -n '/<task id=/{n;:a;p;n;/<\/task>/!ba;s/.*/---/p;}' filename

Will output:

some stuff
---
some other stuff
---

This will search each <task id= on the file and iterate until the next </task> . The s/.*/---/p; part convert the closing tag to a separator, you can remove it and get all strings concatenated.

Answer 3

I made an HTML/XML pattern matcher for something like this.

For example for the first task you can do:

$ xidel /tmp/xxx.xml -e '<task id="0">{.}</task>'
some stuff

Or for all tasks:

$ xidel /tmp/xxx.xml -e '<task>{.}</task>+'
some stuff
some other stuff

Although in your case with only a single element, it is simpler to use XPath:

Get the first task:

$ xidel /tmp/xxx.xml -e //task[@id=0]
some stuff

Get all the task content:

$ xidel /tmp/xxx.xml -e //task
some stuff
some other stuff

Answer 4

This can be done many many ways. The easiest way in my opinion is awk. Put this in a file called task.awk :

BEGIN{x=0;}
/^<\/task>/{x=0;}
{if(x==1)print $0;}
/^<task [^>]*>/{x=1;}

And then if your xml is in task.xml you can:

awk -f task.awk < task.xml

How it works:

1. At the beginning set the flag to false.
2. Then first check to see if we should turn it off because it is a close tag
   • doing this first prevents the close tag from printing
3. Then only print the line if the flag is on
4. Finally check to see if we should turn it on because it is an open tag
   • doing this last prevents the open tag from printing

Answer 5

given this file as source in /tmp/data.xml :

<task id="0">
some1 stuff for id 0
some2 stuff for id 0
</task>

<task id="1">
some1 stuff for id 1
some2 stuff for id 1
</task>

this code:

awk '
/<task id=/{tag_data=$0} 
/<\/task>/{tag_data=tag_data $0 " "; print tag_data} 
{tag_data=tag_data $0 " "}' < /tmp/data

produces the needed result:

<task id="0"><task id="0"> some1 stuff for id 0 some2 stuff for id 0 </task> 
<task id="1"><task id="1"> some1 stuff for id 1 some2 stuff for id 1 </task> 

it does the following: it search for the first opening tag and starts accumulating data in the variable tag_data until it mets the closinig tag. At the closing tag you have all needed data between opening and closing tag in tag_data variable. You can easily modify the code to not store the tags or even parse and store the id in a separate variable.