Using sed in bash to replace lines in a xml file

Question

I'm writing a bash script to change some values in an xml file for hadoop. I've used sed a wee bit before, nothing too complex.

Here is the line in question:

sed "s@<configuration>@${toedit}@g output.txt" /usr/local/hadooptest/etc/hadoop/core-site.xml

I'm using output.txt to test the output. The finished script will insert the code stored in $toedit into this:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
</configuration>

I want to replace the configuration tags with the variable $toedit . In $toedit I saved:

toedit='<configuration>
<property>
  <name>hadoop.tmp.dir</name>
  <value>/app/hadoop/tmp</value>
  <description>A base for other temporary directories.</description>
</property>

<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
</property>'

From googling about the sililar issues that have come up in the past, I know there's something wrong with my variable (either due to {} brackets or '' - the short story being it's empty) but I've no idea why!

I've tried single quotes, double quotes, none, different sed options I seen online (granted some of them I was just following the given answer and hoping for the best) but I've had no luck.

I think it's something tiny, or something silly but I've hit a wall! I see sed questions all over the site but the options I found didn't help me.

I should say that this all is contained in a bash script and the end result is meant to install and configure hadoop according to this tutorial: http://www.bogotobogo.com/Hadoop/BigData_hadoop_Install_on_ubuntu_single_node_cluster.php

I'm practising scripting and I thought it would be a good challenge to try!

Thank you

Answer 1

sed won't allow multiple unescaped newlines in replacement text. You can use awk:

awk -v toedit="$toedit" '/<configuration>/{print toedit; next} 1' file
http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>hadoop.tmp.dir</name>
  <value>/app/hadoop/tmp</value>
  <description>A base for other temporary directories.</description>
</property>

<property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:54310</value>
</property>
</configuration>

Or use perl :

perl -pe "s@<configuration>@$toedit@g" file

Answer 2

Using text processing tools to alter XML can break the XML structure badly if the XML is formatted differently than expected. Use a proper XML handling tool. For example, in xsh :

open /usr/local/hadooptest/etc/hadoop/core-site.xml ;
cd //configuration ;
insert element property into  . ;

cd property ;
set name        'hadoop.tmp.dir' ;
set value       'app/hadoop/tmp' ;
set description 'A base for other temporary directories.' ;

cd .. ;
insert element property append . ;
cd property[2] ;
set name  'fs.default.name' ;
set value 'hdfs://localhost:54310' ;

xinsert text {"\n"} before ..//* ; # Some basic formatting.
save :b ;                          # Create a backup.