Extract a block from file between two lines

Question

I have a file/output containing this :

igw_id = igw-96788cf1
private_route_tables_ids = [
    rtb-c2adcda4,
    rtb-c5a3c3a3,
    rtb-c4adcda2
]
private_subnets_cidrs_ipv4 = [
    10.20.10.0/24,
    10.20.11.0/24,
    10.20.12.0/24
]
private_subnets_ids = [
    subnet-6057333b,
    subnet-6be7bf0c,
    subnet-f13419b8
]
public_route_tables_ids = [
    rtb-74a9c912,
    rtb-c5adcda3,
    rtb-2aabcb4c
]
public_subnets_cidrs_ipv4 = [
    10.20.0.0/24,
    10.20.1.0/24,
    10.20.2.0/24
]
public_subnets_ids = [
    subnet-6157333a,
    subnet-17e7bf70,
    subnet-303f1279
]

I would like to extract all public subnet id and print them without, and white space.

I used this regex

sed -n '/public_subnets_ids/{:a;N;/\]/!ba;s/[[:space:]]//g;s/,/\n/g;s/.*public_subnets_ids\|\].*//g;p}' my_file.txt

And the output is :

=[subnet-6157333a
subnet-17e7bf70
subnet-303f1279

But I would like to get this instead:

subnet-6157333a
subnet-17e7bf70
subnet-303f1279

In fact I told sed to replace spaces and newlines with nothing (s/[[:space:]]//g) and then it also replace the first new line and then brings the first subnet up, so I would to process the regex after the first newline and when I try this

sed -n '/public_subnets_ids = \[[\n\r\s]+\\n/{:a;N;/\]/!ba;s/[[:space:]]//g;s/,/\n/g;s/.*public_subnets_ids\|\].*//g;p}' my_file.txt

It gives no ouput, means it doesn't match anything.

Please can help improve the above regex to give only subnets ids in seperated lines?

Answer 1

awk to the rescue!

$ awk '/^]/{f=0} f{$1=$1; sub(",",""); print} 
       /public_subnets_ids/{f=1}' file

subnet-6157333a
subnet-17e7bf70
subnet-303f1279

Answer 2

Following awk may also help you in same.

awk '/public_subnets_ids/{getline;while($0 !~ /]/){gsub(/^[[:space:]]+|,/,"");print;getline}}'   Input_file

EDIT: Adding non-one liner form of solution with explanation also on same.

awk '
/public_subnets_ids/{         ##Searching for string public_subnets_ids in a line, if yes then do following.
 getline;                     ##Going to next line by using getline utility of awk.
 while($0 !~ /]/){            ##Using while loop till a line is NOT having ]
   gsub(/^[[:space:]]+|,/,"");##Globally substituting initial space and comma with NULL in all lines here.
   print;                     ##Printing the lines here.
   getline                    ##using getline cursor will move to next line of Input_file.
}
}'  Input_file                ##Mentioning the Input_file name here.

Answer 3

Another approach this sed:

sed -n '/public_subnets_ids/,/]/{//!{s/[ ,]*//g;p;}}' file

• '/public_subnets_ids/,/]/ : from line containing public_subnets_ids up to next line containing ]
• //! : in matching lines, except those matching the addresses
• //!{s/[ ,]*//g;p;} : removes commas and space characters and output result