How to extract the text between two patterns using REGEX perl

Question

In the following lines how can I store the lines between " Description: " and " Tag: " in a variable using REGEX PERL and what would be a good datatype to use, string or list or something else?

(I am trying to write a program in Perl to extract the information of a text file with Debian package information and convert it into a RDF(OWL) file(ontology).)

Description: library for decoding ATSC A/52 streams (development) liba52 is a free library for decoding ATSC A/52 streams. The A/52 standard is used in a variety of applications, including digital television and DVD. It is also known as AC-3.

This package contains the development files. Homepage: http://liba52.sourceforge.net/

Tag: devel::library, role::devel-lib

The code I have written so far is:

#!/usr/bin/perl
open(DEB,"Packages");
open(ONT,">>debianmodelling.txt");

$i=0;
while(my $line = <DEB>)
{

    if($line =~ /Package/)
    {
        $line =~ s/Package: //;
        print ONT '  <package rdf:ID="instance'.$i.'">';
        print ONT    '    <name rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</name>'."\n";
    }
elsif($line =~ /Priority/)
{
    $line =~ s/Priority: //;
    print ONT '    <priority rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</priority>'."\n";
}

elsif($line =~ /Section/)
{
    $line =~ s/Section: //;
    print ONT '    <Section rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</Section>'."\n";
}

elsif($line =~ /Maintainer/)
{
    $line =~ s/Maintainer: //;
    print ONT '    <maintainer rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</maintainer>'."\n";
}

elsif($line =~ /Architecture/)
{
    $line =~ s/Architecture: //;
    print ONT '    <architecture rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</architecture>'."\n";
}
elsif($line =~ /Version/)
{
    $line =~ s/Version: //;
    print ONT '    <version rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</version>'."\n";
}
elsif($line =~ /Provides/)
{
    $line =~ s/Provides: //;
    print ONT '    <provides rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</provides>'."\n";
}
elsif($line =~ /Depends/)
{
    $line =~ s/Depends: //;
    print ONT '    <depends rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</depends>'."\n";
}
elsif($line =~ /Suggests/)
{
    $line =~ s/Suggests: //;
    print ONT '    <suggests rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</suggests>'."\n";
}

elsif($line =~ /Description/)
{
    $line =~ s/Description: //;
    print ONT '    <Description rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</Description>'."\n";
}
elsif($line =~ /Tag/)
{
    $line =~ s/Tag: //;
    print ONT '    <Tag rdf:datatype="http://www.w3.org/2001/XMLSchema#string">'.$line.'</Tag>'."\n";
    print ONT '  </Package>'."\n\n";
}
$i=$i+1;
}

Answer 1

my $desc = "Description:";
my $tag  = "Tag:";

$line =~ /$desc(.*?)$tag/;
my $matched = $1;
print $matched;

---

or

---

my $desc = "Description:";
my $tag  = "Tag:";

my @matched = $line =~ /$desc(.*?)$tag/;
print $matched[0];

---

or

---

my $desc = "Description:";
my $tag  = "Tag:";

(my $matched = $line) =~ s/$desc(.*?)$tag/$1/;
print $matched;

---

Additional

---

If your Description and Tag may be on separate lines, you may need to use the /s modifier, to treat it as a single line, so the \n won't wreck it. Example:

$_=qq{Description:foo 
      more description on 
      new line Tag: some
      tag};
s/Description:(.*?)Tag:/$1/s; #notice the trailing slash
print;

Answer 2

Assuming:

my $example; # holds the example text above

You could:

(my $result=$example)=~s/^.*?\n(Description:)/$1/s; # strip up to first marker

$result=~s/(\nTag:[^\n]*\n).+$/$1/s; # strip everything after second marker line

Or

(my $result=$example)=~s/^.*?\n(Description:.+?Tag:[^\n]*\n).*$/$1/s;

Both assume the Tag: value is contained on a single line.

If this is not the case, you might try:

(my $result=$example)=~s/
    (                        # start capture
        Description:         # literal 'Description:'
        .+?                  # any chars (non-greedy) up to
        Tag:                 # literal 'Tag:'
        .+?                  # any chars up to
    )
    (?:                      # either
      \n[A-Z][a-z]+\:        #  another tagged value name 
    |                         # or
      $                       #  end of string
    )
/$1/sx;

Answer 3

I believe that the problem is caused by using a line reading loop for data structured by paragraphs. If you can slurp the file into memory and and apply split with a captured delimiter, the processing will be much smoother:

#!/usr/bin/perl -w

use strict;
use diagnostics;
use warnings;

use English;

# simple sample sub
my $printhead = sub {
  printf "%5s got the tag '%s ...'\n", '', substr( shift, 0, 30 );
};
# map keys/tags? to functions
my %tagsoups = (
    'PackageName' => sub {printf "%5s got the name '%s'\n", '', shift;}
  , 'Description' => sub {printf "%5s got the description:\n---------\n%s\n----------\n", '', shift;}
  , 'Tag'         => $printhead
);
# slurp Packages (fallback: parse using $INPUT_RECORD_SEPARATOR = "Package:")
open my $fh, "<", './Packages-00.txt' or die $!;
local $/; # enable localized slurp mode
my $all = <$fh>;
my @pks = split /^(Package):\s+/ms, $all;
close $fh;
# outer loop: Packages
for (my $p = 1, my $n = 0; $p < scalar @pks; $p +=2) {
  my $blk = "PackageName: " . $pks[$p + 1];
  my @inf = split /\s*^([\w-]+):\s+/ms, $blk;
  printf "%3d %s named %s\n", ++$n, $pks[$p], $inf[ 2 ];
  # outer loop: key-value-pairs (or whatever they are called)
  for (my $x = 1; $x < scalar @inf; $x += 2) {
      if (exists($tagsoups{$inf[ $x ]})) {
          $tagsoups{$inf[ $x ]}($inf[$x + 1]);
      }
  }
}

output for a shortened Packages file from my Ubuntu Linux:

  3 Package named abrowser-3.5-branding
      got the PackageName:
---------
abrowser-3.5-branding
----------
      got the Description:
---------
dummy upgrade package for firefox-3.5 -> firefox
 This is a transitional package so firefox-3.5 users get firefox on
 upgrades. It can be safely removed.
----------
  4 Package named casper
      got the PackageName:
---------
casper
----------
      got the Description:
---------
Run a "live" preinstalled system from read-only media
----------
      got the Tag:
---------
admin::boot, admin::filesystem, implemented-in::shell, protocol::smb, role::plugin, scope::utility, special::c
ompletely-tagged, works-with-format::iso9660
----------

Using a hash for the functions to apply to the extracted parts will keep the details of generating xml out of the parser loops.