Read Hashes from Array in Perl

Question

im new to perl (started yesterday evening). I have a problem with array of hashes.

Following situation: I have several csv-files from a logging application. The logfiles are created daily and are really bad structured. I want to group them together and show some statistics. Reading and parsing the csv isn't really a problem, but i want to store the columns inside a hash for easyer access. Inside the csv-file, the secound column (PhysName) is the column, on which i want to group the data. So i thought, it's a good idea to save only the other columns inside a hash, save several of these hashes inside an array and save the array inside an other hash, where i take the PhysName as Key.

That wasn't a big deal at all:

my %dauerauftraege;

# Glob all CSV-Files in folder
my @files = glob("*.csv");

foreach my $file(@files) {
    # read the file
    open(DATA, "<$file") or die("Could not open File");

    LINE:
    foreach my $line(<DATA>) {
        chomp($line);
        if ($line eq "") {
            next LINE;
        }

        # Split Line into Fields
        my @line_data = split(";", $line);

        my $phys_name = $line_data[1];

        # skip the header file
        if ($phys_name eq "PhysName") {
            next LINE;
        }

        my %values =(
        'date_time' => $line_data[0],
        'sender' => $line_data[2],
        'recipient' => $line_data[3],
        'format' => $line_data[4],
        'transport' => $line_data[5],
        'partnername' => $line_data[6]);

        push(\@{$dauerauftraege{"$phys_name"}}, \%values);
    }
}

When i try to verify this with print(Dumper(%dauerauftraege)) i get the following:

$VAR1 = 'YYYYYYYXXXXXXXX';
$VAR2 = [
          {
            'transport' => 'FTP',
            'format' => 'V1',
            'partnername' => 'A_TEST',
            'date_time' => '2016.07.25 11:16:52',
            'sender' => 'BBB',
            'recipient' => 'CCC'
          },
          {
            'recipient' => 'CCC',
            'sender' => 'BBB',
            'partnername' => 'A_TEST',
            'date_time' => '2016.07.25 11:17:15',
            'format' => 'V1',
            'transport' => 'FTP'
          }
        ];
$VAR3 = 'XXXXXXXYYYYYYYY';
$VAR4 = [
          {
            'format' => 'V2',
            'partnername' => 'S_TEST',
            'date_time' => '2016.07.25 10:15:02',
            'recipient' => 'DDD',
            'sender' => 'AAA',
            'transport' => 'HTTP'
          },
          {
            'transport' => 'HTTP',
            'recipient' => 'DDD',
            'sender' => 'AAA',
            'partnername' => 'S_TEST',
            'format' => 'V2',
            'date_time' => '2016.07.25 10:15:30'
          }
        ];

My first thought was, that Dumper Displays me the odd $VAR's as keys, and the even $VAR's as Values, but after i take a closer look at the hash inside the even $VAR's, i'm pretty shure i have done something wrong while inserting the inner hash (with the columns and values) into the array, stored in the %dauerauftraege Hash.

So, now when it comes to read from the Hash, containing an array of hashes with this code here

# Get the keys to iterate over them
my @dauerauftraege_keys = keys(%dauerauftraege);
print("count dauerauftraege: ".(scalar @dauerauftraege_keys)."\n");
# iterate over the keys to get the array of hashes
foreach my $dauerauftrag_key (@dauerauftraege_keys) {
    # retrive the array
    my @dauerauftrag = $dauerauftraege{"$dauerauftrag_key"};
    print("count hashes inside the array: ".(scalar @dauerauftrag)."\n");
    # iterate over the hashes and print the values
    foreach my $dauerauftrag_values (@dauerauftrag) {
        print(%{$dauerauftra_values}{'date_time'});
        print("\n");
        print(%{$dauerauftra_values}{'sender'});
        print("\n");
        print(%{$dauerauftra_values}{'recipient'});
        print("\n");
        print(%{$dauerauftra_values}{'format'});
        print("\n");
        print(%{$dauerauftra_values}{'transport'});
        print("\n");
        print(%{$dauerauftra_values}{'partnername'});
        print("\n");
    }
}

i always get this error:

count dauerauftraege: 2
count hashes inside the array: 1
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 56, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 57, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 58, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 59, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 60, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 61, <DATA> line 5.
date_timesenderrecipientformattransportpartnernamecount hashes inside the array: 1
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 56, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 57, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 58, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 59, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 60, <DATA> line 5.
Use of uninitialized value in print at ./LastRun_Evaluation.pl line 61, <DATA> line 5.
date_timesenderrecipientformattransportpartnername

That makes me belive, that my problem is in the followinig line:

push(\@{$dauerauftraege{"$phys_name"}}, \%values);

When i carefully read this line, i understand the following: $dauerauftraege{"$phys_name"} returns me a skalar ( $ ) containing an reference to the array, which is dereferenced by @{} to the Array. And now with \\%values i pass an reference to the values Hash to the push Funktion, which add it to the reference to the array, which is indicated by the \\ in front of the @.

I would be verry happy, if anyone could help me with this problem. Thank you alot guys :-)

Answer 1

It appears as though you don't quite understand how to dereference the data structure. Have a look at the following code which I've changed up a bit in hopes that'll make things a bit clearer:

my @keys = keys(%dauerauftraege);

for my $key (@keys) {

    print "*** $key ***\n";

    my $aref = $dauerauftraege{$key};

    for my $values_href (@$aref){
        print "$values_href->{date_time}\n";
        print "$values_href->{sender}\n";
        print "$values_href->{recipient}\n";
        print "$values_href->{format}\n";
        print "$values_href->{transport}\n";
        print "$values_href->{partnername}\n";
    }
    print "\n";
}

See perldsc

Answer 2

I think the problem is how you access your data structure. Try this:

my @dauerauftraege_keys = keys %dauerauftraege;
print("count dauerauftraege: ".(scalar @dauerauftraege_keys)."\n");
# iterate over the keys to get the array of hashes
foreach my $dauerauftrag_key (@dauerauftraege_keys) {
    # retrieve the array
    my $dauerauftrag = $dauerauftraege{"$dauerauftrag_key"}; # ref to array
    my @a = @$dauerauftrag;
    print("count hashes inside the array: ".(scalar @a)."\n");
    # iterate over the hashes and print the values
    foreach my $dauerauftrag_values (@a) { # also a reference
        print $dauerauftrag_values->{'date_time'} . "\n";
        print $dauerauftrag_values->{'sender'} . "\n";
        print $dauerauftrag_values->{'recipient'} . "\n";
        print $dauerauftrag_values->{'format'} . "\n";
        print $dauerauftrag_values->{'transport'} . "\n"; 
        print $dauerauftrag_values->{'partnername'} . "\n";
    }
}

Answer 3

Always use pragma use strict; and use warnings; . They help avoid typos and other problems.

%dauerauftraege is a hash, you have tried to access it like an array. And you're trying to make a loop through the hash, and at the same time to print on each iteration of the entire hash.

I think the code might look something like this:

use strict;
use warnings;
my %dauerauftraege;

# Glob all CSV-Files in folder
my @files = glob("*.csv");

foreach my $file(@files) {
    # read the file
    open(my $fh, "<", $file) or die("Could not open File");

    LINE:
    foreach my $line(<$fh>) {
        chomp($line);
        next LINE unless($line);

        # Split Line into Fields
        my @line_data = split(";", $line);

        my $phys_name = $line_data[1];

        # skip the header file
        next LINE if ($phys_name eq "PhysName");

        @{$dauerauftraege{"$phys_name"}}
          {'date_time', 'sender', 'recipient',
           'format', 'transport', 'partnername'}=@line_data[0,2..6];
    }
}

And print ...

# Get the keys to iterate over them
my @dauerauftraege_keys = keys(%dauerauftraege);
print("count dauerauftraege: ".(scalar @dauerauftraege_keys)."\n");
# iterate over the keys to get the array of hashes
foreach my $dauerauftrag_key (@dauerauftraege_keys) {
    print "\n$dauerauftrag_key:\n";
    print join("\n",
          @{$dauerauftraege{"$dauerauftrag_key"}}{'date_time','sender','recipient',
                          'format','transport','partnername'})."\n";
}