Read CSV file and save in 2 d array

Question

I am trying to read a huge CSV file in 2 D array, there must be a better way to split the line and save it in the 2 D array in one step :s Cheers

my $j = 0;
while (<IN>) 
{

    chomp ;
    my @cols=();
    @cols   = split(/,/); 
    shift(@cols) ; #to remove the first number which is a line header
    for(my $i=0; $i<11; $i++) 
    {
       $array[$i][$j]  = $cols[$i];
    }        
    $j++;    
}

Answer 1

CSV is not trivial. Don't parse it yourself. Use a module like Text::CSV , which will do it correctly and fast.

use strict;
use warnings;

use Text::CSV;

my @data;   # 2D array for CSV data
my $file = 'something.csv';

my $csv = Text::CSV->new;
open my $fh, '<', $file or die "Could not open $file: $!";

while( my $row = $csv->getline( $fh ) ) { 
    shift @$row;        # throw away first value
    push @data, $row;
}

That will get all your rows nicely in @data , without worrying about parsing CSV yourself.

Answer 2

If you ever find yourself reaching for the C-style for loop, then there's a good chance that your program design can be improved.

while (<IN>) {
    chomp;

    my @cols = split(/,/); 
    shift(@cols); #to remove the first number which is a line header

    push @array, \@cols;
}

This assumes that you have a CSV file that can be processed with a simple split (ie the records contain no embedded commas).

Answer 3

Aside: You can simplify your code with:

my @cols = split /,/;

---

Your assignment to $array[$col][$row] uses an unusual subscript order; it complicates life. With your column/row assignment order in the array, I don't think there's a simpler way to do it.

---

Alternative: If you were to reverse the order of the subscripts in the array ( $array[$row][$col] ), you could think about using:

use strict;
use warnings;

my @array;
for (my $j = 0; <>; $j++) # For testing I used <> instead of <IN>
{
    chomp;
    $array[$j] = [ split /,/ ];
    shift @{$array[$j]};   # Remove the line label
}

for (my $i = 0; $i < scalar(@array); $i++)
{
    for (my $j = 0; $j < scalar(@{$array[$i]}); $j++)
    {
        print "array[$i,$j] = $array[$i][$j]\n";
    }
}

Sample Data

label1,1,2,3
label2,3,2,1
label3,2,3,1

Sample Output

array[0,0] = 1
array[0,1] = 2
array[0,2] = 3
array[1,0] = 3
array[1,1] = 2
array[1,2] = 1
array[2,0] = 2
array[2,1] = 3
array[2,2] = 1