Importing CSV with odd rows into MySQL

Question

I'm faced with a problematic CSV file that I have to import to MySQL.
Either through the use of PHP and then insert commands, or straight through MySQL's load data infile.

I have attached a partial screenshot of how the data within the file looks:

The values I need to insert are below "ACC1000" so I have to start at line 5 and make my way through the file of about 5500 lines.

It's not possible to skip to each next line because for some Accounts there are multiple payments as shown below.

I have been trying to get to the next row by scanning the rows for the occurrence of "ACC"

if (strpos($data[$c], 'ACC') !== FALSE){
    echo "Yep ";
} else {
    echo "Nope ";
}

I know it's crude, but I really don't know where to start.

Answer 1

If you have a (foreign key) constraint defined in your target table such that records with a blank value in the type column will be rejected, you could use MySQL's LOAD DATA INFILE to read the first column into a user variable (which is carried forward into subsequent records) and apply its IGNORE keyword to skip those "records" that fail the FK constraint:

LOAD DATA INFILE '/path/to/file.csv'
    IGNORE
    INTO TABLE my_table
    CHARACTER SET utf8
    FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
    LINES  TERMINATED BY '\r\n'
    IGNORE 4 LINES
    (@a, type, date, terms, due_date, class, aging, balance)
    SET account_no = @account_no := IF(@a='', @account_no, @a)

Answer 2

There are several approaches you could take.

1) You could go with @Jorge Campos suggestion and read the file line by line, using PHP code to skip the lines you don't need and insert the ones you want into MySQL. A potential disadvantage to this approach if you have a very large file is that you will either have to run a bunch of little queries or build up a larger one and it could take some time to run.

2) You could process the file and remove any rows/columns that you don't need, leaving the file in a format that can be inserted directly into mysql via command line or whatever.

Based on which approach you decide to take, either myself or the community can provide code samples if you need them.

Answer 3

This snippet should get you going in the right direction:

$file = '/path/to/something.csv';

if( ! fopen($file, 'r') ) { die('bad file'); }

if( ! $headers = fgetcsv($fh) ) { die('bad data'); }

while($line = fgetcsv($fh)) {
  echo var_export($line, true) . "\n";
  if( preg_match('/^ACC/', $line[0] ) { echo "record begin\n"; }
}

fclose($fh);

http://php.net/manual/en/function.fgetcsv.php