Import CSV data into MySQL

Question

Consider the following snippet CSV data from "NASDAQ.csv"

"Symbol,""Name"",""LastSale"",""MarketCap"",""ADR TSO"",""IPOyear"",""Sector"",""industry"",""Summary Quote"",";;
"FLWS,""1-800 FLOWERS.COM, Inc."",""2.9"",""81745200"",""n/a"",""1999"",""Consumer Services"",""Other Specialty Stores"",""http://www.nasdaq.com/symbol/flws"",";;
"FCTY,""1st Century Bancshares, Inc"",""4"",""36172000"",""n/a"",""n/a"",""Finance"",""Major Banks"",""http://www.nasdaq.com/symbol/fcty"",";;
"FCCY,""1st Constitution Bancorp (NJ)"",""8.8999"",""44908895.4"",""n/a"",""n/a"",""Finance"",""Savings Institutions"",""http://www.nasdaq.com/symbol/fccy"",";;

I'm trying to import Symbol, Sector, and Industry into a MySQL table with corresponding fields:

$path = "NASDAQ.csv";
$row = 1;
if (($handle = fopen($path, "r")) !== FALSE) {
  while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
    $row++;
    $entries[] = $data ;
  }
  fclose($handle);
}

foreach ($entries as $line) {
  db_query("
     INSERT INTO us_stocks (symbol, name, sector, industry) 
     VALUES ('%s', '%s', '%s', '%s', '%s')",
     $line[0], $line[1], $line[6], $line[7]
  );
}

The result, however, is not what I expected. In the database, only the Symbol field gets filled, and not even correctly:

symbol      name  sector  industry
----------------------------------
Symbol,"Na
FLWS,"1-80
FCTY,"1st
FCCY,"1st

What am I doing wrong?

[edit]

If I print_r($entries), the output looks like

Array (
  [0] => Array(
    [0] => Symbol,"Name","LastSale","MarketCap","ADR TSO","IPOyear","Sector","industry","Summary Quote",;;
  )
  [1] => Array(
    [0] => FLWS,"1-800 FLOWERS.COM, Inc.","2.9","81745200","n/a","1999","Consumer Services","Other Specialty Stores","http://www.nasdaq.com/symbol/flws",;;
  )
  [2] => Array(
    [0] => FCTY,"1st Century Bancshares, Inc","4","36172000","n/a","n/a","Finance","Major Banks","http://www.nasdaq.com/symbol/fcty",;;
  )
)

[edit2]

I have deleted the first line of the CSV, as suggested. I now have a very quick and dirty way of almost accomplishing what I want. Basically, the thing messes up whenever there's a company name with ", Inc" in it. So I just "glue" it to the name above: $data[1] = $data[1] . $data[2]:

$path = "NASDAQ.csv";
$row = 1;
if (($handle = fopen($path, "r")) !== FALSE) {
  while (($data = fgetcsv($handle, 1000, ";;")) !== FALSE) {
    if ($row < 100) {
      $row++;
      $data = explode(',', $data[0]);
      if (substr($data[2], 0, 1) == ' ') {
        $data[1] = $data[1] . $data[2];
        unset($data[2]);
      }
      $entries[] = $data ;
    }
  }
  fclose($handle);
}

A print_r($entries) now gives:

[0] => Array
    (
        [0] => FLWS
        [1] => "1-800 FLOWERS.COM Inc."
        [3] => "2.9"
        [4] => "81745200"
        [5] => "n/a"
        [6] => "1999"
        [7] => "Consumer Services"
        [8] => "Other Specialty Stores"
        [9] => "http://www.nasdaq.com/symbol/flws"
        [10] => 
    )

Final problem: I don't know how to renumber the keys. So 3 into 2, 4 into 3, etc. so that the output looks like:

[0] => Array
    (
        [0] => FLWS
        [1] => "1-800 FLOWERS.COM Inc."
        [2] => "2.9"
        [3] => "81745200"
        [4] => "n/a"
        [5] => "1999"
        [6] => "Consumer Services"
        [7] => "Other Specialty Stores"
        [8] => "http://www.nasdaq.com/symbol/flws"
        [9] => 
    )

Any help would be greatly appreciated!

Answer 1

I'd say the data isn't "truely" CSV.

"FLWS,""1-800 FLOWERS.COM, Inc."",""2.9"", should be : "FLWS","1-800 FLOWERS.COM, INC.","2.9" - The quotes should wrap the individual fields with commas seperating each field. Usually numeric fields are not wrapped.

Depending on how you load the data, comma's in the data may confuse it. (ie the FLOWERS.COM, INC"

By the way - if it's really CSV - look at: http://dev.mysql.com/doc/refman/5.1/en/load-data.html

Answer 2

As Crontab said, probably it's a problem with quotes. Try:

foreach ($entries as $line) {

  // Escape (see mysql_real_escape_string too) and remove double quotes
  foreach ($line as $k => $v) $line[$k] = mysql_escape_string(trim($v, '"'));

  // Rebuild array
  $line = array_values($line);

  db_query("
    INSERT INTO us_stocks (symbol, name, sector, industry) 
    VALUES ('%s', '%s', '%s', '%s', '%s')",
    $line[0], $line[1], $line[6], $line[7]
 );

}

PS: I don't know if you already escape strings in db_query() .