LOAD DATA INFILE with quotes around timestamp that contains microseconds

Question

I've got a Windows batch file that reads all the CSV files in its directory. Problem I have is that the lines of the CSV have double quotes in the timestamp, and also include a microsend. I'm quite sure the below code has to be close, but its either imports a single line and dumps NULL in the timestamp column, or it indicates that that "SET was unexpected at this time" which I think is due to an unescaped character because this is being run as a Windows batch file My code is below and below that is a sample of the CSV file data.

echo on
setlocal enabledelayedexpansion
FOR %%f IN ("*.csv") DO (
set old=%%~dpnxf
set new=!old:\=\\!
mysql -e "LOAD DATA local INFILE '"!new!"' IGNORE into table db.table  COLUMNS TERMINATED BY ',' OPTIONALLY ENCLOSED BY  '\"'  (prodID,@timeStamp,Ch1,Ch2,Ch3,Ch4,Ch5,Ch6,Ch7,Ch8) SET timeStamp=STR_TO_DATE(@timeStamp,'%%Y-%%m-%%d %%H:%%i:%%s.%%f')" -u user -ppass
  echo %%~nxf DONE
)


12499,"2014-01-28 11:00:00.0",0,0,0,0,0,5,0,0
12499,"2014-01-28 12:00:00.0",0,4,0,0,1,0,0,2
12499,"2014-01-28 13:00:00.0",0,1,0,0,4,0,0,4
12499,"2014-01-28 14:00:00.0",0,0,0,0,0,0,0,7
12499,"2014-01-28 15:00:00.0",0,0,0,0,3,0,0,2

Answer 1

The problem here is that the MySQL import process requires the "%" character be used in date/time formatting. Unfortunately, that character is also claimed by the Windows batch process. The real question is "how to escape the percent characters in the SQL statement such that they are ignored by the for loop in the batch file, ie get sent through to MySQL unchanged?" The answer that seemed to work for me was to use %%^ in the SQL statement, thus:

echo on
setlocal enabledelayedexpansion
FOR %%f IN ("*.csv") DO (
set old=%%~dpnxf
set new=!old:\=\\!
mysql -e "LOAD DATA local INFILE '"!new!"' IGNORE into table db.table  COLUMNS TERMINATED BY ',' OPTIONALLY ENCLOSED BY  '\"'  (prodID,@timeStamp,Ch1,Ch2,Ch3,Ch4,Ch5,Ch6,Ch7,Ch8) SET timeStamp=STR_TO_DATE(@timeStamp,'%%^Y-%%^m-%%^d %%^H:%%^i:%%^s.%%^f')" -u user -ppass
  echo %%~nxf DONE
)

Answer 2

@ECHO OFF >NUL
SETLOCAL enableextensions enabledelayedexpansion

pushd "D:\bat\SO\files"
FOR %%f IN ("*.csv") DO (
  if /I "%%~nxf"=="31212125.csv" (
    set "old=%%~dpnxf"
    set "new=!old:\=\\!"
    echo old="!old!" new="!new!"
    echo(
    FOR /F "usebackq tokens=1,2* delims=," %%G IN ("!old!") DO (
      rem mysql -e "LOAD DATA local INFILE '"!new!"' IGNORE into table db.table  COLUMNS TERMINATED BY ',' OPTIONALLY ENCLOSED BY  '\"'  (prodID,@timeStamp,Ch1,Ch2,Ch3,Ch4,Ch5,Ch6,Ch7,Ch8) SET timeStamp=STR_TO_DATE(@timeStamp,'%%Y-%%m-%%d %%H:%%i:%%s.%%f')" -u user -ppass
      echo G=%%~G [H]=[%%~H] I=%%~I 
    )
  ) else rem echo skipped %%~nxf
)
popd

The above script shows the way how input file lines could be parsed. Operational mysql command is commented up unchanged as imho the question is about .bat scripting rather than mysql . Note that if /I "%%~nxf"=="31212125.csv" is here merely to skip my other test CSVs .

Output :

==>type D:\bat\SO\files\31212125.csv
12499,"2014-01-28 11:00:00.0",0,0,0,0,0,5,0,0
12500,"2014-01-28 12:00:00.0",0,4,0,0,1,0,0,2
12501,"2014-01-28 13:00:00.0",0,1,0,0,4,0,0,4
12502,"2014-01-28 14:00:00.0",0,0,0,0,0,0,0,7
12503,"2014-01-28 15:00:00.0",0,0,0,0,3,0,0,2

==>D:\bat\SO\31212125.bat
old="D:\bat\SO\files\31212125.csv" new="D:\\bat\\SO\\files\\31212125.csv"

G=12499 [H]=[2014-01-28 11:00:00.0] I=0,0,0,0,0,5,0,0
G=12500 [H]=[2014-01-28 12:00:00.0] I=0,4,0,0,1,0,0,2
G=12501 [H]=[2014-01-28 13:00:00.0] I=0,1,0,0,4,0,0,4
G=12502 [H]=[2014-01-28 14:00:00.0] I=0,0,0,0,0,0,0,7
G=12503 [H]=[2014-01-28 15:00:00.0] I=0,0,0,0,3,0,0,2

==>