在Ruby中读取包含特殊字符的CSV文件并存储到SQL Server中

Question

I'm trying to import a CSV file (UTF-8 encoding) in Ruby (2.0.0) in to my database (MSSQL 2008R2, COLLATION French_CI_AS ), but the special characters (French accents on vowels) are not stored properly : éèçôü becomes éèçôü (or other similar jibberish).

I use this piece of code to read the file :

CSV.foreach(file, col_sep: ';', encoding: "utf-8") do |row|
   # ...
end

I tried various encoding in the CSV options ( utf-8 , iso-8859-1 , windows-1252 ), but none would store the special characters correctly.

Before you ask, my database collation supports those characters, since we have successfully imported data containing those using PHP importers. If I dump the data using puts or a file logger, everything is correct.

Is something wrong with my code, or do I need to specify something else (like the ruby class file encoding for example) ?

Thanks

EDIT : The data saving is done by a PHP REST API that works fine with accented characters. It stores data as it is received.

In Ruby, I parse my data, store it in an object and then send the JSON-encoded object in the body of my PUT request. But if I use an SQL query directly from Ruby, the problem remains :

query = <<-SQL
    UPDATE MyTable SET MyTable_title = '#{row_data['title']}' WHERE MyTable_id = '#{row_data['id']}'
SQL
res = db.execute query

Answer 1

I was thinking that this had something to do with the encoding type on your CSV file, so started digging around on that. I did find that windows-1252 encoding will insert control characters.

You can read more about it here: Converting special charactes such as ü and à back to their original, latin alphbet counterparts in C#