在Solr管理员中上传CSV文件时出现问题

Question

I have a application which has a search facility and i am using Solr for searching. I am trying to upload the data as CSV file. But the data is not uploading to the Solr core properly.

Here is the curl command i am using

curl 'http://localhost:8983/solr/test_import/update/csv?commit=true&separator=%09&escape=%5c&encapsulator=%22' --data-binary @/tmp/college_data_20180809164959.csv -H 'Content-type:application/csv'

this gives me an error

java.io.IOException: (line 0) invalid char between encapsulated token end delimiter\\n\\tat org.apache.solr.internal.csv.CSVParser.encapsulatedTokenLexer

If remove encapsulator=%22 it uploads but not in good format.

This his how it got uploaded:

{
        "id":"8adb5378-aa58-427d-8ff4-fca4f31c96e6",
        "ID_College_Name_State_City_Address":["43387,,,,"],
        "_version_":1608318488833687552,
        "ID_College_Name_State_City_Address_str":["43387,,,,"]},
      {
        "id":"e29a0435-95c5-4d3c-bddf-eacef22f6859",
        "ID_College_Name_State_City_Address":["43388,apsce,,,"],
        "_version_":1608318488835784704,
        "ID_College_Name_State_City_Address_str":["43388,apsce,,,"]}

This is my csv file structure

"ID","College_Name","State","City","Address"
"43387","","","",""
"43388","apsce","","",""

Please help me in resolving this issue. Please let me know if you need any further information about this problem.

Answer 1

Your CSV file should be parsed perfectly fined with the default values for CSV parsing. Drop all the parameters you're giving.

The error message is because you've given the separator parameter as %09 , which is the TAB character. Your values are not separated by a TAB character, but by the standard , .

separator=%09 # separated by TAB (wrong)
escape=%5c # escaped by \ (default)
encapsulator=%22 # encapsulated by " (default)

Since the parser is looking for values separated by <TAB> , having multiple " between separators indicates a parse error (which happens since , isn't given as a separator).