got unexpected EOF while looking for matching `"' error when running on mac

Question

run_cmd="spark-submit \
$SPARK_OPTIONS \
--conf spark.hadoop.fs.default.name=file:/// \
--conf spark.hadoop.fs.defaultFS=file:/// \
--py-files \
${TARGET}/test.zip \
$TEST_PY \
$RAW_DATA_FILE \
$OUTPUT \
--route $AGG_OUTPUT1 \
--origin $AGG_OUTPUT2 \
--first $AGG_OUTPUT3" #line 71

echo $run_cmd
echo $run_cmd | bash 
#line 75                       

The code is like above, it can run successfully on Ubuntu. However, when I run it on my macbook, the spark-submit finishes normally and output is also generated correctly, but then it outputs an error, it really sounds unreasonable. Also, if spark-submit exited abnormally, it won't trigger this error.

./test.sh: line 71: unexpected EOF while looking for matching `"'
./test.sh: line 75: syntax error: unexpected end of file

Answer 1

You haven't posted all the relevant code, only some lines near 60~75. The error you are getting happens when you have an unclosed " somewhere before the posted code. For example:

a="

b="something"

If you run this script with bash , it will report:

script.sh: line 3: unexpected EOF while looking for matching `"'
script.sh: line 4: syntax error: unexpected end of file

As in your case, the error is reported not on the line that didn't close the " , but somewhere else. What happens, Bash interprets the value of a as \\n\\nb= , and then there is an opening " after something that's never closed.

The same thing is happening in your code. Look for a " that isn't properly closed earlier in your script.