将Hive查询输出存储在Shell脚本变量中

Question

When i try to execute the below 2 queries via shell script, I'm unable to echo the output:

Querys:

close_val=`hive -e "select count(1) from tst_db.dim_calendar where calendar_id =20171102"`
close_val=$(hive -S -e "USE cars_dev; select count(1) from dim_calendar where calendar_id =20171103")

echo $close_val

Error:

2017-11-03 00:55:11,505 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 2017-11-03 00:55:11,508 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 2017-11-03 00:55:11,508 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 2017-11-03 00:55:11,508 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 2017-11-03 00:55:11,508 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 2017-11-03 00:55:11,508 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 2017-11-03 00:55:11,508 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1011)) - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2017-11-03 00:55:11,715 WARN [main] conf.HiveConf (HiveConf.java:initialize(1155)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore. 1 ./run_daily_jobs_night.sh: line 42: [: too many arguments

Answer 1

I'm not sure of what you're expecting to do with the close_val value but in your situation you seem to expect a number however it seems there's an issue in hive and you get a stacktrace instead. The trace received is not a simple string but is viewed as several parameters to your echo which exceed the number it handles.

using a command like:

$ # Notice the cut at the end of the command
$ close_val=$(hive -S -e "USE cars_dev; select count(1) from dim_calendar where calendar_id =20171103" | cut -f 1)

will insure that your echo command will work, but here your real issue is your hive command which is failing