在shell脚本中执行hadoopfs

Question

I trying to run a bash script of the following :

#!/bin/bash
CURRENT_HDFS_PATH=`hadoopfs -ls -t -r /$CLEAN_HDFS_PATH | tail -1 | awk -F ' ' '{print $8}'`
echo "Here is the last (most current) file in the history folder to be downloaded=$CURRENT_HDFS_PATH"

The above does not produce any result at all. Please note that CLEAN_HDFS_PATH=/temp/local-*.inprogress

When I use the following in command line:

hadoopfs -ls -t -r  '/temp/local-*.inprogress' | tail -1 | awk -F ' ' '{print $8}'

I get the answer from the command line.

What am I doing wrong in my script ?

Cheers,

Answer 1

Is the name of the file literally local-*.inprogress ? If so, your problem is wildcard expansion within script. Add double quotes around the variable and see if that works like:

CURRENT_HDFS_PATH=`hadoopfs -ls -t -r "/$CLEAN_HDFS_PATH" | tail -1 | awk -F ' ' '{print $8}'`