Apache Drill query on multiple directories not working

Question

My data files are partitioned in subdirectories by year.

If i query using year directory, sql works fine.

select  count(*) from dfs.`/drill/perf/logs/2010`;

But if i query the parent directory getting following error.

select  count(*) from dfs.`/drill/perf/logs`;

Error

Jan 14, 2015 1:43:52 PM org.eigenbase.sql.validate.SqlValidatorException <init>
SEVERE: org.eigenbase.sql.validate.SqlValidatorException: Table 'dfs./drill/perf/logs' not found
Jan 14, 2015 1:43:52 PM org.eigenbase.util.EigenbaseException <init>
SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1, column 23 to line 1, column 25:     Table 'dfs./drill/perf/logs' not found
Query failed: Query failed: Failure validating SQL. org.eigenbase.util.EigenbaseContextException:   From line 1, column 23 to line 1, column 25: Table 'dfs./drill/perf/logs' not found 

Drill documentation links:

https://cwiki.apache.org/confluence/display/DRILL/Querying+Directories https://cwiki.apache.org/confluence/display/DRILL/Partition+Pruning

Answer 1

I found the issue. Issue was when i created subdirectories in mac using finder, it created .DS_Store hidden file, which is messing up the Apache Drill Query.