[英]Is there any way to check if file exists in Apache beam sdk without using pipeline
I am currently trying to check if file exists on cluster by using the MatchResult status. 我目前正在尝试使用MatchResult状态检查文件在群集上是否存在。 Even if the file exists it is returning the status as NOT_FOUND. 即使文件存在,它也会返回状态为NOT_FOUND。
MatchResult matchResult = FileSystems.match(output string)
if(matchResult.status() == MatchResult.Status.NOT_FOUND){
//do some operation
//print file does not exist
}
else{
//print file exists
}
the above code is returning status as not found even if file exists. 即使文件存在,上述代码仍返回未找到的状态。
What kind of file system is on your cluster? 您的集群上使用哪种文件系统? AFAIK, Beam file system supports only S3, GCS, HDFS and local FS. AFAIK,Beam文件系统仅支持S3,GCS,HDFS和本地FS。 Also, it would worth to print out "matchResult" and see the full path if it matches with your real path on cluster. 另外,如果它与群集上的实际路径匹配,则值得打印出“ matchResult”并查看完整路径。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.