简体   繁体   English

有没有办法在不使用管道的情况下检查Apache Beam sdk中是否存在文件

[英]Is there any way to check if file exists in Apache beam sdk without using pipeline

I am currently trying to check if file exists on cluster by using the MatchResult status. 我目前正在尝试使用MatchResult状态检查文件在群集上是否存在。 Even if the file exists it is returning the status as NOT_FOUND. 即使文件存在,它也会返回状态为NOT_FOUND。

MatchResult matchResult = FileSystems.match(output string)

if(matchResult.status() == MatchResult.Status.NOT_FOUND){
//do some operation 
//print file does not exist
}
else{
//print file exists
}

the above code is returning status as not found even if file exists. 即使文件存在,上述代码仍返回未找到的状态。

What kind of file system is on your cluster? 您的集群上使用哪种文件系统? AFAIK, Beam file system supports only S3, GCS, HDFS and local FS. AFAIK,Beam文件系统仅支持S3,GCS,HDFS和本地FS。 Also, it would worth to print out "matchResult" and see the full path if it matches with your real path on cluster. 另外,如果它与群集上的实际路径匹配,则值得打印出“ matchResult”并查看完整路径。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 org.apache.beam.sdk.util.UserCodeException 使用 Samza Runner 执行 Beam Pipeline - org.apache.beam.sdk.util.UserCodeException while executing Beam Pipeline using the Samza Runner 使用 Apache Beam Java SDK 读取 Parquet 文件而不提供架构 - Read Parquet file using Apache Beam Java SDK without providing schema Java:使用 apache 光束管道读取存储在存储桶中的 excel 文件 - Java: read excel file stored in a bucket using apache beam pipeline 有没有办法使用Apache Beam管道进行动态查询或执行多个查询? - Is there a way to have a dynamic query or execute multiple queries with an apache beam pipeline? Apache 光束管道摄取“大”输入文件(超过 1GB)不会创建任何输出文件 - apache beam pipeline ingesting “Big” input file (more than 1GB) doesn't create any output file 在 GCP 数据流使用 apache beam 完成工作后,有什么方法可以进行处理吗? - Is there any way to do processing after GCP dataflow has completed the job using apache beam? Apache Beam-跳过管道步骤 - Apache Beam - skip pipeline step Apache Beam - 在管道中添加延迟 - Apache Beam - adding a delay into a pipeline 如何从 apache 光束 java sdk 中的 minIO 读取文件 - How to read a file from minIO in apache beam java sdk 使用 XmlIo 读取 apache 光束中的 xml 文件 - Reading an xml file in apache beam using XmlIo
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM