[英]spark-shell error: value builder is not a member of object com.amazonaws.services.s3.model.PutObjectRequest
I'm just getting started with EMR Hadoop/spark etc., I am trying to use spark-shell to run a scala code to upload a file to EMRFS S3 location however I am receiving below error -我刚刚开始使用 EMR Hadoop/spark 等,我正在尝试使用 spark-shell 运行 scala 代码以将文件上传到 EMRFS S3 位置但是我收到以下错误 -
Without any Import If I run =>没有任何导入如果我运行 =>
val bucketName = "bucket"
val outputPath = "test.txt"
scala> val putRequest = PutObjectRequest.builder.bucket(bucketName).key(outputPath).build()
<console>:27: error: not found: value PutObjectRequest
val putRequest = PutObjectRequest.builder.bucket(bucketName).key(outputPath).build()
^
Once I add the Import package for PutObjectRequest I still get a different error.为 PutObjectRequest 添加 Import package 后,我仍然会收到不同的错误。
scala> import com.amazonaws.services.s3.model.PutObjectRequest
import com.amazonaws.services.s3.model.PutObjectRequest导入 com.amazonaws.services.s3.model.PutObjectRequest
scala> val putRequest = PutObjectRequest.builder.bucket(bucketName).key(outputPath).build()
<console>:28: error: value builder is not a member of object com.amazonaws.services.s3.model.PutObjectRequest
val putRequest = PutObjectRequest.builder.bucket(bucketName).key(outputPath).build()
^
I'm not sure what I am missing.我不确定我错过了什么。 Any help would be appreciated!
任何帮助,将不胜感激!
Note: Spark version is 2.4.5注:Spark 版本为 2.4.5
Instead of using the builder create the object of PutObjectRequest via a suitable constructor.不使用构建器,而是通过合适的构造函数创建 PutObjectRequest 的 object。 Also, create a connection to S3 using AmazonS3ClientBuilder.
此外,使用 AmazonS3ClientBuilder 创建到 S3 的连接。
import com.amazonaws.regions.Regions
import com.amazonaws.services.s3.AmazonS3ClientBuilder
import com.amazonaws.services.s3.model.ObjectMetadata
import com.amazonaws.services.s3.model.PutObjectRequest
import java.io.File
val clientRegion = Regions.DEFAULT_REGION
val bucketName = "*** Bucket name ***"
val fileObjKeyName = "*** File object key name ***"
val fileName = "*** Path to file to upload ***"
val s3Client = AmazonS3ClientBuilder.standard.withRegion(clientRegion).build
// Upload a file as a new object with ContentType and title specified.
val request = new PutObjectRequest(bucketName, fileObjKeyName, new File(fileName))
val metadata = new ObjectMetadata()
metadata.setContentType("plain/text")
metadata.addUserMetadata("title", "someTitle")
request.setMetadata(metadata)
s3Client.putObject(request)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.