Mongo Spark 连接器和 mongo 3.2，root 用户无法读取数据库

Question

I use the official mongo spark connector.

• my spark version is 2.0
• my mongo version is 3.2.x
• my spark mongo connector is 1.1.0

On my database i have one admin with root role, so he has all right.

i have created a config as follows :

     val readConfig = ReadConfig(Map("spark.mongodb.auth.uri" -> "mongodb://<userName>:<password>@<ip>:27017/admin",
"spark.mongodb.input.uri" -> "mongodb://<ip>:27017/MyDatabase.myCollection"))

but when i try to read some data i get an error "not authorized to execute command."

i don't understand why my root user is not authorized.

Answer 1

It's because "spark.mongodb.auth.uri" is not a configuration setting.

As the input uri doesn't have the authentication parameters the read is not authorised.

Try:

 val readConfig = ReadConfig(Map(
     "uri" -> "mongodb://<userName>:<password>@<ip>:27017/myDatabase.myCollection?authSource=admin"))

or:

 val readConfig = ReadConfig(Map(
     "uri" -> "mongodb://<userName>:<password>@<ip>:27017",  // uses the default db to auth against (admin)
     "database" -> "myDatabase",
     "collection" -> "myCollection"))

Answer 2

To avoid a full scan, you can do as below:

val rdd = MongoSpark.load(sc)

val aggregatedRdd = rdd.withPipeline(Seq(Document.parse("{ $match: { test : { $gt : 5 } } }")))
println(aggregatedRdd.count)
println(aggregatedRdd.first.toJson)

Answer 3

Find below implementation for Spark SQL Mongo Spark connector for admin DB having authentication:
1] Mongo uri:- mongodb://userid:pw@ip:port
2] sparkSessionCtx = SparkSession
  .builder()      
  .appName(appName)
  .config("spark.mongodb.input.uri", uri+ "/" + dbName + "." + collName+ "?authSource=admin")
  .config("spark.mongodb.output.uri", uri+ "/" + dbName + "." + collName+ "?authSource=admin")
  .getOrCreate()
3] val readConfiguration = ReadConfig(Map("database" -> dbName, "collection" -> collectionName, "readPreference.name" -> "primaryPreferred" ), Some(ReadConfig(sparkSessionCtx)))
mongodataframe = MongoSpark.load(sparkSessionCtx , readConfiguration)