从Pig连接到Mongodb副本集URI

Question

I'm trying to store data from Pig into a Mongodb database with replica set of two hosts:

STORE out INTO 'mongodb://user:pass@mongoserver1.com:27017,mongoserver2.com:27017/my_db.my_collection?replicaSet=set-1234
USING com.mongodb.hadoop.pig.MongoUpdateStorage('{"_id":"\$id"}', '{\$set:{"_id":"\$uid","data":"\$data"}}','id:chararray,data:chararray','','{upsert:true}');

But I'm getting the following exception:

pig script failed to validate: java.lang.IllegalArgumentException:
java.net.URISyntaxException: Relative path in absolute URI: mongoserver2.com:27017/my_db.my_collection%3FreplicaSet=set-1234 

Which seems to be related to a problem parsing correctly the MongoClientURI format.

Is there a way to connect to from Pig to a MongoDB multi-host replica set?

Answer 1

As long as you specify the replica set name in the MongoURI, the MongoDB Java Driver will automatically detect replica set failover and discover the primary. Even if the address of the (primary) machine was not listed in the URI connection string.

You can test this by specifying the secondary for example :

STORE out INTO     
  'mongodb://user:pass@mongoserver2.com:27017/my_db.my_collection?replicaSet=replicaSetName'
USING com.mongodb.hadoop.pig.MongoInsertStorage('', '');

The behaviour above is tested on: mongo-java-driver-3.2.2.jar , mongo-hadoop-pig-1.5.0.jar and mongo-hadoop-core-1.5.0.jar .

Note that in your example you had mistyped replicaSet in the URI, although it may be a typo while writing question here.