Apache Lucene - S3 directory

Question

How to configure apache lucene data directory and index directory location as s3 location?

My java code will run on EC2 and apache lucene data directory and index directory location as s3 location.

Sample Java Code: from Local Diretory

String indexDir = "C:\\Lucene\\Index";
String dataDir  = "C:\\Lucene\\Data";

To S3

String indexDir = "https://s3.amazonaws.com/my-s3-demo/Lucene/Index";
String dataDir  = "https://s3.amazonaws.com/my-s3-demo/Lucene/Data";

highly appreciate any sample code. And my java code for local storage here

Answer 1

Sounds like you need to configure your dir path based on your target system. Depending on what you work with you're able to achieve this with property or configuration files.

This path should be used for directory initialization.

Directory indexDirectory = FSDirectory.open(...); // use path here

Answer 2

Have a look at the lucene-s3directory which I wrote. It writes Lucene indices to AWS S3 directly and does not need a local filesystem. It's pretty early stage so use with caution.

S3Directory dir = new S3Directory("my-lucene-index");
dir.create();
// use it in your code in place of FSDirectory, for example
dir.close();
dir.delete();