Can I load the output of a pig script storing the output in a Pig storage in Java ?
The last line of the pig script I am referring to is something like -
STORE D INTO '$output' USING PigStorage();
I want to import the relation in java for further processing and rendering in UI .
If yes, how can I do it?
Yes, you can do it. Use Embedded Pig concept , it provides PigServer which helps to run the pigscript in java program.
import java.io.IOException;
import org.apache.pig.PigServer;
public class WordCount {
public static void main(String[] args) {
PigServer pigServer = new PigServer();
try {
pigServer.registerJar("/mylocation/tokenize.jar");
runMyQuery(pigServer, "myinput.txt";
}
catch (IOException e) {
e.printStackTrace();
}
}
public static void runMyQuery(PigServer pigServer, String inputFile) throws IOException {
pigServer.registerQuery("A = load '" + inputFile + "' using TextLoader();");
pigServer.registerQuery("B = foreach A generate flatten(tokenize($0));");
pigServer.registerQuery("C = group B by $1;");
pigServer.registerQuery("D = foreach C generate flatten(group), COUNT(B.$0);");
pigServer.store("D", "myoutput");
}
}
Here are the more examples : http://www.programcreek.com/java-api-examples/index.php?api=org.apache.pig.PigServer
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.