正则表达式在PIG中带有双引号

Question

I'm writing a pig script to process an access log from a sophos proxy.

Each line is like:

2015:01:13-00:00:01 AR-BADC-FAST-01 httpproxy[27983]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.20.7.210" dstip="10.24.2.7" user="" ad_domain="" statuscode="302" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0x9ac68d0" url="http://www.google.com" exceptions="av,auth,content,url,ssl,certcheck,certdate,mime,cache,fileextension" error="" authtime="0" dnstime="1" cattime="0" avscantime="0" fullreqtime="239428" device="0" auth="0"

So I managed to do it in Java with MapReduce, using the following regex: \\"([^\\"]*)\\" to get the values between the quotes and then process it. Now I want to do the same with pig, but I'm not able to apply the regex to the each of the lines.

I'm doing:

input = load './http.log' as (line : chararray);
splt = foreach input generate FLATTEN(REGEX_EXTRACT_ALL(line,'(\\"([^\\"]*)\\")'));
dump splt;

And the result of the dump is: ().

There is something that I'm missing with the use of REGEX_EXTRACT_ALL or I have to escape some characters of the regex in a different way?

Thanks!

Answer 1

I managed to extract the values with a different approach, because I just wanted some of the fields of the line.

In order to get the values I'm doing:

splt = FOREACH A GENERATE
    FLATTEN(REGEX_EXTRACT(line,'.*url="([^"]*)".*',1)) AS url,
    FLATTEN(REGEX_EXTRACT(line,'.*fullreqtime="([^"]*)".*',1)) AS duration,
    FLATTEN(REGEX_EXTRACT(line,'.*size="([^"]*)".*',1)) AS bytes;

And then I can continue with the rest of the script