Java中解析日志文件所需的匹配模式

Question

As I want to parse the log file using matching pattern to satisfy the following conditions I am poor in java not sure to correct. Please help on this.

    total count of report requests with size >=100kB
    total count of report requests with size <100kB
    total count of all non report requests
    total response time of report requests with size >=100kB
    total response time of report requests with size <100kB
    total response time of all non report requests
    total size of report requests with size >=100kB
    total size of report requests with size <100kB
    total size of all non report requests

Log file is the format

 100.10.200.20 - - [19/Apr/2016:09:21:45 -0400] "GET /test/report/RenderForm.do?formId=18483&formType=CCT_APP_AP_TR_RSP HTTP/1.1" 200 418032  

Written code in Java but sure my condition and pattern is not correct and My doubt is how to get the response time and to correct the below java code

total response time of report requests with size >=200kB
      total response time of report requests with size <200kB
    total response time of all non report requests

Any body help me to correct this code and satisfy the above all following conditions please.

Thanks in Advance.

Code:

 import java.io.BufferedReader;
  import java.io.FileReader;
  import java.io.IOException;
  import java.util.regex.Matcher;
  import java.util.regex.Pattern;

  import org.apache.log4j.Logger;



  public class Test3 {

    static Logger logger = Logger.getLogger(Test3.class);

    public static final int RETURN_CODE_SUCCESS = 0;

    public static final int RETURN_CODE_ERROR = 1;


    public static void main(String[] args) {

        BufferedReader br = null;

        try {

            String sCurrentLine;

            int nSize = 0;
            float nSizeKB = 0;

            int count = 0;
            int countNON=0, countlt100 = 0;

            String[] arLine = new String[0];
            br = new BufferedReader(new FileReader("C:\\testing.txt"));

            float gt100ReportGen=0;
            float lt100ReportGen=0;
            float gt100ResTime=0;
            float lt100ResTime=0;
            float nonReportGenSize=0;
            float nonReportResTime=0;

            while ((sCurrentLine = br.readLine()) != null) {
                Boolean bCondition1 = false;
                Boolean bCondition2 = false;
                Boolean bcondition3 = false;

                // Check the line contains the date and time

                //String pattern = ".*\\s(\\d*)";

                //Pattern r = Pattern.compile(pattern);

              //  Matcher m = r.matcher(sCurrentLine);

                Pattern p = Pattern.compile("\\d\\d\\d\\d-\\d\\d-\\d\\d@\\d\\d:\\d\\d:\\d\\d");

                Matcher m = p.matcher(sCurrentLine);

                arLine = sCurrentLine.split(" ");
                nSize = arLine.length-1;
                nSizeKB = nSize/1024;

                 if (sCurrentLine.contains("report")){

                     count++;

                    if (nSizeKB < 100) {

                        //total size of ReportGen requests with size <100kB
                        lt100ReportGen=lt100ReportGen+nSizeKB;

                        bCondition1 = true;

                        //total count of ReportGen requests with size <100kB
                        countlt100++;
                    }
                    if (nSizeKB >= 100) {

                        // total size of ReportGen requests with size >=100kB
                        gt100ReportGen=gt100ReportGen+nSizeKB;
                        bCondition2 = true;
                    }
                 }  else {
                     //total count of all non ReportGen requests
                     countNON++;
                     //total size of all non ReportGen requests
                     nonReportGenSize=nonReportGenSize+nSizeKB;
                     bcondition3=true;
                 }

                 if (m.find()) {

                     if (bCondition1) {
                         //total response time of ReportGen requests with size <100kB
                         lt100ResTime=lt100ResTime+nSizeKB;
                     }
                     if (bCondition2) {
                         //total response time of ReportGen requests with size >=100kB
                         gt100ResTime=gt100ResTime+nSizeKB;
                     }
                     if (bcondition3){
                         //total response time of all non ReportGen requests
                         nonReportResTime=nonReportResTime+nSizeKB;
                     }
                 }
            }

            logger.info("total count of ReportGen requests with size <100kB: "+countlt100);
            logger.info("# of Non-ReportGEN Requests: "+countNON);
            logger.info("total response time of ReportGen requests with size >=100kB: "+gt100ReportGen);
            logger.info("total response time of ReportGen requests with size <100kB: "+lt100ReportGen);
            logger.info("total response time of all non ReportGen requests: "+nonReportResTime);
            logger.info("total size of ReportGen requests with size <100kB: "+lt100ResTime);
            logger.info("total size of ReportGen requests with size >=100kB: "+gt100ResTime);
            logger.info("total size of all non ReportGen requests: "+nonReportGenSize);


            StringBuilder stringBuilder = new StringBuilder();

            stringBuilder.append("total count of ReportGen requests with size <100kB: "+ countlt100);
            stringBuilder.append(", # of Non-ReportGEN Requests: "+countNON);
            stringBuilder.append(", total response time of ReportGen requests with size >=100kB: "+gt100ReportGen);
            stringBuilder.append(", total response time of ReportGen requests with size <100kB: "+lt100ReportGen);
            stringBuilder.append(",total response time of all non ReportGen requests: "+nonReportResTime);
            stringBuilder.append(",total size of ReportGen requests with size <100kB: "+lt100ResTime);
            stringBuilder.append(",total size of ReportGen requests with size >=100kB: "+gt100ResTime);
            stringBuilder.append(",total size of all non ReportGen requests: "+nonReportGenSize);


            logger.info(stringBuilder.toString());

            System.exit(RETURN_CODE_SUCCESS);


        } catch (IOException e) {
            e.printStackTrace();

            System.exit(RETURN_CODE_ERROR);

        } finally {
            try {
                if (br != null)br.close();
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }



    }
  }

Answer 1

You can get count and size results without regex as follows:

            List<String> log = Arrays.asList(
            "100.10.200.20 - - [19/Apr/2016:09:21:45 -0400] \"GET /test/report/RenderForm.do?formId=18483&formType=CCT_APP_AP_TR_RSP HTTP/1.1\" 200 418032",
            "100.10.200.20 - - [19/Apr/2016:09:21:45 -0400] \"GET /test/nonreport/RenderForm.do?formId=18483&formType=CCT_APP_AP_TR_RSP HTTP/1.1\" 200 418032",
            "100.10.200.20 - - [19/Apr/2016:09:21:45 -0400] \"GET /test/report/RenderForm.do?formId=18483&formType=CCT_APP_AP_TR_RSP HTTP/1.1\" 200 18032",
            "100.10.200.20 - - [19/Apr/2016:09:21:45 -0400] \"GET /test/nonreport/RenderForm.do?formId=18483&formType=CCT_APP_AP_TR_RSP HTTP/1.1\" 200 18032"
            );

    int countRequestsGE100 = 0;
    int countRequestsL100 = 0;
    int countNonReportRequests = 0;
    int sizeRequestsGE100 = 0;
    int sizeRequestsL100 = 0;
    int sizeNonReportRequests = 0;
    int urlOffset = 6;
    int sizeOffset = 9;
    for(String s:log)
    {
        String[] array = s.split(" ");
        int size = Integer.valueOf(array[sizeOffset]).intValue();
        if (array[urlOffset].contains("/report/"))
        {
            if (Integer.valueOf(array[sizeOffset]).intValue() >= 100000)
            {
                countRequestsGE100++;
                sizeRequestsGE100 += size;
            }
            else
            {
                countRequestsL100++;
                sizeRequestsL100 += size;
            }
        }
        else
        {
            countNonReportRequests++;
            sizeNonReportRequests += size;
        }
    }

    System.out.print(
            "total count of report requests with size >=100kB: " + countRequestsGE100 + "\n" +
            "total count of report requests with size <100kB: " + countRequestsL100 + "\n" +
            "total count of all non report requests: " + countNonReportRequests + "\n" +
            "total size of report requests with size >=100kB: " + sizeRequestsGE100 + "\n" +
            "total size of report requests with size <100kB: " + sizeRequestsL100 + "\n" +
            "total size of all non report requests: " + sizeNonReportRequests + "\n"
            );

But I don't see a place in your line where to calculate total response time