从tomcat日志中提取SQL查询

Question

I need to develop a JAVA application by following an object oriented design pattern for extracting sql queries from Tomcat logs. I have the log file. I need to extract sql queries from that file. I am new to this field. What are the basics i need to know to do this or give some good references.

As i cant share the whole log file , i am sharing some part of it:

(Dao.java:execute:73) SELECT ORDER_STATUS_TYPE_ID, NAME FROM ORD.ORDER_STATUS_TYPE [DEBUG] [RMI TCP Connection(5)-127.0.0.1 05 Feb 2018 15:00:10] (TraceInterceptor.java:writeToLog:26) []Leaving TypeDaoImpl.getOrderStatusType(): 1062ms: [com.xyz.abc.api.is.model.type.OrderStatusType@2ac9ecc3, com.xyz.abc.api.is.model.type.OrderStatusType@76777395, com.xyz.abc.api.is.model.type.OrderStatusType@6ad8ddb7, com.xyz.abc.api.is.model.type.OrderStatusType@6738868f, com.xyz.abc.api.is.model.type.OrderStatusType@272c15f,

Answer 1

I am posting an answer assuming you cannot edit the DAO.java . If you can edit DAO.java introduce some characters like ||<< to mark the starting and end of a printing query. That way you won't have make an exhaustive list of all the SQL commands.

public static void main(String[] args) throws IOException {
        long start = System.currentTimeMillis();
        FileInputStream inputStream = null;
        Scanner sc = null;
        try {
            inputStream = new FileInputStream("C:\\Users\\absin\\Desktop\\catalina.out");
            sc = new Scanner(inputStream, "UTF-8");
            while (sc.hasNextLine()) {
                String line = sc.nextLine();
                if (line.toLowerCase().contains("select ") || line.toLowerCase().contains("insert ")
                        || line.toLowerCase().contains("update ")) {
                    // DO something with the found log entry
                    System.out.println(line);
                }
            }
            // note that Scanner suppresses exceptions
            if (sc.ioException() != null) {
                throw sc.ioException();
            }
        } finally {
            if (inputStream != null) {
                inputStream.close();
            }
            if (sc != null) {
                sc.close();
            }
        }
        System.out.println("Time taken : " + (System.currentTimeMillis() - start) + " milliseconds");
    }

It prints a lot of log lines with query, and takes ~ 12 seconds on a 40 mb log file. You can also use regex, if you are familiar with them.

Answer 2

Your question suggests that you already have a log file which contains the SQL. Although there are much better and faster ways to parse this file using different operating system tools but since you are constrained with using Java application, you can try using regular expression to parse the log files and get the queries. Some help with code can be found here .