Java進程高CPU

Question

下面是我的代碼和線程轉儲。 我不知道為什么CPU會100％運行。 有人可以在這里幫忙嗎？ 下面的方法是可疑的，我只是想解析一個結果集，並使用Apache CSV（commons-csv-1.5.jar）將其寫入csv。 當我評論調用此方法的行時，cpu保持在3％。

public static void writeResultSetToFile(ResultSet resultSet, String fileName) {

    BufferedWriter writer = null;
    CSVPrinter csvPrinter = null;

    //If the file with the same filename already exist, a date stamp is appended to the end of the file.
    if(checkIfFileExist(fileName)) {
        LOGGER.info("FILE EXIST:"+fileName);
        String fileNamePostFix = new SimpleDateFormat(Constants.FORMAT_yyyyMMddHHmm).format(new Date());
        fileName=fileName.concat(Constants.UNDERSCORE).concat(fileNamePostFix);
        LOGGER.info("WRITING TO FILE: "+fileName);
    }

    try {
        ResultSetMetaData metadata = resultSet.getMetaData();
        writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(fileName),StandardCharsets.UTF_8));

        //Fetch the column header from the metadata and inserts to an arrayList
        columnCount = metadata.getColumnCount();
        List<String> headerList = new ArrayList<String>(0);
        for (int i = 1; i <= columnCount; i++) {
            headerList.add(metadata.getColumnName(i));
        }
        String[] headerArray = new String[headerList.size()];
        headerArray = headerList.toArray(headerArray);

        //Creates a csv printer with the column names fetched from the database 
        csvPrinter = new CSVPrinter(writer, CSVFormat.DEFAULT.withHeader(headerArray).withDelimiter(Constants.C_DELIMITER));

        recordCount = 0;
        List<String> valueList = new ArrayList<String>(0);
        while (resultSet.next()) {
            recordCount++;
            for (int i = 1; i <= columnCount; i++) {
                valueList.add(resultSet.getString(i));
            }
            csvPrinter.printRecord(valueList);
            valueList = new ArrayList<String>(0);
        }

    } catch (SQLException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    finally {
        if (writer != null) {
            try {
                writer.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        if (csvPrinter != null) {
            try {
                csvPrinter.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

螺紋轉儲

Full thread dump OpenJDK 64-Bit Server VM (24.171-b01 mixed mode):

"pool-2-thread-1" prio=10 tid=0x00007f2eb839d000 nid=0x264e runnable [0x00007f2ea6dc4000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:153)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        - locked <0x00000000e01bc220> (a java.io.BufferedInputStream)
        at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:346)
        at org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:423)
        at org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:405)
        at org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
        at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
        at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_FetchResults(TCLIService.java:515)
        at org.apache.hive.service.cli.thrift.TCLIService$Client.FetchResults(TCLIService.java:502)
        at com.cloudera.hive.hivecommon.api.HS2Client.fetchNRows(HS2Client.java:321)
        at com.cloudera.hive.hive.api.ExtendedHS2Client.fetchNRows(ExtendedHS2Client.java:499)
        at com.cloudera.hive.hivecommon.api.HS2Client.fetchRows(HS2Client.java:301)
        at com.cloudera.hive.hivecommon.dataengine.BackgroundFetcher.run(BackgroundFetcher.java:138)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
        at java.lang.Thread.run(Thread.java:748)

"Service Thread" daemon prio=10 tid=0x00007f2eb80b4800 nid=0x2562 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x00007f2eb80b2000 nid=0x2561 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x00007f2eb80af000 nid=0x2560 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00007f2eb80ad000 nid=0x255f waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00007f2eb807f800 nid=0x255e in Object.wait() [0x00007f2eb45f4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000e00247f0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
        - locked <0x00000000e00247f0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" daemon prio=10 tid=0x00007f2eb807d800 nid=0x255d in Object.wait() [0x00007f2eb46f5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000e000e1c8> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:503)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
        - locked <0x00000000e000e1c8> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x00007f2eb800b800 nid=0x2557 runnable [0x00007f2ec05fb000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.String.split(String.java:2289)
        at java.lang.String.split(String.java:2355)
        at com.cloudera.hive.hivecommon.dataengine.HiveJDBCQueryAnalyserUtils.queryAnalysis(HiveJDBCQueryAnalyserUtils.java:49)
        at com.cloudera.hive.hivecommon.api.HS2Buffer.getData(HS2Buffer.java:181)
        at com.cloudera.hive.hivecommon.api.HS2Client.getData(HS2Client.java:705)
        at com.cloudera.hive.hivecommon.dataengine.HiveJDBCResultSet.getData(HiveJDBCResultSet.java:265)
        at com.cloudera.hive.jdbc.common.SForwardResultSet.getData(SForwardResultSet.java:4590)
        at com.cloudera.hive.jdbc.common.SForwardResultSet.getString(SForwardResultSet.java:2138)
        at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.utils.FileUtils.writeResultSetToFile(FileUtils.java:153)
        at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.utils.DatabaseUtils.executeQueryAndWriteToFile(DatabaseUtils.java:135)
        at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.processor.JDBCProcessor.processCustomQueries(JDBCProcessor.java:84)
        at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.processor.JDBCProcessor.process(JDBCProcessor.java:47)
        at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.main.App.main(App.java:49)

"VM Thread" prio=10 tid=0x00007f2eb8077000 nid=0x255c runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f2eb8021000 nid=0x2558 runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f2eb8023000 nid=0x2559 runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f2eb8025000 nid=0x255a runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f2eb8027000 nid=0x255b runnable

"VM Periodic Task Thread" prio=10 tid=0x00007f2eb80bf800 nid=0x2563 waiting on condition

JNI global references: 234

Heap
 PSYoungGen      total 160768K, used 126191K [0x00000000f5500000, 0x00000000ffd80000, 0x0000000100000000)
  eden space 149504K, 84% used [0x00000000f5500000,0x00000000fd033ef8,0x00000000fe700000)
  from space 11264K, 0% used [0x00000000fe700000,0x00000000fe708000,0x00000000ff200000)
  to   space 10240K, 0% used [0x00000000ff380000,0x00000000ff380000,0x00000000ffd80000)
 ParOldGen       total 349184K, used 192583K [0x00000000e0000000, 0x00000000f5500000, 0x00000000f5500000)
  object space 349184K, 55% used [0x00000000e0000000,0x00000000ebc11df8,0x00000000f5500000)
 PSPermGen       total 21504K, used 12023K [0x00000000d5a00000, 0x00000000d6f00000, 0x00000000e0000000)
  object space 21504K, 55% used [0x00000000d5a00000,0x00000000d65bde38,0x00000000d6f00000)

Answer 1

好吧，這不是答案，而是一些需要空間的提醒：

1）您正在使用從零容量開始的ArrayList ...它必須經常調整其緩沖區的大小，因為arraylist的大小會像這樣增長： 0 -> 1 -> 2 -> 4 -> 7 -> 11 -> 17 。 所以代替List<String> valueList = new ArrayList<String>(0); 使用List<String> headerList = new ArrayList<String>(columnCount); 代替valueList = new ArrayList<String>(0); 使用valueList.clear();

2）您應該使用resource-try-catch，它更容易處理。

3）代碼-或更確切地說：在幕后-進行了大量解析...解析SQL結果，然后“解析”數據（必須以CSV兼容的方式進行轉義）。 不要小看這個！ 更重要的是，如果此CSVPrinter進行了其他一些格式設置，例如使文本文件看起來像表格一樣的額外間距。

4）另外，Apache庫不知道它們的速度，也不知道它們的資源效率！

5）手動沖洗（如注釋中所述）不利於性能！

6）從外觀writeResultSetToFile() ，您的方法writeResultSetToFile()是從循環中調用的。 我不知道，但是如果您告訴我們“ 3％的使用率”，這聽起來像是一項持續的任務，即循環。 因此，假設存在一個循環，該循環直接導致3％的CPU使用率，似乎每秒循環數千次。 如果現在-在每次調用中-檢索並存儲兩百萬行文本，這將減慢整個程序的速度。 相當。 也許您不應該如此頻繁地寫入該文件？ 也許一分鍾一次就足夠了嗎？ 但是，通常您可能會考慮使用與主循環並行運行的解耦線程。