SonarQube 分析未显示代码覆盖率

Question

I have a Jenkins project to do SonarQube analysis of my NodeJS project. I added istanbul as a dependency to my project's package.json . In the Jenkins build configuration, first I run a shell script:

cd ./project-name
npm install
node_modules/.bin/istanbul cover ./node_modules/.bin/_mocha path-to-unit-tests
node_modules/.bin/istanbul report cobertura
cd ..

This installs the dependencies, runs the tests and generates a code coverage report and generates a cobertura-coverage.xml file.

After the shell script, I run a Invoke Standalone SonarQube Analysis with the following properties for code coverage:

sonar.java.coveragePlugin=cobertura
sonar.dynamicAnalysis=reuseReports
sonar.cobertura.reportPath=./project-name/coverage/cobertura-coverage.xml

The Jenkins job runs successfully with a SonarQube dashboard describing various things such as the lines of code, technical debt, issues and so on for the project. But the code coverage for the unit tests doesn't show up on the SonarQube dashboard. I made sure that the dashboard has the Unit Tests widget.

Version of SonarQube server: 5.2
Version of JavaScript plguin used on SonarQube: 2.9
Version of Cobertura plugin used on SonarQube: 1.6.3
Version of Cobertura plugin used on Jenkins: 1.9.7
Version of NodeJS plugin used on Jenkins: 0.2.1

I verified that the workspace does have the cobertura-coverage.xml file. Also checked the build console logs and found no bugs. I have also tried pushing the code coverage using LCOV format before:

sonar.dynamicAnalysis=reuseReports
sonar.javascript.lcov.reportPath=./project-name/coverage/lcov.info

The report doesn't get published to SonarQube even though the coverage report does get generated in Jenkins workspace. I looked at the workspace contents and verified. The console logs show the coverage report getting generated. Also tried

sonar.dynamicAnalysis=reuseReports
sonar.javascript.lcov.reportPath=project-name/coverage/lcov.info

and

sonar.dynamicAnalysis=reuseReports
sonar.cobertura.reportPath=project-name/coverage/cobertura-coverage.xml

to no avail. I also restarted the Jenkins and SonarQube servers 2 times each. Looked at many similar questions on StackOverflow and elsewhere but didn't find anything that works.

If I add a post-build action Publish Cobetura Coverage Report and specify the path to the cobetura-coverage.xml file in the Cobertura xml report pattern field, the code coverage report does get published in Jenkins.

Looked at the background task logs in SonarQube and saw an exception.

java.lang.IllegalStateException: Cannot persist sources of project-key:project-name/node_modules/jscs/jscs-browser.js
at     org.sonar.server.computation.step.PersistFileSourcesStep$FileSourceVisitor.visitFile(PersistFileSourcesStep.java:132) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.component.DepthTraversalTypeAwareCrawler.visitNode(DepthTraversalTypeAwareCrawler.java:72) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:44) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:91) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:47) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.component.DepthTraversalTypeAwareCrawler.visitChildren(DepthTraversalTypeAwareCrawler.java:91) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.component.DepthTraversalTypeAwareCrawler.visit(DepthTraversalTypeAwareCrawler.java:47) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.step.PersistFileSourcesStep.execute(PersistFileSourcesStep.java:89) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:39) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.taskprocessor.report.ReportTaskProcessor.process(ReportTaskProcessor.java:53) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.taskprocessor.CeWorkerRunnableImpl.executeTask(CeWorkerRunnableImpl.java:78) [sonar-server-5.2.jar:na]
at org.sonar.server.computation.taskprocessor.CeWorkerRunnableImpl.run(CeWorkerRunnableImpl.java:55) [sonar-server-5.2.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45-internal]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45-internal]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45-internal]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45-internal]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45-internal]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45-internal]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45-internal]
Caused by: org.apache.ibatis.exceptions.PersistenceException: 
### Error updating database.  Cause: com.mysql.jdbc.PacketTooBigException:     Packet for query is too large (7989143 > 4194304). You can change this value on the server by setting the max_allowed_packet' variable.
### The error may involve org.sonar.db.source.FileSourceMapper.insert-Inline
### The error occurred while setting parameters
### SQL: INSERT INTO file_sources (project_uuid, file_uuid, created_at, updated_at, binary_data, line_hashes, data_hash,     src_hash, data_type, revision)     VALUES (?, ?, ?,     ?, ?, ?,     ?, ?,?,     ?)
### Cause: com.mysql.jdbc.PacketTooBigException: Packet for query is too large (7989143 > 4194304). You can change this value on the server by setting the max_allowed_packet' variable.
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:154) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:141) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:51) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:52) ~[mybatis-3.2.7.jar:3.2.7]
at com.sun.proxy.$Proxy84.insert(Unknown Source) ~[na:na]
at org.sonar.db.source.FileSourceDao.insert(FileSourceDao.java:117) ~[sonar-db-5.2.jar:na]
at org.sonar.server.computation.step.PersistFileSourcesStep$FileSourceVisitor.persistSource(PersistFileSourcesStep.java:160) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.step.PersistFileSourcesStep$FileSourceVisitor.visitFile(PersistFileSourcesStep.java:130) ~[sonar-server-5.2.jar:na]
... 18 common frames omitted
Caused by: com.mysql.jdbc.PacketTooBigException: Packet for query is too large (7989143 > 4194304). You can change this value on the server by setting the max_allowed_packet' variable.
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3540) ~[mysql-connector-java-5.1.35.jar:5.1.35]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2417) ~[mysql-connector-java-5.1.35.jar:5.1.35]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582) ~[mysql-connector-java-5.1.35.jar:5.1.35]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2535) ~[mysql-connector-java-5.1.35.jar:5.1.35]
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1911) ~[mysql-connector-java-5.1.35.jar:5.1.35]
at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1203) ~[mysql-connector-java-5.1.35.jar:5.1.35]
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:172) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:44) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:69) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.executor.ReuseExecutor.doUpdate(ReuseExecutor.java:50) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:105) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:71) ~[mybatis-3.2.7.jar:3.2.7]
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:152) ~[mybatis-3.2.7.jar:3.2.7]
... 25 common frames omitted

So now I've updated the shell script code coverage generation line to

node_modules/.bin/istanbul cover -x node_modules ./node_modules/.bin/_mocha path-to-unit-tests

Still getting the exception. Based on max_allowed_packet, I don't have MySQL , I need to bump up the value of MAX_ALLOWED_PACKET in the database settings. Just done that and retriggered the Jenkins job for SonarQube analysis. The exception disappeared. The background task in SonarQube was successful. But I still don't see Unit test code coverage in the dashboard. There are no other exceptions. When I click on 'Configure Widgets' button, the Unit test widget has 'No data' label on it. When I go back to the dashboard, the Unit test widget disappears.

Any idea what I am missing?

Answer 1

The first time (~a year back) when I started exploring "Sonarqube" and while doing set up everything worked fine. I was able to see all the metrics along with " code coverage ".

For that used version was : sonarqube-6.7.5, sonar-scanner: sonar-scanner-3.2.0.1227-macosx

After that, when I tried today for some other project but I struggled almost ~3 hours to identify why code coverage is not being displayed in the sonarqube report though everything is configured properly.

After spending some time I have found below two blog topics which helps me to understand why I was not getting the coverage report.

If I read it correctly then it seems that there was a bug where after " lcov " report path was not recognized though it was configured accurately.

Reference links:

unable-to-get-code-coverage-for-javascript

Fix: js-coverage-doesnt-work-anymore

Used version: Docker image of sonarqube:8.3.0 & newtmitch/sonar-scanner as a sonar-scanner.

For reference, Dockerfile from my local setup which is used to run the sonar-scanner:

FROM newtmitch/sonar-scanner AS sonarqube_scan
COPY . /usr/src
RUN ls -list
RUN sonar-scanner \
    -Dsonar.host.url="http://localhost:9000" \
    -Dsonar.projectKey="test-node-app" \
    -Dsonar.projectVersion="1.0" \
    -Dsonar.sources="/usr/src" \
    -Dsonar.language="javascript" \
    -Dsonar.sourceEncoding="UTF-8" \
    -Dsonar.dynamicAnalysis="reuseReports" \
    -Dsonar.javascript.lcov.reportPaths="/usr/src/app/coverage/lcov.info"

I hope someone this helpful!

Happy learning!

Answer 2

The lcov report publishing worked after I had fixed the exception mentioned in the question. But sometimes, I randomly see another exception. Moreover, the Unit test widget on the SonarQube dashboard shows only the Conditionals coverage. Need to fine tune the solution more.

sonar.dynamicAnalysis=reuseReports
sonar.javascript.lcov.reportPath=project-name/coverage/lcov.info