Maven test not running my `@QuarkusTest` with custom test profile

Question

I noticed a @QuarkusTest test class with custom test profile is found but none of the test cases in that class is running when I run mvn test . They are not skipped, not ignored, but just not running. This test has a custom test profile like this:

    /**
     * Create a test profile to provide config values only for this test
     */
    public static class EventSenderTestProfile implements QuarkusTestProfile {
        public EventSenderTestProfile() {

        }
        /**
         * Returns additional config to be applied to the test. This
         * will override any existing config (including in application.properties),
         * however existing config will be merged with this (i.e. application.properties
         * config will still take effect, unless a specific config key has been overridden).
         */
        public Map<String, String> getConfigOverrides() {
            return Map.of(
                    "prop1", "dummy",
                    "prop2", "dummy"
            );
        }

        /**
         * Returns enabled alternatives.
         *
         * This has the same effect as setting the 'quarkus.arc.selected-alternatives' config key,
         * however it may be more convenient.
         */
        public Set<Class<?>> getEnabledAlternatives() {
            return Collections.emptySet();
        }

        /**
         * Allows the default config profile to be overridden. This basically just sets the quarkus.test.profile system
         * property before the test is run.
         *
         */
        public String getConfigProfile() {
            return this.getClass().getName();
        }
    }

I am using Java 11 and maven-surefire-plugin:3.0.0-M5 .

When running in Intellij IDEA individually they run and pass, but in Maven logs:

[INFO] Running com.company.dep.app.event.sender.EventSenderTest
2022-09-02 13:17:41,724 INFO  [org.jbo.threads] (main) JBoss Threads version 3.4.2.Final
2022-09-02 13:17:42,600 WARN  [io.qua.mic.dep.bin.mpm.MicroprofileMetricsProcessor] (build-10) This application uses the MP Metrics API. The micrometer extension currently provides a compatibility layer that supports the MP Metrics API, but metric names and recorded values will be different. Note that the MP Metrics compatibility layer will move to a different extension in the future.
2022-09-02 13:17:43,095 INFO  [io.qua.arc.pro.BeanProcessor] (build-34) Found unrecommended usage of private members (use package-private instead) in application beans:
    - @Inject field com.company.dep.app.event.sender.EventSenderTest#eventSenderRestApi,
    - @Inject field com.company.dep.app.event.sender.EventSenderTest#appConfiguration,
    - @Inject field com.company.dep.app.event.acknowledgement.token2.SecurityInterceptor#resourceInfo
    - and 2 more - please enable debug logging to see the full list
2022-09-02 13:17:43,276 WARN  [io.qua.dep.ste.ClassTransformingBuildStep] (build-11) Cannot transform io.smallrye.context.Jdk12CompletableFutureWrapper as its containing application archive could not be found.
2022-09-02 13:17:43,293 WARN  [io.qua.dep.ste.ClassTransformingBuildStep] (build-11) Cannot transform io.smallrye.context.Jdk12CompletionStageWrapper as its containing application archive could not be found.
2022-09-02 13:17:47,593 WARN  [com.company.dep.app.eve.ack.tok.SecurityToken2Handler] (main) 
---------------------------------------------------------------------------
--------                                                              --------
----     !!! WARNING - RUNNING WITH LOCALLY AUTO-GENERATED KEY !!!     ----
--------                                                           --------
---------------------------------------------------------------------------
2022-09-02 13:17:47,905 INFO  [org.apa.cam.qua.cor.CamelBootstrapRecorder] (main) Bootstrap runtime: org.apache.camel.quarkus.main.CamelMainRuntime
2022-09-02 13:17:48,005 INFO  [com.company.dep.app.con.appConfiguration] (main) Configuration mapping initialized: 
2022-09-02 13:17:48,005 INFO  [com.company.dep.app.con.appConfiguration] (main) .....(configuration mappings)
2022-09-02 13:17:48,117 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Routes startup (total:2 started:2)
2022-09-02 13:17:48,117 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started route1 (activemq://queue:events)
2022-09-02 13:17:48,117 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started route2 (direct://receive.ack.response)
2022-09-02 13:17:48,117 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.14.1 (camel-1) started in 141ms (build:0ms init:126ms start:15ms)
2022-09-02 13:17:48,133 INFO  [com.company.dep.bas.log.depLogger] (main) Application is up
2022-09-02 13:17:48,134 INFO  [com.company.dep.bas.log.depLogger] (main) Application is up
2022-09-02 13:17:48,247 INFO  [com.company.dep.bas.shu.run.ShutdownController] (main) Setting pre shutdown sleep time to 0 seconds.
2022-09-02 13:17:48,248 INFO  [io.quarkus] (main) Quarkus 2.7.5.Final on JVM started in 17.066s. Listening on: http://localhost:8081
2022-09-02 13:17:48,249 INFO  [io.quarkus] (main) Profile com.company.dep.app.event.sender.EventSenderTest$EventSenderTestProfile activated. 
2022-09-02 13:17:48,249 INFO  [io.quarkus] (main) Installed features: [camel-activemq, camel-bean, camel-core, camel-direct, camel-jms, cdi, config-yaml, hibernate-validator, dep-shutdown-controller, jacoco, logging-json, micrometer, reactive-routes, rest-client, rest-client-jsonb, resteasy-reactive, resteasy-reactive-jsonb, security, smallrye-context-propagation, smallrye-fault-tolerance, smallrye-health, smallrye-jwt, smallrye-openapi, swagger-ui, vertx]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.739 s - in com.company.dep.app.event.sender.EventSenderTest

Notice they are not ran, but they are not skipped either.

I have another @QuarkusTest with no custom test profile. That runs well.

It's a test profile defined as static class in the same test. If I move the inner class out to another file, tests now are found, but skipped:

[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 6.975 s - in com.company.dep.app.event.sender.EventSenderTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 6.975 s - in com.company.dep.app.event.sender.EventSenderTest

Does that mean I cannot use custom test profile in Quarkus test?

Not all tests are working; due to some config properties problem, some @QuarkusTest fail to start.

Answer 1

OK I think I found out why. Quarkus is not starting properly, as some @ConfigMapping error happens. But that error is not shown for this test, but only in other @ConfigMapping test running before it, and this test is not marked as error, but just ignored as not skipped, not executed, no success and no failure.

I see this issue: https://github.com/quarkusio/quarkus/issues/12319 , seems by design. I must say this is very misleading design.

Googling "Maven ignores my QuarkusTest" takes me to this related question which mentions about downgrade surefire plugin, so I turn on -X maven debug option and search surefire in the log. I noticed that surefire has some report saved in target/surefire-reports and opening them leads me to this line:

<testcase name="matchAndSendShouldReturnNullWhenNotMatching" classname="com.company.dep.app.event.sender.EventSenderTest" time="0">
    <skipped type="org.opentest4j.TestAbortedException">
        <![CDATA[ org.opentest4j.TestAbortedException: Boot failed ]]>
    </skipped>
...

Searching this exception leads me to the github issue. Another typical programmer day saved by Google.

Solution: fix all boot errors preventing Quarkus to start, and this test passes.