尝试与 golang testify/suite 并行运行测试失败

Question

I have several tests using testify/suite package and I execute them in parallel as follows

type IntegrationSuite struct {
    suite.Suite
}

func TestIntegrationSuite(t *testing.T) {
    suite.Run(t, &IntegrationSuite{})
}

func (is *IntegrationSuite) TestSomething() {
    is.T().Log("\tIntegration Testing something")
    for i := range myTestTable {
        i := i
        is.T().Run("Testing "+myTestTable[i].scenarioName, func(_ *testing.T) {
            is.T().Parallel()
...

func (is *IntegrationSuite) TestSomethingElse() {
    is.T().Log("\tIntegration Testing something else")
    for i := range myOtherTestTable {
        i := i
        is.T().Run("Testing "+myOtherTestTable[i].scenarioName, func(_ *testing.T) {
            is.T().Parallel()
...
        })

However this panics with

panic: testing: t.Parallel called multiple times [recovered]
        panic: testing: t.Parallel called multiple times

How can one leverage parallelism with the specific package?

Answer 1

You're calling t.Parallel() on the wrong instance of testing.T .

You're spinning up multiple subtests using:

is.T().Run("Testing "+myOtherTestTable[i].scenarioName, func(_ *testing.T) {...}

But instead of marking the subtest as parallel, you're marking the outer parent test as parallel for every subtest you start:

is.T().Parallel() // marks outer test as parallel

Instead, bind the subtest func's testing.T param to a variable and call Parallel on the subtest:

        is.T().Run("Testing "+myOtherTestTable[i].scenarioName, func(t *testing.T) {
            t.Parallel() // this is the subtest's T instance
            // test code
        })

This marks all subtests as parallel within the parent test. If you also want to mark the parent test as parallel with other tests in the package, call is.T().Parallel() once at the beginning of TestSomethingElse (not within the subtest)

For more details on parallelism of subtests, see: Control of Parallelism

Answer 2

Please note that the testify suite is currently not "thread safe" and therefore does not support concurrency. There are cases in which tests pass successfully even though they should fail.

Please find the current status as well the reported issues here: https://github.com/stretchr/testify/issues/187