黄瓜测试用例一起运行时会失败，但是当我单独运行它们时它们会通过

Question

I am using Spring with Cucumber (and IntelliJ and Gradle). My test case fails when run together and passes when run individually. (The fails 2 out of 3 times, sometimes it works...)

I also tried to quarantine the problematic Scenario or Senario combinations, but no luck... I tried introducing @After and @Before hooks to reset the value of of the account value... I even tried switching the position of senarios, nothing helps...

I really hope someone can help me with this problem

The features:

Feature: Cash Withdrawal
  Scenario: Successful withdrawal from an account in credit
    Given my account has been credited with $100.00
    When I withdraw $20
    Then $20 should be dispensed
    And the balance of my account should be $80.00
  Scenario: Unsuccessful withdrawal due to technical fault
    Given my account is in credit
    But the cash slot has developed a fault
    When I request some of my money
    Then I should see an out-of-order message
    And $0 should be dispensed
    And the balance of my account should be unchanged
  Scenario: Unsuccessful withdrawal due to insufficient ATM funds
    Given my account is in credit
    And the ATM contains $10
    When I withdraw $20
    Then I should see an ask-for-less-money message
    And $0 should be dispensed
    And the balance of my account should be unchanged

And my stepdefinitions:

public class AccountSteps {

    @Autowired
    Account account;

    private Money originalBalance;

    @Given("^my account has been credited with (\\$\\d+\\.\\d+)$")
    public void myAccountHasBeenCreditedWith$(
            @Transform(MoneyConverter.class) Money amount)
            throws Throwable {
        account.credit(amount);
    }

    @Given("^my account is in credit$")
    public void myAccountIsInCredit$() throws Throwable {
        originalBalance = new Money(30, 00);
        account.credit(originalBalance);
    }

    @Then("^the balance of my account should be unchanged$")
    public void theBalanceOfMyAccountShouldBeUnchanged() throws Throwable {

        checkBalanceIs(originalBalance);
    }

    @Then("^the balance of my account should be (\\$\\d+\\.\\d+)$")
    public void theBalanceOfMyAccountShouldBe$(
            @Transform(MoneyConverter.class) Money amount) throws Throwable {

        checkBalanceIs(amount);
    }

    private void checkBalanceIs(Money amount) throws Throwable {
        int timeoutMilliSecs = 3000;
        int pollIntervalMilliSecs = 100;

        while (!account.getBalance().equals(amount) && timeoutMilliSecs > 0) {
            Thread.sleep(pollIntervalMilliSecs);
            timeoutMilliSecs -= pollIntervalMilliSecs;
        }

        Assert.assertEquals(
                "Incorrect account balance -",
                amount,
                account.getBalance());
    }
}

public class CashSlotSteps {

    @Autowired
    TestCashSlot cashSlot;

    @Given("^\\$(\\d+) should be dispensed$")
    public void $ShouldBeDispensed(int dollars) throws Throwable {
        Assert.assertEquals("Incorrect amount dispensed -", dollars,
                cashSlot.getContents());
    }

    @Given("^the cash slot has developed a fault$")
    public void theCashSlotHasDevelopedAFault() throws Throwable {
        cashSlot.injectFault();
    }

    @Given("^the ATM contains \\$(\\d+)$")
    public void theATMContains$(int dollars) throws Throwable {
        cashSlot.load(dollars);
    }
}

public class TellerSteps {

    @Autowired
    private Account account;

    @Autowired
    private AtmUserInterface teller;

    @When("^I withdraw \\$(\\d+)$")
    public void iWithdraw$(int amount) throws Throwable {
        teller.withdrawFrom(account, amount);
    }

    @Given("^I request some of my money$")
    public void iRequestSomeOfMyMoney() {
        int dollarsRequested = 10;
        teller.withdrawFrom(account, dollarsRequested);
    }

    @Then("^I should see an out-of-order message$")
    public void iShouldSeeAnOutOfOrderMessage() throws Throwable {
        Assert.assertTrue(
                "Expected error message not displayed",
                teller.isDisplaying("Out of order"));
    }

    @Then("^I should see an ask-for-less-money message$")
    public void iShouldSeeAnAskForLessMoneyMessage() throws Throwable {
        Assert.assertTrue(
                "Expected error message not displayed",
                teller.isDisplaying("Insufficient ATM funds"));
    }
}

Answer 1

You are obviously mutating a variable which is shared between two tests which is not getting reset.

This is usually

1. A static mutable variable
2. A member variable on a singleton service

I'm not sure how you are managing your services but if using spring you might want to investigate @DirtiesContext for case 2. Eg:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes={...}) 
@DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
public class MyTest { ... }

Further info here

Answer 2

I have some hooks, for this example. To delete the account.

public class ResetHooks {
    @Before(order = 1)
    public void reset() {
        System.setProperty("webdriver.gecko.driver","C:\\...\\geckodriver.exe");
        if (!Base.hasConnection()) {
            Base.open(
                    "com.mysql.jdbc.Driver",
                    "jdbc:mysql://localhost/bank",
                    "user", "password");
        }

        Account.deleteAll();

        TransactionQueue.clear();
    }

}

-------------edit----------

My Cumcumber xml for spring:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
            http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
            http://www.springframework.org/schema/context
            http://www.springframework.org/schema/context/spring-context-3.0.xsd">

    <context:annotation-config/>

    <context:component-scan base-package="hooks, nicebank, support" />

    <bean class="support.AtmUserInterface" scope="cucumber-glue" />
    <bean class="support.TestCashSlot" scope="cucumber-glue" />

    <bean class="support.AccountFactory" factory-method="createTestAccount"
          lazy-init="true" scope="cucumber-glue" />

    <bean class="org.openqa.selenium.support.events.EventFiringWebDriver"
          scope="cucumber-glue" destroy-method="close">
        <constructor-arg>
            <bean class="org.openqa.selenium.firefox.FirefoxDriver"
                  scope="cucumber-glue"/>
        </constructor-arg>
    </bean>
</beans>