Need a way for tests to run sequentially in TestNG using xmlSuites

Question

I do not run my testcases through testng xml but use

TestListener listener=new TestListener();
    XmlSuite suite=new XmlSuite();
    suite.setName("Test Results");
    suite.setParallel(ParallelMode.NONE);
    suite.setThreadCount(Integer.parseInt(TestProperties.THREAD_COUNT.toString()));
    List<XmlSuite> suits=new ArrayList<XmlSuite>();
    suits.add(suite);


    List<XmlPackage> xpackage=new ArrayList<XmlPackage>();
    xpackage.add(new XmlPackage(TestProperties.TESTNG_PACKAGE.toString()));


    XmlTest test=new XmlTest(suite);
    test.setPackages(xpackage);
    //test.setParallel(ParallelMode.NONE);
    String groups=TestProperties.TESTNG_GROUP.toString();
    System.out.println("groups are:"+groups);
    String groupArray[]=groups.split(",");
    List<String> includedGroups=new ArrayList<String>();
    includedGroups.addAll(Arrays.asList(groupArray));
    test.setIncludedGroups(includedGroups);


    TestNG tng=new TestNG();
    tng.setOutputDirectory("test-output");
    tng.setXmlSuites(suits);
    //tng.addListener((ITestNGListener) listener);
    tng.run();
    System.exit(0)

I need my tests to run sequentially because of a lot of factors. I tried giving parallelMode.none, preserve order etc, but my testcases are being run in a weird way usually numerical. For eg after 200, my testcase order in code would be 211,212,213 etc and after that 201,202 etc. I need it run in that order. But right now, after 200, the testcase run would be 201. How can i make the testcase run with the order in which they are specified.

I also tried giving priority to the testcases 211,212 etc, but that too didnt work. I added method inteceptor but my line number always give 1.

public class PriorityInterceptor implements IMethodInterceptor {

public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {

    Comparator<IMethodInstance> comparator = new Comparator<IMethodInstance>() {
        private int getLineNo(IMethodInstance mi) {
        int result = 0;

        String methodName = mi.getMethod().getMethodName();
        String className  = mi.getMethod().getConstructorOrMethod().getDeclaringClass().getCanonicalName();
        ClassPool pool    = ClassPool.getDefault();

        try {
            CtClass cc        = pool.get(className);
            CtMethod ctMethod = cc.getDeclaredMethod(methodName);
            System.out.println(methodName);
            result            = ctMethod.getMethodInfo().getLineNumber(0);
            System.out.println("result:"+result+"and method:"+ctMethod);
        } catch (NotFoundException e) {
            e.printStackTrace();
        }

        return result;
        }

        public int compare(IMethodInstance m1, IMethodInstance m2) {
            System.out.println(getLineNo(m1) - getLineNo(m2));
        return getLineNo(m1) - getLineNo(m2);
        }
    };

    IMethodInstance[] array = methods.toArray(new IMethodInstance[methods.size()]);
    Arrays.sort(array, comparator);
    System.out.println("22222222222222222222222222222222222222222222222222222222");
    System.out.println(Arrays.asList(array));
    return Arrays.asList(array);
    }

Answer 1

There is something called preserve-order = true in testng XML file. You could use that one to preserve the execution order.

For programmatic execution of testng: Setting preserve order at suite level

    XmlSuite xmlsuite = new XmlSuite();
    xmlsuite.setPreserveOrder(true);

Setting preserve order at test level The test here means the test tag not the @Test annotation.

List<XmlTest> tests = xmlsuite.getTests();
        for (XmlTest test : tests) {
            test.setPreserveOrder(true);
            .....
}

If you want some order in the execution of @Test, you could always use priority and dependsOnMethods along with your @Test annotation. If you use dependsOnMethods as specified in the below example, then the method2 will only start after the method1 has successfully run. If method1 failed due to some reason method2 will be marked as skipped.

Example:

@Test(priority=1)
public void method1(){}

@Test(priority=2)
public void method2(){}

@Test
public void method1(){}

@Test(dependsOnMethods="method1")
public void method2(){}

For execution using testng XML file

<suite name="Test_Suite_Name" verbose="1" preserve-order="true">
    <test name="TEST_TESTCLASSA" preserve-order="true">
        <classes>
            <class name="testpackage.TestClassA"/>
        </classes>
    </test>
    <test name="TEST_OTHER_TESTS" preserve-order="true">
        <classes>
            <class name="testpackage.TestClassB"/>
            <class name ="testpackage.TestClassC"/>
            <class name="testpackage.TestClassD"/>
            <class name="testpackage.TestClassE"/>
            <class name="testpackage.TestClassF"/>
            <class name="testpackage.TestClassG"/>
        </classes>
    </test>
</suite>

An important note If you use the preserve-order flag, execution will happen in the order as they appear in the List in case of programmatic execution and in the order as they appear in the testng xml file.

One suggestion While using programmatic execution of testng, please do not hardcode classes in your code. You could always parse an existing testng XML file and tweak it as per your requirement during the runtime using programmatic testng then trigger the test execution.