自动化测试 - 从 Katalon Recorder 脚本到 Protactor - 这是一个好的解决方案吗？

Question

I'm new in the Automation Testing world. I don't know exactly how to implement that for an application built in Angular, with Bambo, git (we are using also JIRA) and the best strategy and tools to use.

In my team, we created a lot of tests with Katalon Recorder for doing “automatic” testing from Katalon Recorder. Our next step is to execute automatically these tests from a server or for instance Bamboo. We didn't currently get a licence for using Katalon Studio.

I tried to export a test from Katalon Recorder in protactor (typescript):

import { browser, by, element, Key, logging, ExpectedConditions as EC } from 'protractor';

describe('Create Commitment Numberring - Grant', () => {

    beforeAll(async () => { });
    beforeEach(async () => { });

    it('should do something', async () => {
        await element(by.linkText("Implementing Measures")).click();
        await browser.sleep(1200);
        await element(by.linkText("Commitment Numbering")).click();
        await browser.sleep(1200);
        await element(by.xpath("//commitment-numbering-home/ux-panel/div[2]/div/div[2]/div/div/div[2]/ux-dropdown-button/ux-button/button")).click();
        await element(by.xpath("//div[starts-with(@id,'mat-menu-panel-')]/div/span[2]/span/ux-dropdown-button-item/div/button")).click();


    // WARNING: unsupported command assertAttribute. Object= {"command":"assertAttribute","target":"//*[@id=\"commitment-numbering-modal\"]/div[2]@style","value":"display: block;"}

        expect(await element(by.xpath("//*[@label='Grant']")).getText()).toContain(`Grant`);


    // WARNING: unsupported command storeEval. Object= {"command":"storeEval","target":"getComputedStyle(document.querySelector(\".ux-badge--accent\")).backgroundColor","value":"backgroundGrant"}



    // WARNING: unsupported command assertEval. Object= {"command":"assertEval","target":"\"${backgroundGrant}\" == \"rgb(255, 214, 23)\"","value":"true"}

        expect(await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[1]/uxmodalheader/ux-layout-horizontal/div/div[2]/uxlayouthorizontalcenter/span/b/em")).getText()).toContain(`New Grant`);
        expect(await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[1]/uxmodalheader/ux-layout-horizontal/div/div[3]/uxlayouthorizontalright/div/ux-button/button/span[2]")).getText()).toContain(`Check data`);


    // WARNING: unsupported command storeEval. Object= {"command":"storeEval","target":"getComputedStyle(document.querySelector(\"html body.modal-open app-root ux-layout-app-shell div#app-wrapper.sidebar--open.sidebar-state-close-with-icons.xxl div#main div#main-content commitment-numbering-home.ng-star-inserted commitment-numbering-modal.ng-star-inserted ux-modal#commitment-numbering-modal div#commitment-numbering-modal.eui-u-anim.ux-modal.eui-u-anim--slideInDown.eui-u-anim--fast.show div.cdk-drag.ux-modal__dialog.cdk-drag-disabled div.ux-modal__content div.cdk-drag-handle.ux-modal__header uxmodalheader.ng-star-inserted ux-layout-horizontal div.ux-layout-horizontal div.ux-layout-horizontal__right uxlayouthorizontalright div.flex-container.button-actions.ng-star-inserted ux-button.ng-star-inserted button.ux-button.ux-button--warning.ux-button--with-label\")).backgroundColor","value":"backgroundCheckDataButton"}



    // WARNING: unsupported command assertEval. Object= {"command":"assertEval","target":"\"${backgroundCheckDataButton}\" == \"rgb(242, 149, 39)\"","value":"true"}

        expect(await element(by.xpath("//*[@label='Status']/div/label/span")).getText()).toContain(`Status`);
        expect(await element(by.xpath("//*[@formcontrolname='statusId']/div")).getText()).toContain(`Active`);
        expect(await element(by.xpath("//*[@label='Unit']/div/label/span")).getText()).toContain(`Unit`);
        await element(by.xpath("//*[@formcontrolname='orgId']")).click();
        await element(by.xpath("//*[@formcontrolname='orgId']/select")).element(by.cssContainingText('option', 'ESTAT.A.5')).click();
        expect(await element(by.xpath("//*[@label='Commitment Reference']/div/label/span")).getText()).toContain(`Commitment Reference`);
        expect(await element(by.xpath("//*[@class='signature-year']")).getText()).toContain(`Signature Year`);
        expect(await element(by.xpath("//*[@class='sequence']")).getText()).toContain(`Sequence Number`);
        await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[2]/uxmodalbody/commitment-numbering-form/div[1]/ux-card/div/div[2]/form/div[3]/div/ux-form-group/div/div/div/div/div/ux-datepicker/div/div/ux-button/button")).click();
        await element(by.xpath("//mat-calendar[starts-with(@id,'mat-datepicker-')]/div/mat-multi-year-view/table/tbody/tr[2]/td[2]/div")).click();
        expect(await element(by.xpath("//*[@class='sequence-number']")).getText()).toContain(`XXXX`);
        expect(await element(by.xpath("//*[@label='Credit Type']/div/label/span")).getText()).toContain(`Credit Type`);


    // WARNING: unsupported command assertSelectOptions. Object= {"command":"assertSelectOptions","target":"//*[@formcontrolname='creditTypeId']/select","value":"Operational credits,Administrative credits"}

        await element(by.xpath("//*[@formcontrolname='creditTypeId']/select[1]")).click();
        await element(by.xpath("//*[@formcontrolname='creditTypeId']/select[1]")).element(by.cssContainingText('option', 'Administrative credits')).click();
        expect(await element(by.xpath("//*[@label='Commitment BIFI-ID']/div/label/span")).getText()).toContain(`Commitment BIFI-ID`);
        await element(by.xpath("//*[@formcontrolname='bifiId']/select[1]")).click();
        await element(by.xpath("//*[@formcontrolname='bifiId']/select[1]/")).element(by.cssContainingText('option', '1')).click();


    // WARNING: unsupported command storeValue. Object= {"command":"storeValue","target":"//*[@formcontrolname='bifiId']/select[1]/","value":"bifiIDValue"}

        expect(await element(by.xpath("//*[@label='Comment']/div/label/span")).getText()).toContain(`Comment`);
        await element(by.xpath("//*[@formcontrolname='comment']/span/textarea")).click();
        await element(by.xpath("//*[@formcontrolname='comment']/span/textarea")).sendKeys('Creation of a new FD Cnum - Grant - CSB - A.5 - Seb');
        await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[1]/uxmodalheader/ux-layout-horizontal/div/div[3]/uxlayouthorizontalright/div/ux-button/button")).click();
        expect(await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[1]/uxmodalheader/ux-layout-horizontal/div/div[3]/uxlayouthorizontalright/div/ux-button/button/span[2]")).getText()).toContain(`Edit`);


    // WARNING: unsupported command storeEval. Object= {"command":"storeEval","target":"getComputedStyle(document.querySelector(\".flex-container > ux-button:nth-child(1) > button:nth-child(1)\")).backgroundColor","value":"backgroundEditButton"}



    // WARNING: unsupported command assertEval. Object= {"command":"assertEval","target":"\"${backgroundEditButton}\" == \"rgb(0, 68, 148)\"","value":"true"}

        expect(await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[3]/uxmodalfooter/ux-layout-horizontal/div/div[3]/uxlayouthorizontalright/ux-button[1]/button/span[2]")).getText()).toContain(`Save`);


    // WARNING: unsupported command storeEval. Object= {"command":"storeEval","target":"getComputedStyle(document.querySelector(\"div.show:nth-child(2) > div:nth-child(1) > div:nth-child(1) > div:nth-child(3) > uxmodalfooter:nth-child(1) > ux-layout-horizontal:nth-child(1) > div:nth-child(1) > div:nth-child(3) > uxlayouthorizontalright:nth-child(1) > ux-button:nth-child(1) > button:nth-child(1)\")).backgroundColor","value":"backgroundSaveButton"}



    // WARNING: unsupported command assertEval. Object= {"command":"assertEval","target":"\"${backgroundSaveButton}\" == \"rgb(70, 122, 57)\"","value":"true"}

        expect(await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[3]/uxmodalfooter/ux-layout-horizontal/div/div[3]/uxlayouthorizontalright/ux-button[2]/button/span")).getText()).toContain(`Close`);


    // WARNING: unsupported command storeEval. Object= {"command":"storeEval","target":"getComputedStyle(document.querySelector(\"div.show:nth-child(2) > div:nth-child(1) > div:nth-child(1) > div:nth-child(3) > uxmodalfooter:nth-child(1) > ux-layout-horizontal:nth-child(1) > div:nth-child(1) > div:nth-child(3) > uxlayouthorizontalright:nth-child(1) > ux-button:nth-child(2) > button:nth-child(1)\")).backgroundColor","value":"backgroundCloseComm"}



    // WARNING: unsupported command assertEval. Object= {"command":"assertEval","target":"\"${backgroundCloseComm}\" == \"rgb(255, 255, 255)\"","value":"true"}

        expect(await element(by.xpath("//*[@label='Status']/div/label/span")).getText()).toContain(`Status`);
        expect(await element(by.xpath("//*[@formcontrolname='statusId']/div")).getText()).toContain(`Active`);
        expect(await element(by.xpath("//*[@label='Unit']/div/label/span")).getText()).toContain(`Unit`);
        expect(await element(by.xpath("//*[@formcontrolname='orgId']/div")).getText()).toContain(`ESTAT.A.5`);
        expect(await element(by.xpath("//*[@label='Commitment Reference']/div/label/span")).getText()).toContain(`Commitment Reference`);
        expect(await element(by.xpath("//*[@class='signature-year']")).getText()).toContain(`Signature Year`);
        expect(await element(by.xpath("//*[@formcontrolname='dtSignatureYear']/div/div/div/input")).getAttribute('value')).toContain(`2021`)
        expect(await element(by.xpath("//*[@class='sequence']")).getText()).toContain(`Sequence Number`);
        expect(await element(by.xpath("//*[@class='sequence-number']")).getText()).toContain(`XXXX`);
        expect(await element(by.xpath("//*[@label='Credit Type']/div/label/span")).getText()).toContain(`Credit Type`);
        expect(await element(by.xpath("//*[@formcontrolname='creditTypeId']/div")).getText()).toContain(`Administrative credits`);
        expect(await element(by.xpath("//*[@label='Commitment BIFI-ID']/div/label/span")).getText()).toContain(`Commitment BIFI-ID`);
        expect(await element(by.xpath("//*[@formcontrolname='bifiId']/div")).getText()).toContain(`${bifiIDValue}`);
        expect(await element(by.xpath("//*[@label='Comment']/div/label/span")).getText()).toContain(`Comment`);
        expect(await element(by.xpath("//*[@formcontrolname='comment']/div/div")).getText()).toContain(`Creation of a new FD Cnum - Grant - CSB - A.5 - Seb`);
        await element(by.xpath("//commitment-numbering-home/commitment-numbering-modal/ux-modal/div[2]/div/div/div[3]/uxmodalfooter/ux-layout-horizontal/div/div[3]/uxlayouthorizontalright/ux-button[1]/button")).click();


    // WARNING: unsupported command assertAttribute. Object= {"command":"assertAttribute","target":"//*[@id=\"commitment-numbering-modal\"]/div[2]@style","value":"display: none;"}



    // WARNING: unsupported command assertElementPresent. Object= {"command":"assertElementPresent","target":"//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div","value":""}



    // WARNING: unsupported command assertElementPresent. Object= {"command":"assertElementPresent","target":"//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div","value":""}



    // WARNING: unsupported command storeAttribute. Object= {"command":"storeAttribute","target":"//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div@class","value":"validationAlertBox"}



    // WARNING: unsupported command assertEval. Object= {"command":"assertEval","target":"storedVars[\"validationAlertBox\"].includes(\"success\")","value":"true"}



    // WARNING: unsupported command assertElementPresent. Object= {"command":"assertElementPresent","target":"//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div/div/div","value":""}

        expect(await element(by.xpath("//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div/div/div/span")).getText()).toContain(`Create Success`);


    // WARNING: unsupported command assertElementPresent. Object= {"command":"assertElementPresent","target":"//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div/div/div/p","value":""}

        expect(await element(by.xpath("//html/body/app-root/ux-layout-app-shell/div/ux-growl/div/div/div/div/p")).getText()).toContain(`Commitment Numbering created successfully`);
        await browser.sleep(3000);


    // WARNING: unsupported command assertElementNotPresent. Object= {"command":"assertElementNotPresent","target":"//html/body/app-root/ux-layout-app-shell/div/div[2]/div","value":""}

    });

    afterEach(async () => {
        // Assert that there are no errors emitted from the browser
        const logs = await browser.manage().logs().get(logging.Type.BROWSER);
        expect(logs).not.toContain(jasmine.objectContaining({
            level: logging.Level.SEVERE,
        } as logging.Entry));
    });

});

1 - As you can see when exporting in this format, a lot of warnings appear. In another format there are less warning. I'm afraid when choosing this export format, that the tests will not correctly work, what do you think about that ?

2 - Is it the best format for exporting the tests suits in order to use it for Automation Testing?

3 - Could you please explain me how to implement the automation testing, which are the best strategy and tools to use?

Thanks in advance for your help

Answer 1

when you read through my answer, keep in mind that I'm not a fan of record n play tools in general

Before I even give you feedback about your questions, I must mention that protractor is being deprecated soon (look up for more info) and thus it shouldn't be chosen for long term projects

Now back to your questions

1. this format may work, but will be super brittle and flaky
2. the problem here, you only have 2 options - export existing tests or write them from scratch yourself. You pick
3. For the second part of this question no right answer here, only what's right for you. I'm completing my research about the next tool, my favourite picks as of 2021 in descending order are - playwright, webdriverIO, cypress. The first part is very, very long topic. But what I learnt from my mistakes

• test your own code as you write it, almost each line, do not assume it works
• think ahead of time what you will need in future. There things that will affect the foundation of your framework
• don't duplicate code, it will be easier to maintain it afterwards

But I have a question in turn too, why did the team decide to convert Katalon tests into actual scripts? What was wrong with Katalon test suites? Please comment below