How to Run Cypress Test Case Parallelly in Cypress Cloud and CI/CD GitHub actions
Hello My Friend!!
In today’s world where time equals money, it is essential to lower the execution time for various actions. When doing testing, the same holds true. One of the most popular methods for running more test cases in less time is to execute tests concurrently
In this blog we are going to cover:
- How we can set up /Integration with Cypress Cloud
- Parallel Cypress Test case execution In CI/CD GitHub Action
- In the end, you will see the comparison report in time taken by test cases in execution in Cypress Cloud with different sets of machines.
Let’s walk through Parallel Test Execution with Cypress first before learning how to run tests in parallel using Cypress.
For demo purposes, I am using Cypress Version 12.1.0
What is Parallel Test Execution?
In parallel test execution, Several test cases are executed concurrently rather than sequentially. Parallel test execution has gained popularity as a result of the growing demand for cross-browser testing in the current industry. Testers can execute test cases simultaneously across numerous browsers thanks to it. As a result, the testers will save a lot of time and effort.
Parallel Test execution with Cypress
Cypress makes it simple to run tests concurrently by running your test cases on CI Tools. On the local device, the tester can also run the test cases concurrently. However, it is not advised because it will use up more resources and slow the CPU. Running it while configuring parallel execution on the CI tools is always advantageous.
What is Cypress Cloud?
Cypress Cloud is an enterprise-ready, web-based companion to the Cypress app. Using Cypress cloud we can online access the recorded test results, orchestrates test runs across multiple machines, provides rich analytics and diagnostics, and integrates those insights with your favorite tools.
Benefits of Cypress Cloud
Analyze and diagnose
Store the full history of your test results, with video clips, screenshots, and full stack traces. On the Latest Runs page, you can quickly examine the current condition of your program. Rich Analytics helps you spot troubling trends, and Flaky Test Management helps you discover unreliable tests. Associate related tests with grouping to see results broken down by browser and OS.
Run tests in parallel, in priority order, or not at all
With Smart Orchestration features, you can run tests across a swarm of machines simultaneously while Cypress Cloud coordinates runners and balances test loads — no setup required! You can prioritize recently failed specs to surface problems earlier and cancel whole test runs on failure to save on resource usage. You can also cancel in-progress runs manually from Cypress Cloud if you need to.
Integrate with source control providers
Keep failing code out of your GitHub, GitLab, and Bitbucket repositories with status checks that prevent commits from being merged until your Cypress tests are successful to ensure rock-solid reliability. Surface test results directly in your PRs with pull request comments that include test run statistics, specific failure details, and deep links to results in Cypress Cloud for fast debugging.
Collaborate and organize
Cypress can integrate with Slack and Jira, two of the most widely used collaboration systems. Deliver test results along with helpful supporting information straight to a specialized Slack channel. You may also immediately create bidirectionally linked Jira tickets from specific test failures if you have the Team, Business, or Enterprise plans.
Before Setup the Cypress Cloud we need to create Cypress Project that we can link with Cypress Cloud
Integration with Cypress Cloud
Before doing the setup of Cypress Cloud let’s create Cypress Project first
Create Cypress Project
Pre-request
Cypress latest version is installed using the command “npm install cypress” to install the latest version. GitHub Repo Link for the Code
- Create a Cypress Project let’s say “cypress_github_Action”
- Create two .spec files to run the Test cases in Cypress Cloud
The Package.json file is attached below
Set up Cypress Cloud For Parallel Test Execution
Step 1
Open the URL https://cloud.cypress.io/login
Step 2
Log in with any of the above options
Step 3
In My case I am log-in with GitHub and Project CypressV12_GitHub is already created
Step 4
Once you set up your project to record, generated unique projectId for your project and automatically insert it into your Cypress configuration file.
Or We Can Copy/Paste Project Id from the below screen as well.
Once all the above steps are done integration with Cypress Cloud Is Finished
What are GitHub Actions
GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository or deploy merged pull requests to production.
GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository. For example, you can run a workflow to automatically add the appropriate labels whenever someone creates a new issue in your repository. GitHub provides Linux, Windows, and macOS virtual machines to run your workflows, or you can host your own self-hosted runners in your own data center or cloud infrastructure.
Set Up CI/CD GitHub Action For Parallel Test Execution
Step 1
Create a repository in my case let’s say “cypress_github_Action”.Initially, there is no code in created repo
Step 2
Push the code to the newly created repo. Below are the test cases that we are using to run in GitHub Action and Cypress Cloud. We have created two .spec qaautomationlabs_Flow1.cy.js and qaautomationlabs_Flow2.cy.js.
Code attached Belowqaautomationlabs_Flow1.cy.js :- /// <reference types=”cypress” /> describe(“UI QAAutomationLabs.com”, { testIsolation: false }, () => { it(“Open URL”, () => { cy.visit(“https://qaautomationlabs.com/”); }); it(“Click on Read More “, () => { cy.get(“.staticslider-button”).click(); }); it(“Verify Particular Blog “, () => { cy.contains( “Running End-to-End Cypress Test cases In Google Cloud Build Pipeline” ); }); it(“Click on Blogs”, () => { cy.contains(“Blog”).scrollIntoView().click({ force: true }); }); it(“Search the datas”, () => { cy.get(‘[id=”wp-block-search__input-2″]’).scrollIntoView(); cy.get(‘[id=”wp-block-search__input-2″]’) .click({ force: true }) .type(“cypress”); cy.get(‘[id=”search-icon”]’).click({ force: true }); cy.contains(“Search Results for: cypress”); }); }); — – – – – – – – – – – – – – – – – – – – — – – – — – – – – – – – – – – qaautomationlabs_Flow2.cy.js :- describe(“API QAAutomationLabs.com”, { testIsolation: false }, () => { it(“GET API testing Using Cypress API Plugin”, () => { cy.request(“GET”, “https://reqres.in/api/users?page=2”).should( (response) => { expect(response.status).to.eq(200); } ); }); it(“POST API testing Using Cypress API Plugin”, () => { cy.request(“POST”, “https://reqres.in/api/users”, { name: “morpheus”, job: “leader”, }).should((response) => { expect(response.status).to.eq(201); }); }); it(“PUT API testing Using Flip Plugin”, () => { cy.request(“PUT”, “https://reqres.in/api/users/2”, { name: “morpheus”, job: “zion resident”, }).should((response) => { expect(response.status).to.eq(200); }); }); it(“DELETE API testing Using Cypress API Plugin”, () => { cy.request(“DELETE”, “https://reqres.in/api/users/2”).should((response) => { expect(response.status).to.eq(204); }); }); });
Step 3
After pushing the code click on the “Actions” Tab on GitHub
Step 4
After clicking on Actions Tab go to the bottom and click on Configure under Automation -> Manual workflow Section (Highlighted below)
What is Workflow
A workflow is a configurable automated process that will run one or more jobs. Workflows are defined by a YAML file checked into your repository and will run when triggered by an event in your repository, or they can be triggered manually, or at a defined schedule. Workflows are defined in the .github/workflows directory in a repository,
Step 5
Under project/.github/workflow/manual.yml created.
Update the .yml file for parallel execution of Test cases in GitHub Action
In the below .yml, we can see that we have passed record:true and parallel:true
So as we push the code. Test case start execute parallelly in two place
- In CI/CD GitHub action with provided container
- Also, the Test case start executing in parallel in TWO machines in Cypress Cloud
Both the test case start parallelly in GitHub Action and Cypress Cloudname: QAAutomationLabs.com Parallel Cypress Tests
on: [push]
jobs:
test:
name: Cypress Cloud Tests
runs-on: ubuntu-latest
container: cypress/browsers:node18.12.0-chrome107
strategy:
# when one test fails, DO NOT cancel the other
# containers, because this will kill Cypress processes
# leaving Cypress Cloud hanging …
# https://github.com/cypress-io/github-action/issues/48
fail-fast: false
matrix:
# run 2 copies of the current job in parallel
containers: [1, 2]
steps:
– name: Checkout
uses: actions/checkout@v3
# because of “record” and “parallel” parameters
# these containers will load balance all found tests among themselves
– name: QAAutomationLabs.com Tests
uses: cypress-io/github-action@v4
with:
record: true
parallel: true
group: ‘QAAutomationLabs’
env:
# pass the Cypress Cloud record key as an environment variable
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
# pass the GitHub token lets this action correctly
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
One Improtant setting that we have to do for to set the environment variable “CYPRESS_RECORD_KEY”
Open the particular repository in GitHub, In my case its cypress_github_Action
- Click on Setting
3. Click on Secrets
4. We have to Add the “Repository secrets” by entering Name (e.g CYPRESS_RECORD_KEY)
Enter Secret Key “XXXXXXXXXXX” which can be copied from Cypress Cloud Setting screenshot attached below
Test Case Execution In GitHub and Cypress Cloud
As we push the code test cases to start executing automatically in both palace
GitHub
In GitHub as the code is pushed both test cases start executing parallelly at the same time. We can see in below screenshot Under Jobs
Checking the history of both jobs, we can see that qaautomationlabs_Flow1.cy.js has been executed on Job 1: Cypress Cloud Tests(1)
The Second Test case qaautomationlabs_Flow2.cy.js has been executed on Job 2: Cypress Cloud Tests(2)
Cypress Cloud
In Cypress Cloud, we can see that both test cases are executed successfully. In the below screenshot, we can see the test results
Both the test cases are passed in the different machines with running in parallel.
The first test case ran in Machine 1 machine and passed successfully
The second test case ran in Machine 2 and passed successfully
Comparison of Time Taken by 5 Specs In Different Machine Sets in Cypress Cloud
Below is the comparison when we execute the 5 .specs file with 24 test cases (it block) by executing in different machine size
1. When executing 5 .specs (24 test cases) In 1 Machine :
In the below screenshot, you can see 5 Spec with 1 Machine with 24 Tc time taken ~1:49S
2. When executing 5 .specs (24 test cases) In 2 Machine :
In the below screenshot, you can see out of 5 Spec (3 Spec run in Machine 1 and 2 Spec in Machine 2) with 24 Tc. time taken ~0.55s.
3. When executing 5 .specs (24 test cases) In 3 Machine :
In the below screenshot, you can see out of 5 Spec (3 Spec run in Machine 1,2 Spec in Machine 2, and 1 Spec in Machine 3) with 24 Tc. time taken ~0.42s.
4. When executing 5 .specs (24 test cases) In 4 Machine :
In the below screenshot, you can see out of 5 Spec (2 Spec run in Machine 1, 1 Spec in Machine 2, 1 Spec in Machine 3, and 1 Spec in Machine 4) with 24 Tc. time taken ~0.30s.
5. When executing 5 .specs (24 test cases) In 5 Machine :
In the below screenshot, you can see out of 5 Spec (1 Spec run in Machine 1 and 1 Spec in Machine 2, 1 Spec in Machine 3,1 Spec in Machine 4, and 1 Spec in Machine 5) with 24 Tc take ~0.27s.
In the below table, we can see as the number of Machine increases for parallel test case execution, the time of execution is reduced
Conclusion
In addition to integrating with GitHub Actions, Cypress also works with GitLab, Bitbuckets, and CircleCI, Cypress also offers parallelization and continuous testing.
Cypress’s use of parallel execution is highly advantageous because it speeds up the continuous testing process. It helps execute a lot of test cases in a short amount of time.