What is Regression Testing?

Regression testing is a black box testing techniques. It is used to authenticate a code change in the software does not impact the existing functionality of the product. Regression testing is making sure that the product works fine with new functionality, bug fixes, or any change in the existing feature.

Regression testing is a type of software testing. Test cases are re-executed to check the previous functionality of the application is working fine, and the new changes have not produced any bugs.

Regression testing can be performed on a new build when there is a significant change in the original functionality. It ensures that the code still works even when the changes are occurring. Regression means Re-test those parts of the application, which are unchanged.

Regression tests are also known as the Verification Method. Test cases are often automated. Test cases are required to execute many times and running the same test case again and again manually, is time-consuming and tedious too.

Example of Regression testing

Here we are going to take a case to define the regression testing efficiently:

Consider a product Y, in which one of the functionality is to trigger confirmation, acceptance, and dispatched emails. It also needs to be tested to ensure that the change in the code not affected them. Regressing testing does not depend on any programming language like Java, C++, C#, etc. This method is used to test the product for modifications or any updates done. It ensures that any change in a product does not affect the existing module of the product. Verify that the bugs fixed and the newly added features not created any problem in the previous working version of the Software.

When can we perform Regression Testing?

We do regression testing whenever the production code is modified.

We can perform regression testing in the following scenario, these are:

1. When new functionality added to the application.

Example:

A website has a login functionality which allows users to log in only with Email. Now providing a new feature to do login using Facebook.

2. When there is a Change Requirement.

Example:

Remember password removed from the login page which is applicable previously.

3. When the defect fixed

Example:

Assume login button is not working in a login page and a tester reports a bug stating that the login button is broken. Once the bug fixed by developers, tester tests it to make sure Login Button is working as per the expected result. Simultaneously, tester tests other functionality which is related to the login button.

4. When there is a performance issue fix

Example:

Loading of a home page takes 5 seconds, reducing the load time to 2 seconds.

5. When there is an environment change

Example:

When we update the database from MySql to Oracle.

How to perform Regression Testing?

The need for regression testing comes when software maintenance includes enhancements, error corrections, optimization, and deletion of existing features. These modifications may affect system functionality. Regression Testing becomes necessary in this case.

Regression testing can be performed using the following techniques:

1. Re-test All:

Re-Test is one of the approaches to do regression testing. In this approach, all the test case suits should be re-executed. Here we can define re-test as when a test fails, and we determine the cause of the failure is a software fault. The fault is reported, we can expect a new version of the software in which defect fixed. In this case, we will need to execute the test again to confirm that the fault fixed. This is known as re-testing. Some will refer to this as confirmation testing.

The re-test is very expensive, as it requires enormous time and resources.

2. Regression test Selection:

• In this technique, a selected test-case suit will execute rather than an entire test-case suit.
• The selected test case suits divided in two cases
  1. Reusable Test cases.
  2. Obsolete Test cases.
• Reusable test cases can use in succeeding regression cycle.
• Obsolete test cases can't use in succeeding regression cycle.

3. Prioritization of test cases:

Prioritize the test case depending on business impact, critical and frequently functionality used. Selection of test cases will reduce the regression test suite.

What are the Regression Testing tools?

Regression Testing is a vital part of the QA process; while performing the regression we may face the below challenges:

• Time Consuming
  Regression Testing consumes a lot of time to complete. Regression testing involves existing tests again, so testers are not excited to re-run the test.
• Complex
  Regression Testing is complex as well when there is a need to update any product; lists of the test are also increasing.
• Communicating business rule
  Regression Testing ensures the existing product features are still in working order. Communication about regression testing with a non-technical leader can be a difficult task. The executive wants to see the product move forward and making a considerable time investment in regression testing to ensure existing functionality working can be hard.
• Identify Impact Area
• Test Cases Increases Release by Release
• Less Resources
• No Accuracy
• Repetitive Task
• Monotonous Job

Regression Testing Process

The regression testing process can be performed across the builds and the releases.

Regression testing across the builds

Whenever the bug fixed, we retest the Bug, and if there is any dependent module, we go for a Regression Testing.

For example, How we perform the regression testing if we have different builds as Build 1, Build 2, and Build 3, which having different scenarios.

Build1

• Firstly the client will provide the business needs.
• Then the development team starts developing the features.
• After that, the testing team will start writing the test cases; for example, they write 900 test cases for the release#1 of the product.
• And then, they will start implementing the test cases.
• Once the product is released, the customer performs one round of acceptance testing.
• And in the end, the product is moved to the production server.

Build2

• Now, the customer asks for 3-4 extra (new) features to be added and also provides the requirements for the new features.
• The development team starts developing new features.
• After that, the testing team will start writing the test case for the new features, and they write about 150 new test cases. Therefore, the total number of the test case written is 1050 for both the releases.
• Now the testing team starts testing the new features using 150 new test cases.
• Once it is done, they will begin testing the old features with the help of 900 test cases to verify that adding the new feature has damaged the old features or not.
• Here, testing the old features is known as Regression Testing.
• Once all the features (New and Old) have been tested, the product is handed over to the customer, and then the customer will do the acceptance testing.
• Once the acceptance testing is done, the product is moved to the production server.

Build3

• After the second release, the customer wants to remove one of the features like Sales.
• Then he/she will delete all the test cases which are belonging to the sales module (about 120 test cases).
• And then, test the other feature for verifying that if all the other features are working fine after removing the sales module test cases, and this process is done under the regression testing.

Note:

• Testing the stable features to ensure that it is broken because of the changes. Here changes imply that the modification, addition, bug fixing, or the deletion.
• Re-execution of the same test cases in the different builds or releases is to ensure that changes (modification, addition, bug fixing, or the deletion) are not introducing bugs in stable features.

Regression Testing Across the Release

The regression testing process starts whenever there is a new Release for same project because the new feature may affect the old elements in the previous releases.

To understand the regression testing process, we will follow the below steps:

Step1

There is no regression testing in Release#1 because there is no modification happen in the Release#1 as the release is new itself.

Step2

The concept of Regression testing starts from Release#2 when the customer gives some new requirements.

Step3

After getting the new requirements (modifying features) first, they (the developers and test engineers) will understand the needs before going to the impact analysis.

Step4

After understanding the new requirements, we will perform one round of impact analysis to avoid the major risk, but here the question arises who will do the Impact analysis?

Step5

The impact analysis is done by the customer based on their business knowledge, the developer based on their coding knowledge, and most importantly, it is done by the test engineer because they have the product knowledge.

Note: If a single person does, he/she may not cover all the impact areas, so we include all persons so that we may cover a maximum impact area, and Impact Analysis should be done at the early stages of the releases.

Step6

Once we are done with the impact area, then the developer will prepare the impact area (document), and the customer will also prepare the impact area document so that we can achieve the maximum coverage of impact analysis.

Step7

After completing the impact analysis, the developer, the customer, and the test engineer will send the Reports# of the impact area documents to the Test Lead. And in the meantime, the test engineer and the developer are busy working on the new test case.

Step8

Once the Test lead gets the Reports#, he/she will consolidate the reports and stored in the test case requirement repository for the release#1.

Note: Test case Repository: Here, we will save all the test case of releases.

Step9

After that, the Test Lead will take the help of RTM and pick the necessary regression test case from the test case repository, and those files will be placed in the Regression Test Suite.

Note:

• The test lead will store the regression test case in the regression test suite for no further confusion.
• Regression test suite: Here, we will save all the impact area test documents.
• Regression Test Cases: These are the test cases of the old releases text document which need to be re-executed as we can see in the below image:

Step10

After that, when the test engineer has done working on the new test cases, the test lead will assign the regression test case to the test engineer.

Step11

When all the regression test cases and the new features are stable and pass, then check the impact area using the test case until it is durable for old features plus the new features, and then it will be handed over to the customer.

Types of Regression Testing

The different types of Regression Testing are as follows:

1. Unit Regression Testing [URT]
2. Regional Regression Testing[RRT]
3. Full or Complete Regression Testing [FRT]

1) Unit Regression Testing [URT]

In this, we are going to test only the changed unit, not the impact area, because it may affect the components of the same module.

Example1

In the below application, and in the first build, the developer develops the Search button that accepts 1-15 characters. Then the test engineer tests the Search button with the help of the test case design technique.

Now, the client does some modification in the requirement and also requests that the Search button can accept the 1-35 characters. The test engineer will test only the Search button to verify that it takes 1-35 characters and does not check any further feature of the first build.

Example2

Here, we have Build B001, and a defect is identified, and the report is delivered to the developer. The developer will fix the bug and sends along with some new features which are developed in the second Build B002. After that, the test engineer will test only after the defect is fixed.

• The test engineer will identify that clicking on the Submit button goes to the blank page.
• And it is a defect, and it is sent to the developer for fixing it.
• When the new build comes along with the bug fixes, the test engineer will test only the Submit button.
• And here, we are not going to check other features of the first build and move to test the new features and sent in the second build.
• We are sure that fixing the Submitbutton is not going to affect the other features, so we test only the fixed bug.

Therefore, we can say that by testing only the changed feature is called the Unit Regression Testing.

2) Regional Regression testing [RRT]

In this, we are going to test the modification along with the impact area or regions, are called the Regional Regression testing. Here, we are testing the impact area because if there are dependable modules, it will affect the other modules also.

For example:

In the below image as we can see that we have four different modules, such as Module A, Module B, Module C, and Module D, which are provided by the developers for the testing during the first build. Now, the test engineer will identify the bugs in Module D. The bug report is sent to the developers, and the development team fixes those defects and sends the second build.

In the second build, the previous defects are fixed. Now the test engineer understands that the bug fixing in Module D has impacted some features in Module A and Module C. Hence, the test engineer first tests the Module D where the bug has been fixed and then checks the impact areas in Module A and Module C. Therefore, this testing is known as Regional regression testing.

While performing the regional regression testing, we may face the below problem:

Problem:

In the first build, the client sends some modification in requirement and also wants to add new features in the product. The needs are sent to both the teams, i.e., development and testing.

After getting the requirements, the development team starts doing the modification and also develops the new features based on the needs.

Now, the test lead sends mail to the clients and asks them that all are the impact areas that will be affected after the necessary modification have been done. Therefore, the customer will get an idea, which all features are needed to be tested again. And he/she will also send a mail to the development team to know which all areas in the application will be affected as a result of the changes and additions of new features.

And similarly, the customer sends a mail to the testing team for a list of impact areas. Hence, the test lead will collect the impact list from the client, development team, and the testing team as well.

This Impact list is sent to all the test engineers who look at the list and check if their features are modified and if yes, then they do regional regression testing. The impact areas and modified areas are all tested by the respective engineers. Every test engineer tests only their features that could have been affected as a result of the modification.

The problem with this above approach is that the test lead may not get the whole idea of the impact areas because the development team and the client may not have so much time to revert his/her mails.

Solution

To resolve the above problem, we will follow the below process:

When a new build comes along with the latest features and bug fixes, the testing team will arrange the meeting where they will talk about if their features are affecting because of the above modification. Therefore, they will do one round of Impact Analysis and generate the Impact List. In this particular list, the test engineer tries to enclose the maximum probable impact areas, which also decreases the chance of getting the defects.

When a new build comes, the testing team will follow the below procedure:

• They will do smoke testing to check the basic functionality of an application.
• Then they will test new features.
• After that, they will check the changed features.
• Once they are done with checking the changed features, the test engineer will re-test the bugs.
• And then they will check the impact area by performing the regional regression testing.

Disadvantage of using Unit and Regional Regression testing

Following are some of the drawbacks of using unit and Regional regression testing:

• We may miss some impact area.
• It is possible that we may identify the wrong impact area.

Note: We can say that the major work we do on the regional regression testing will lead us to get more number of defects. But, if we will perform the same dedication to work on the full regressing testing, we will get less number of defects. Therefore, we can determine here that enhancement in the testing effort will not help us to get more defects.

3) Full Regression testing [FRT]

During the second and the third release of the product, the client asks for adding 3-4 new features, and also some defects need to be fixed from the previous release. Then the testing team will do the Impact Analysis and identify that the above modification will lead us to test the entire product.

Therefore, we can say that testing the modified features and all the remaining (old) features is called the Full Regression testing.

When we perform Full Regression testing?

We will perform the FRT when we have the following conditions:

• When the modification is happening in the source file of the product. For example, JVM is the root file of the JAVA application, and if any change is going to happen in JVM, then the entire JAVA program will be tested.
• When we have to perform n-number of changes.

Note:

The regional regression testing is the ideal approach of regression testing, but the issue is, we may miss lots of defects while performing the Regional Regression testing.

And here we are going to solve this issue with the help of the following approach:

• When the application is given for the testing, the test engineer will test the first 10-14 cycle, and will do the RRT.
• Then for the 15th cycle, we do FRT. And again, for the next 10-15 cycle, we do Regional regression testing, and for the 31th cycle, we do the full regression testing, and we will continue like this.
• But for the last ten cycle of the release, we will perform only complete regression testing.

Therefore, if we follow the above approach, we can get more defects.

The drawback of doing regression testing manually repeatedly:

• Productivity will decrease.
• It is a difficult job to do.
• There is no consistency in test execution.
• And the test execution time is also increased.

Hence, we will go for the automation to get over with these issues; when we have n-number of the regression test cycle, we will go for the automation regression testing process.

Automated Regression Testing Process

Generally we go for the automation whenever there are multiple releases or multiple regression cycle or there is the repetitive task.

The automation regression testing process can be done in the following steps:

Note1:

The process of testing the application by using some tools is known as automation testing.

Suppose if we take one sample example of a Login module, then how we can perform the regression testing.

Here, the Login can be done in two ways, which are as follows:

Manually: In this, we will perform regression only one and twice.

Automation: In this, we will do the automation multiple times as we have to write the test scripts and do the execution.

Note2: In real-time, if we have faced some issues such as:

Step1

When the new release starts, we don't go for the automation because there is no concept of regression testing and regression test case as we understood this in the above process.

Step2

When the new release and the enhancement starts, we have two teams, i.e., manual team and the automation team.

Step3

The manual team will go through the requirements and also identify the impact area and hand over the requirement test suite to the automation team.

Step4

Now, the manual team starts working on the new features, and the automation team will start developing the test script and also start automating the test case, which means that the regression test cases will be converted into the test script.

Step5

Before they (automation team) start automating the test case, they will also analyze which all cases can be automated or not.

Step6

Based on the analysis, they will start the automation i.e., converting every regression test cases into the test script.

Step7

During this process, they will take help of the Regression cases because they don't have product knowledge as well as the tool and the application.

Step8

Once the test script is ready, they will start the execution of these scripts on the new application [old feature]. Since, the test script is written with the help of the regression feature or the old feature.

Step9

Once the execution is completed, we get a different status like Pass/fail.

Step10

If the status is failed, which means it needs to be re-confirmed manually, and if the Bug exists, then it will report to the concerned developer. When the developer fixes that bug, the Bug needs to be re-tested along with the Impact area by the manual test engineer, and also the script needs to be re-executed by the automation test engineer.

Step11

This process goes on until all the new features, and the regression feature will be passed.

Benefits of doing regression testing by the automation testing:

• Accuracy always exists because the task is done by the tools and tools never get bored or tired.
• The test script can be re-used across multiple releases.
• Batch execution is possible using the automation i.e.; all the written test scripts can be executed parallel or simultaneously.
• Even though the number of regression test case increase release per release, and we don't have to increase the automation resource since some regression case are already automated from the previous release.
• It is a time-saving process because the execution is always faster than the manual method.

How to select test cases for regression testing?

It was found from industry inspection. The several defects reported by the customer were due to last-minute bug fixes. These creating side effects and hence selecting the Test Case for regression testing is an art, not an easy task.

Regression test can be done by:

• A test case which has frequent defects
• Functionalities which are more visible to users.
• Test cases verify the core features of the product.
• All integration test cases
• All complex test cases
• Boundary value test cases
• A sample of successful test cases
• Failure of test cases

Regression Testing Tools

If Software undergoes frequent changes, regression testing costs also increase. In those cases, manual execution of test cases increases test execution time as well as costs. In that case, automation testing is the best choice. The duration of automation depends on the number of test cases that remain reusable for successive regression cycles.

Following are the essential tools used for regression testing:

Selenium

Selenium is an open-source tool. This tool used for automated testing of a web application. For browser-based regression testing, selenium used. Selenium used for UI level regression test for web-based application.

Ranorex Studio

All in one regression test automation for desktop, web, and mobile apps with built-in Selenium Web Driver. Ranorex Studio includes full IDE plus tools for codeless automation.

Quick Test Professional (QTP)

QTP is an automated testing tool used for Regression and Functional Testing. It is a Data-Driven, keyword-based tool. It used VBScript language for automation. If we open the QTP tool, we see the three buttons which are Record, Play and Stop. These buttons help to record every click and action performed on the computer system. It records the actions and play it back.

Rational Functional Tester (RTF)

Rational functional tester is a Java tool used to automate the test cases of software applications. RTF used for automating regression test cases, and it also integrates with the rational functional tester.

For more information about regression and automation testing tools refer to the below link:

https://www.javatpoint.com/automation-testing-tool

Regression Testing and Configuration Management

Configuration Management in the regression testing becomes imperative in Agile Environments, where a code is continuously modified. To ensure a valid regression test, we must follow the steps:

• Changes are not allowed in the code during the regression testing phase.
• A regression test case must be unaffected developer changes.
• The database used for regression testing must be isolated; changes are not allowed in the database.

Differences between Retesting and Regression Testing

Re-testing Testing means testing the functionality or bug again to ensure the code fixed. If not set, defects need not be re-opened. If fixed, the defect closed.

Re-testing is a type of testing which performed to check the test-cases that were unsuccessful in the final execution are successfully pass after the defects repaired.

Regression Testing means testing the software application when it undergoes a code change to ensure that new code has not affected other parts of the Software.

Regression testing is a type of testing executed to check whether a code has not changed the existing functionality of the application.

Differences between the Re-testing and Regression Testing are as follows:

Advantages of Regression Testing

Advantages of Regression Testing are:

• Regression Testing increases the product's quality.
• It ensures that any bug fix or changes do not impact the existing functionality of the product.
• Automation tools can be used for regression testing.
• It makes sure the issues fixed do not occur again.

Disadvantages of Regression Testing

There are several advantages of Regression Testing though there are disadvantages as well.

• Regression Testing should be done for small changes in the code because even a slight change in the code can create issues in the existing functionality.
• If in case automation is not used in the project for testing, it will time consuming and tedious task to execute the test again and again.

Conclusion

Regression Testing is one of the essential aspects as it helps to deliver a quality product which saves organizations time and money. It helps to provide a quality product by making sure that any change in the code does not affect the existing functionality.

For automating the regression test cases, there are several automation tools available. A tool should have the ability to update the test suite as the regression test suit needs to be updated frequently.