What is Regression Testing?
Regression testing is a black box testing techniques. It is used to authenticate a code change in the software does not impact the existing functionality of the product. Regression testing is making sure that the product works fine with new functionality, bug fixes, or any change in the existing feature.
Regression testing is a type of software testing. Test cases are re-executed to check the previous functionality of the application is working fine, and the new changes have not produced any bugs.
Regression testing can be performed on a new build when there is a significant change in the original functionality. It ensures that the code still works even when the changes are occurring. Regression means Re-test those parts of the application, which are unchanged.
Regression tests are also known as the Verification Method. Test cases are often automated. Test cases are required to execute many times and running the same test case again and again manually, is time-consuming and tedious too.
Example of Regression testing
Here we are going to take a case to define the regression testing efficiently:
Consider a product Y, in which one of the functionality is to trigger confirmation, acceptance, and dispatched emails. It also needs to be tested to ensure that the change in the code not affected them. Regressing testing does not depend on any programming language like Java, C++, C#, etc. This method is used to test the product for modifications or any updates done. It ensures that any change in a product does not affect the existing module of the product. Verify that the bugs fixed and the newly added features not created any problem in the previous working version of the Software.
When can we perform Regression Testing?
We do regression testing whenever the production code is modified.
We can perform regression testing in the following scenario, these are:
1. When new functionality added to the application.
A website has a login functionality which allows users to log in only with Email. Now providing a new feature to do login using Facebook.
2. When there is a Change Requirement.
Remember password removed from the login page which is applicable previously.
3. When the defect fixed
Assume login button is not working in a login page and a tester reports a bug stating that the login button is broken. Once the bug fixed by developers, tester tests it to make sure Login Button is working as per the expected result. Simultaneously, tester tests other functionality which is related to the login button.
4. When there is a performance issue fix
Loading of a home page takes 5 seconds, reducing the load time to 2 seconds.
5. When there is an environment change
When we update the database from MySql to Oracle.
How to perform Regression Testing?
The need for regression testing comes when software maintenance includes enhancements, error corrections, optimization, and deletion of existing features. These modifications may affect system functionality. Regression Testing becomes necessary in this case.
Regression testing can be performed using the following techniques:
1. Re-test All:
Re-Test is one of the approaches to do regression testing. In this approach, all the test case suits should be re-executed. Here we can define re-test as when a test fails, and we determine the cause of the failure is a software fault. The fault is reported, we can expect a new version of the software in which defect fixed. In this case, we will need to execute the test again to confirm that the fault fixed. This is known as re-testing. Some will refer to this as confirmation testing.
The re-test is very expensive, as it requires enormous time and resources.
2. Regression test Selection:
3. Prioritization of test cases:
Prioritize the test case depending on business impact, critical and frequently functionality used. Selection of test cases will reduce the regression test suite.
What are the Regression Testing tools?
Regression Testing is a vital part of the QA process; while performing the regression we may face the below challenges:
Regression Testing Process
The regression testing process can be performed across the builds and the releases.
Regression testing across the builds
Whenever the bug fixed, we retest the Bug, and if there is any dependent module, we go for a Regression Testing.
For example, How we perform the regression testing if we have different builds as Build 1, Build 2, and Build 3, which having different scenarios.
Regression Testing Across the Release
The regression testing process starts whenever there is a new Release for same project because the new feature may affect the old elements in the previous releases.
To understand the regression testing process, we will follow the below steps:
There is no regression testing in Release#1 because there is no modification happen in the Release#1 as the release is new itself.
The concept of Regression testing starts from Release#2 when the customer gives some new requirements.
After getting the new requirements (modifying features) first, they (the developers and test engineers) will understand the needs before going to the impact analysis.
After understanding the new requirements, we will perform one round of impact analysis to avoid the major risk, but here the question arises who will do the Impact analysis?
The impact analysis is done by the customer based on their business knowledge, the developer based on their coding knowledge, and most importantly, it is done by the test engineer because they have the product knowledge.
Note: If a single person does, he/she may not cover all the impact areas, so we include all persons so that we may cover a maximum impact area, and Impact Analysis should be done at the early stages of the releases.
Once we are done with the impact area, then the developer will prepare the impact area (document), and the customer will also prepare the impact area document so that we can achieve the maximum coverage of impact analysis.
After completing the impact analysis, the developer, the customer, and the test engineer will send the Reports# of the impact area documents to the Test Lead. And in the meantime, the test engineer and the developer are busy working on the new test case.
Once the Test lead gets the Reports#, he/she will consolidate the reports and stored in the test case requirement repository for the release#1.
Note: Test case Repository: Here, we will save all the test case of releases.
After that, the Test Lead will take the help of RTM and pick the necessary regression test case from the test case repository, and those files will be placed in the Regression Test Suite.
After that, when the test engineer has done working on the new test cases, the test lead will assign the regression test case to the test engineer.
When all the regression test cases and the new features are stable and pass, then check the impact area using the test case until it is durable for old features plus the new features, and then it will be handed over to the customer.
Types of Regression Testing
The different types of Regression Testing are as follows:
1) Unit Regression Testing [URT]
In this, we are going to test only the changed unit, not the impact area, because it may affect the components of the same module.
In the below application, and in the first build, the developer develops the Search button that accepts 1-15 characters. Then the test engineer tests the Search button with the help of the test case design technique.
Now, the client does some modification in the requirement and also requests that the Search button can accept the 1-35 characters. The test engineer will test only the Search button to verify that it takes 1-35 characters and does not check any further feature of the first build.
Here, we have Build B001, and a defect is identified, and the report is delivered to the developer. The developer will fix the bug and sends along with some new features which are developed in the second Build B002. After that, the test engineer will test only after the defect is fixed.
Therefore, we can say that by testing only the changed feature is called the Unit Regression Testing.
2) Regional Regression testing [RRT]
In this, we are going to test the modification along with the impact area or regions, are called the Regional Regression testing. Here, we are testing the impact area because if there are dependable modules, it will affect the other modules also.
In the below image as we can see that we have four different modules, such as Module A, Module B, Module C, and Module D, which are provided by the developers for the testing during the first build. Now, the test engineer will identify the bugs in Module D. The bug report is sent to the developers, and the development team fixes those defects and sends the second build.
In the second build, the previous defects are fixed. Now the test engineer understands that the bug fixing in Module D has impacted some features in Module A and Module C. Hence, the test engineer first tests the Module D where the bug has been fixed and then checks the impact areas in Module A and Module C. Therefore, this testing is known as Regional regression testing.
While performing the regional regression testing, we may face the below problem:
In the first build, the client sends some modification in requirement and also wants to add new features in the product. The needs are sent to both the teams, i.e., development and testing.
After getting the requirements, the development team starts doing the modification and also develops the new features based on the needs.
Now, the test lead sends mail to the clients and asks them that all are the impact areas that will be affected after the necessary modification have been done. Therefore, the customer will get an idea, which all features are needed to be tested again. And he/she will also send a mail to the development team to know which all areas in the application will be affected as a result of the changes and additions of new features.
And similarly, the customer sends a mail to the testing team for a list of impact areas. Hence, the test lead will collect the impact list from the client, development team, and the testing team as well.
This Impact list is sent to all the test engineers who look at the list and check if their features are modified and if yes, then they do regional regression testing. The impact areas and modified areas are all tested by the respective engineers. Every test engineer tests only their features that could have been affected as a result of the modification.
The problem with this above approach is that the test lead may not get the whole idea of the impact areas because the development team and the client may not have so much time to revert his/her mails.
To resolve the above problem, we will follow the below process:
When a new build comes along with the latest features and bug fixes, the testing team will arrange the meeting where they will talk about if their features are affecting because of the above modification. Therefore, they will do one round of Impact Analysis and generate the Impact List. In this particular list, the test engineer tries to enclose the maximum probable impact areas, which also decreases the chance of getting the defects.
When a new build comes, the testing team will follow the below procedure:
Disadvantage of using Unit and Regional Regression testing
Following are some of the drawbacks of using unit and Regional regression testing:
Note: We can say that the major work we do on the regional regression testing will lead us to get more number of defects. But, if we will perform the same dedication to work on the full regressing testing, we will get less number of defects. Therefore, we can determine here that enhancement in the testing effort will not help us to get more defects.
3) Full Regression testing [FRT]
During the second and the third release of the product, the client asks for adding 3-4 new features, and also some defects need to be fixed from the previous release. Then the testing team will do the Impact Analysis and identify that the above modification will lead us to test the entire product.
Therefore, we can say that testing the modified features and all the remaining (old) features is called the Full Regression testing.
When we perform Full Regression testing?
We will perform the FRT when we have the following conditions:
The regional regression testing is the ideal approach of regression testing, but the issue is, we may miss lots of defects while performing the Regional Regression testing.
And here we are going to solve this issue with the help of the following approach:
Therefore, if we follow the above approach, we can get more defects.
The drawback of doing regression testing manually repeatedly:
Hence, we will go for the automation to get over with these issues; when we have n-number of the regression test cycle, we will go for the automation regression testing process.
Automated Regression Testing Process
Generally we go for the automation whenever there are multiple releases or multiple regression cycle or there is the repetitive task.
The automation regression testing process can be done in the following steps:
The process of testing the application by using some tools is known as automation testing.
Suppose if we take one sample example of a Login module, then how we can perform the regression testing.
Here, the Login can be done in two ways, which are as follows:
Manually: In this, we will perform regression only one and twice.
Automation: In this, we will do the automation multiple times as we have to write the test scripts and do the execution.
Note2: In real-time, if we have faced some issues such as:
When the new release starts, we don't go for the automation because there is no concept of regression testing and regression test case as we understood this in the above process.
When the new release and the enhancement starts, we have two teams, i.e., manual team and the automation team.
The manual team will go through the requirements and also identify the impact area and hand over the requirement test suite to the automation team.
Now, the manual team starts working on the new features, and the automation team will start developing the test script and also start automating the test case, which means that the regression test cases will be converted into the test script.
Before they (automation team) start automating the test case, they will also analyze which all cases can be automated or not.
Based on the analysis, they will start the automation i.e., converting every regression test cases into the test script.
During this process, they will take help of the Regression cases because they don't have product knowledge as well as the tool and the application.
Once the test script is ready, they will start the execution of these scripts on the new application [old feature]. Since, the test script is written with the help of the regression feature or the old feature.
Once the execution is completed, we get a different status like Pass/fail.
If the status is failed, which means it needs to be re-confirmed manually, and if the Bug exists, then it will report to the concerned developer. When the developer fixes that bug, the Bug needs to be re-tested along with the Impact area by the manual test engineer, and also the script needs to be re-executed by the automation test engineer.
This process goes on until all the new features, and the regression feature will be passed.
Benefits of doing regression testing by the automation testing:
How to select test cases for regression testing?
It was found from industry inspection. The several defects reported by the customer were due to last-minute bug fixes. These creating side effects and hence selecting the Test Case for regression testing is an art, not an easy task.
Regression test can be done by:
Regression Testing Tools
If Software undergoes frequent changes, regression testing costs also increase. In those cases, manual execution of test cases increases test execution time as well as costs. In that case, automation testing is the best choice. The duration of automation depends on the number of test cases that remain reusable for successive regression cycles.
Following are the essential tools used for regression testing:
Selenium is an open-source tool. This tool used for automated testing of a web application. For browser-based regression testing, selenium used. Selenium used for UI level regression test for web-based application.
All in one regression test automation for desktop, web, and mobile apps with built-in Selenium Web Driver. Ranorex Studio includes full IDE plus tools for codeless automation.
Quick Test Professional (QTP)
QTP is an automated testing tool used for Regression and Functional Testing. It is a Data-Driven, keyword-based tool. It used VBScript language for automation. If we open the QTP tool, we see the three buttons which are Record, Play and Stop. These buttons help to record every click and action performed on the computer system. It records the actions and play it back.
Rational Functional Tester (RTF)
Rational functional tester is a Java tool used to automate the test cases of software applications. RTF used for automating regression test cases, and it also integrates with the rational functional tester.
For more information about regression and automation testing tools refer to the below link:
Regression Testing and Configuration Management
Configuration Management in the regression testing becomes imperative in Agile Environments, where a code is continuously modified. To ensure a valid regression test, we must follow the steps:
Differences between Retesting and Regression Testing
Re-testing Testing means testing the functionality or bug again to ensure the code fixed. If not set, defects need not be re-opened. If fixed, the defect closed.
Re-testing is a type of testing which performed to check the test-cases that were unsuccessful in the final execution are successfully pass after the defects repaired.
Regression Testing means testing the software application when it undergoes a code change to ensure that new code has not affected other parts of the Software.
Regression testing is a type of testing executed to check whether a code has not changed the existing functionality of the application.
Differences between the Re-testing and Regression Testing are as follows:
Advantages of Regression Testing
Advantages of Regression Testing are:
Disadvantages of Regression Testing
There are several advantages of Regression Testing though there are disadvantages as well.
Regression Testing is one of the essential aspects as it helps to deliver a quality product which saves organizations time and money. It helps to provide a quality product by making sure that any change in the code does not affect the existing functionality.
For automating the regression test cases, there are several automation tools available. A tool should have the ability to update the test suite as the regression test suit needs to be updated frequently.
Next TopicSmoke Testing