Code Coverage and Test Coverage in Python

In this tutorial, we will learn some important differences between code coverage and test coverage. We will also understand the basics of these concepts. These concepts are quite similar so it can be difficult to differentiate. Acquiring knowledge about these principles will allow you to identify the important aspects of your projects that may lack comprehensive test coverage, ultimately enhancing the robustness of your application.

In a general sense, code coverage is an objective which means when your code is executed within a test, it's deemed to have achieved full code coverage. On the other hand, test coverage is subjective and can be influenced by the elements you consider and the scope you define.

What is Code Coverage?

When your project grows larger, it becomes tough to determine that all parts of your code are thoroughly tested. This also makes it difficult to figure out how much of your code isn't tested and where those untested parts are. This is where code coverage becomes valuable. It reveals the parts of your code that lack test coverage, enabling you to explore and identify these gaps effectively.

Code coverage works by examining the parts of your code that are used during testing. It also gives you a percentage to show how much of your code is tested. Think of it like shaping clay - to achieve 100% code coverage, you adjust your tests just like molding a piece of clay into a specific shape.

Suppose your code is like a big jigsaw puzzle, and each puzzle piece is a line of code. Code coverage keeps track of which puzzle pieces have been put in place during testing and which ones are still missing. It tells you the percentage of the puzzle that is complete.

This information is helpful because it shows you which parts of your code might have issues or bugs because they haven't been tested yet. It's like a map that guides you to the areas that need more attention and testing, making your software more reliable.

Sure, here are the characteristics of code coverage explained in simple terms:

Measuring Testing Progress - Code coverage helps you see how much of your code has been tested. It's like a progress bar that tells you how far you've come in testing your software.
Uncovered Areas - It highlights the parts of your code that haven't been tested. These are like dark corners in a room that need a closer look.
Percentage Indicator - Code coverage gives you a percentage to show how much of your code is tested. It's like a scorecard for your testing efforts.
Detecting Gaps - It helps you find gaps in your testing. Just like a puzzle with missing pieces, code coverage identifies areas where you might have overlooked testing.
Improving Reliability - By focusing on the uncovered parts, you can make your software more reliable. It's like fixing weak spots in a fence to make it more secure.
Guidance for Testing - It's same as a roadmap that shows you where to concentrate your testing efforts, making your testing strategy more effective.
Quality Assurance - Code coverage contributes to the overall quality of your software by ensuring that all code is thoroughly tested, reducing the chances of unexpected issues.

In summary, code coverage is a useful tool that helps you ensure that your software is well-tested and reliable by indicating what parts of your code need more attention.

Code Coverage in Python

Code coverage in Python is a way to measure how much of your Python code has been tested by your test cases. It helps you identify which parts of your Python program are covered by tests and which parts are not. This is important for ensuring that your code is robust and reliable. We will use the unittest module along with the coverage module. The coverage module is used to get the code coverage. We can install it using the following command.

Let's understand the following example -

Example -Sample.py

def multiply_even(num1, num2):
    if num1 % 2 == 0 and num2 % 2 == 0:
        return num1 * num2
    else:
        return None

def divide_by_5(num1, num2):
    if num1 % 5 == 0 and num2 % 5 == 0:
        return num1 / num2
    else:
        return None

def concatenate_strings(str1, str2):
    if isinstance(str1, str) and isinstance(str2, str):
        return str1 + str2
    else:
        return None

def common_factors(num1, num2):
    factors1 = [i for i in range(1, num1 + 1) if num1 % i == 0]
    factors2 = [i for i in range(1, num2 + 1) if num2 % i == 0]
    common_factors = list(set(factors1) & set(factors2))
    
    if common_factors:
        return common_factors
    else:
        return None

def is_palindrome(word):
    if isinstance(word, str) and word == word[::-1]:
        return True
    else:
        return False

Test Cases - sample_test.py

import unittest
from sample import (
    sum_negative,
    sum_positive,
    multiply_even,
    divide_by_5,
    concatenate_strings,
    common_factors,
    is_palindrome,
)

class TestFunctions(unittest.TestCase):

    def test_sum_negative(self):
        self.assertEqual(sum_negative(-5, -3), -8)
        self.assertEqual(sum_negative(5, -3), None)
        self.assertEqual(sum_negative(-5, 3), None)

    def test_sum_positive(self):
        self.assertEqual(sum_positive(5, 3), 8)
        self.assertEqual(sum_positive(-5, 3), None)
        self.assertEqual(sum_positive(5, -3), None)

    def test_multiply_even(self):
        self.assertEqual(multiply_even(2, 4), 8)
        self.assertEqual(multiply_even(3, 4), None)
        self.assertEqual(multiply_even(2, 5), None)

    def test_divide_by_5(self):
        self.assertEqual(divide_by_5(10, 5), 2.0)
        self.assertEqual(divide_by_5(15, 7), None)
        self.assertEqual(divide_by_5(10, 3), None)

    def test_concatenate_strings(self):
        self.assertEqual(concatenate_strings("Hello, ", "World!"), "Hello, World!")
        self.assertEqual(concatenate_strings("Hello, ", 123), None)
        self.assertEqual(concatenate_strings(123, "World!"), None)

    def test_common_factors(self):
        self.assertEqual(common_factors(12, 18), [1, 2, 3, 6])
        self.assertEqual(common_factors(7, 12), [1])
        self.assertEqual(common_factors(7, 13), None)

    def test_is_palindrome(self):
        self.assertTrue(is_palindrome("racecar"))
        self.assertTrue(is_palindrome("madam"))
        self.assertFalse(is_palindrome("hello"))
        self.assertFalse(is_palindrome(123))

if __name__ == '__main__':
    unittest.main()

Now run the following commands to get the total coverage.

coverage run -m unittest discover
coverage report -m

Test Coverage

Test coverage is like a measure of how well our tests cover all the different things our code can do. Imagine you have a bunch of clay, and you want to make everything you possibly can with it. Our previous test that covered 100% of the code is not enough when it comes to test coverage because there are so many things you can create with clay. So, when we talk about test coverage, we mean making sure our tests consider all the different possibilities and scenarios in our code.

Characteristics of Test Coverage

The characteristics of test coverage are important aspects that describe how effectively tests assess the code. Here are the key characteristics of test coverage:

Scope: Test coverage evaluates the extent to which code is exercised by tests. It measures which portions of the code are tested and which are not.
Measurement Units: Test coverage can be measured using various units, such as lines of code (line coverage), branches (branch coverage), statements (statement coverage), or functions (function coverage).
Percentage: Test coverage is often expressed as a percentage. For example, 70% code coverage means that 70% of the code has been exercised by the tests.
Code Paths: Test coverage can also consider different execution paths through the code, including the main or alternative paths, to ensure that all logical flows are tested.
Branch Coverage: This measures the percentage of decision points (branches) that have been taken during testing, ensuring that both true and false branches are executed.
Statement Coverage: It focuses on assessing whether each statement in the code has been executed by tests.
Function or Method Coverage: This evaluates whether all functions or methods in the code have been invoked by tests.
Boundary Coverage: Ensures that tests cover the edges and boundaries of input ranges, helping detect potential issues at these critical points.
Code Complexity: Some coverage metrics consider code complexity factors like cyclomatic complexity, which accounts for branching and decision points in the code.

These characteristics help developers and testers assess the comprehensiveness and effectiveness of their testing efforts, ensuring that the code is well-exercised and potential issues are identified.

Test coverage in Python

In contrast to code coverage, where only a limited number of assertions were required, we'll need to include a greater number of assertions in this scenario. When working with the sample code mentioned in the previous section, we have a set of additional assertions to consider.

Let's understand the following example -

Example -

import unittest

def multiply_even(a, b):
    if a % 2 == 0 and b % 2 == 0:
        return a * b
    else:
        return None

def divide_by_5(a, b):
    if b == 0:
        return None
    if a % 5 == 0:
        return a / b
    else:
        return None

def concatenate_strings(str1, str2):
    if isinstance(str1, str) and isinstance(str2, str):
        return str1 + str2
    else:
        return None

def common_factors(x, y):
    factors = []
    for i in range(1, min(x, y) + 1):
        if x % i == 0 and y % i == 0:
            factors.append(i)
    return factors

def is_palindrome(s):
    if isinstance(s, str):
        s = s.lower().replace(" ", "")
        return s == s[::-1]
    return False

class TestFunctions(unittest.TestCase):

    def test_multiply_even(self):
        self.assertEqual(multiply_even(2, 4), 8)
        self.assertEqual(multiply_even(3, 4), None)
        self.assertEqual(multiply_even(2, 5), None)

    def test_divide_by_5(self):
        self.assertEqual(divide_by_5(10, 5), 2.0)
        self.assertEqual(divide_by_5(15, 7), None)
        self.assertEqual(divide_by_5(10, 3), None)

    def test_concatenate_strings(self):
        self.assertEqual(concatenate_strings("Hello, ", "World!"), "Hello, World!")
        self.assertEqual(concatenate_strings("Hello, ", 123), None)
        self.assertEqual(concatenate_strings(123, "World!"), None)

    def test_common_factors(self):
        self.assertEqual(common_factors(12, 18), [1, 2, 3, 6])
        self.assertEqual(common_factors(7, 12), [1])

    def test_is_palindrome(self):
        self.assertTrue(is_palindrome("racecar"))
        self.assertTrue(is_palindrome("madam"))
        self.assertFalse(is_palindrome("hello"))
        self.assertFalse(is_palindrome(123))

Conclusion

This tutorial included the essential differences between code coverage and test coverage, shedding light on their significance in software development. Code coverage is an objective measure, ensuring that all parts of the code are executed during tests, helping identify untested areas. On the other hand, test coverage is more subjective, focusing on the comprehensiveness of tests based on different scenarios and potential outcomes.

The provided Python code samples demonstrate how to implement test coverage using the unittest framework and the coverage module. These test cases thoroughly assess functions, offering insights into their behavior and reliability.

Next TopicCounting-pairs-x-y-in-an-array-where-xy-yx

← prev next →