Javatpoint Logo
Javatpoint Logo

Python doctest Module | Document and Test Code

In this tutorial, we will learn about the doctest module. It is a testing framework that helps us to document and test code simultaneously. This module allows us to document and test our code, which is essential to coding. By default, we can use the docstring to write the class description or function to provide a better understanding of the code.

We won't need to install any third party to use the doctest module. However, one should have a basic understanding of Python.

Documentation Example

Documenting the code is the best practice, and the senior developer suggests to follow this practice often. It would be right to say coding and its documentation are equally important. Python offers various methods to document a project, application, module, or script. External documentation is typically needed for larger projects, while smaller projects can use descriptive names, comments, and docstrings for documentation purposes.

As we can see that get_value() method helps to get the given value. We have properly documented the above function which makes it more readable and maintainable. In the above code, we have also used the comments to explain what code does and why. In Python, comments starts with the # symbol and can occupy its own line.

On the other hand, the comments have some drawbacks -

  • The interpreter or compiler ignores the comments, which makes them unreadable at runtime.
  • As code evolves, comments and docstrings can often become outdated and remain unchanged.

Documentation strings are a very useful feature of Python that can help us to document our code as we code. And it has an advantage over the comments because the interpreter doesn't ignore them.

The docstring is the active part of the code; we can access it at runtime. Python provides __doc__ special attributes on our packages, modules, classes, methods, and functions.

Python allows for the inclusion of docstrings in packages, modules, classes, methods, and functions. To write effective docstrings, it is recommended to follow the conventions and guidelines outlined in PEP 257.

Introduction to Python's doctest Module

In this section, you will be introduced to the Python doctest module. This module is a standard library component and does not require installation of any external library for use in everyday coding. You will learn about doctest, including its purpose and when to use it. The discussion will begin with an overview of what doctest is.

How it works

The doctest is a lightweight testing framework that allows us to do quick and straightforward test automation. The doctest module in Python can read test cases from the project documentation and code docstrings. This framework comes packaged with the Python interpreter, aligning with the "batteries-included" philosophy.

Doctest searches for text resembling Python interactive sessions in our documentation and docstrings. It then separates the text into executable code and expected results, runs the code, and compares the execution results with the desired results. We can use doctest from either your code or command line.

Writing doctest Tests in Python

We have got the understanding of the doctest so far. Now, we will learn to check how to return value of the functions, methods, and other callable. We will also learn how to create test cases for code. The most common use case of the testing is checking the return value of functions, methods and other callables. In the following example, we have a function add(a, b) that accepts two arguments and return the sum of two values.

Example -

We can document the above code for a better understanding. After adding the documentation, it will look like as below.

The docstring includes two examples of using add() function. There is an initial line >>>, which has the add() with two numeric arguments, and the second line is the expected output.

To run the doctest, we can use the following command.

The above command will not work or it won't display output on the screen. In the other words, the doctest ran all the test cases but won't find failing test. We can see the verbose about the program using the below command.

Output -

1 items had no tests:
    add
0 tests in 1 items.
0 passed and 0 failed.
Test passed.

The above command ran with the -v option, which produces the detailed output. The first two highlighted lines demonstrate the tests and expected results. The subsequent line indicates the test's success with the word "ok," indicating that the targeted test has passed. In this example, both tests have passed, as shown in the last highlighted line.

The input and expected output are specified in the docstring, and the doctest module uses this information to test the function's output. The docstring is processed and the resulting text is executed as Python shell commands, and the output is compared with the expected result extracted from the docstring.

In the below example, we will write the factorial() function which will return the factorial of the given number. Let's understand the following example.

Example -

Output:

Trying:
    factorial(5)
Expecting:
    120
ok
Trying:
    factorial(6)
Expecting:
    720
ok
1 items had no tests:
    add
1 items passed all tests:
   2 tests in add.factorial
2 tests in 2 items.
2 passed and 0 failed.
Test passed.

Now we test for the failure. What if we change the logic?

Output:

Failed example:
    factorial(6)
Expected:
    720
Got:
    1
1 items had no tests:
    add
**********************************************************************
1 items had failures:
   2 of   2 in add.factorial
2 tests in 2 items.
0 passed and 2 failed.
***Test Failed*** 2 failures.

As we see that, it shows the failed test. If we set the verbose as False then it will return only failed message.

Exploring Some Limitation of doctest

One of the major limitations of doctest compared to other testing frameworks is the absence of features similar to fixtures in pytest or the setup and teardown methods in unittest. It must be written in each affected docstring if setup and teardown code is required. An alternative is a unittest API, which offers setup and teardown options.

Another limitation of doctest is its strict comparison of the expected output with the actual output, requiring an exact match. If even a single character does not match, the test will fail, making it challenging to test certain Python objects accurately.

Let's understand the following limitation of the doctest.

Example -

The above student class has name and favorite subject variables. The class initializer converts the input subjects into the set object. We get the favorite subjects of a student from the favorite_subjects() property. As we know, sets store the element in random order. Hence our test case will fail most of the time. Now we run the following command to see the output.

Output -

$ python -m doctest -v add.py
Trying:
    peter = Student("Peter", {"English", "Math", "Science"})
Expecting nothing
ok
Trying:
    peter.favorite_subjects
Expecting:
    {"English", "Math", "Science"}
**********************************************************************
File "C:\Users\User\Desktop\my_project\site_checker\add.py", line ?, in add.Student.favorite_subjects
Failed example:
    peter.favorite_subjects
Exception raised:
    Traceback (most recent call last):
      File "C:\Python\lib\doctest.py", line 1329, in __run
        exec(compile(example.source, filename, "single",
      File "", line 1, in 
        peter.favorite_subjects
    AttributeError: 'Student' object has no attribute 'favorite_subjects'
3 items had no tests:
    add
    add.Student
    add.Student.__init__
**********************************************************************
1 items had failures:
   1 of   2 in add.Student.favorite_subjects
2 tests in 4 items.
1 passed and 1 failed.
***Test Failed*** 1 failures.

In the first test, the object of Student class is instantiated, which doesn't have any expected output. The second output checks the expected output against the function's actual output. The outputs are different because sets are unordered collections, which make the test fail.

To solve such issue, we can use the sorted() function in the doctest test. Let's see the following solution.

Example -

Now we have used the sorted() function, which returns the sorted list of subjects. We also need to update the expected output to get all tests passed.

Another limitation of doctest is that it needs the capability to run tests with multiple sets of input arguments and expected outputs. It is referred to as parameterization and requires the testing framework to automatically run the target test with all combinations and evaluate if they all pass.

Although doctest does not have built-in support for parameterizations, we can still achieve the benefits of having multiple test cases with a single test function using certain techniques. Parameterizations enable you to efficiently create a large number of test cases and improve your test coverage, ultimately increasing your productivity.

Example -

In the code, we create two lists to store input arguments and expected outputs for the even_numbers() function. The loop then utilizes the zip() function to iterate over both lists in parallel and run a test in each iteration. The test compares the actual output of even_numbers() with the corresponding expected output from the second list.

Command Line Doctest

So far, we have learned the basics of the doctest and running tests from the command line. The most bare-bones ways to use the below command is with the target file or module as an argument as we did in the previous section. We have already learned about the -v and -verbose command to get the doctest detailed report of the all tests along with summary. In addition to this command-line option, doctest also accepts additional options.

Option Description
-h. - --help It shows the command-line help for the doctest.
-o or -option It specifies one or more doctest option flags or directives to use while running your tests
-f, --fail-fast It stops running your doctest tests after the first failure

We will likely use doctest from the command line in most situations. The most intricate option among the ones listed in the table is the -o or --option, due to the extensive list of flags that can be utilized with this option.

Conclusion

Now you are familiar with the process of writing code examples that serve both as documentation and test cases. To execute your examples as tests, you utilized the doctest module from the Python standard library. This module provides you with a simple testing framework that is easy to learn, enabling you to quickly begin automating your testing procedure.







Youtube For Videos Join Our Youtube Channel: Join Now

Feedback


Help Others, Please Share

facebook twitter pinterest

Learn Latest Tutorials


Preparation


Trending Technologies


B.Tech / MCA