Unit Testing - Scientific Computing with Python

Unit testing is a method of testing your code by writing tests for individual functions.

Let’s say you write a package which provides a function. You want to convince someone (espeically yourself) that the function works as intended. An excellent way to do this is to write unit tests, which (assuming they pass) demonstrate that your function does what is supposed to do, at least for the situations you test.

unittest package¶

unittest is a built-in package which provides unit testing capabilities.

Generally, you define classes that inherit from unittest.TestCase. Then you can add methods which test different functionality.

import unittest

import numpy as np

'foo'.upper().isupper()

True

import unittest

class TestStringMethods(unittest.TestCase):

    def test_upper(self):
        self.assertEqual('foo'.upper(), 'FOO')

    def test_isupper(self):
        self.assertTrue('ABC'.isupper())
        self.assertFalse('Foo'.isupper())

    def test_split(self):
        s = 'hello world'
        self.assertEqual(s.split(), ['hello', 'world'])
        # check that s.split fails when the separator is not a string
        with self.assertRaises(TypeError):
            s.split(2)

We can then run our tests using unittest.main() (the arguments below are passed in so we can run in Jupyter).

unittest.main(argv=['first-arg-is-ignored'], exit=False)

.F..F
======================================================================
FAIL: test_exact (__main__.TestFloatArithmetic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython-input-11-bc6eae005c7e>", line 7, in test_exact
    self.assertEqual(1.2 - 1.0, 0.2 )
AssertionError: 0.19999999999999996 != 0.2

======================================================================
FAIL: test_upper (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython-input-13-0da140a724b9>", line 6, in test_upper
    self.assertEqual('abc'.upper(), 'FOO')
AssertionError: 'ABC' != 'FOO'
- ABC
+ FOO


----------------------------------------------------------------------
Ran 5 tests in 0.006s

FAILED (failures=2)

<unittest.main.TestProgram at 0x7f1ff0192fd0>

When using floating-point numbers, use assertAlmostEqual instead of assertEqual to test for numerical equality

1.2 - 1.0

0.19999999999999996

class TestFloatArithmetic(unittest.TestCase):
    
    def test_approx(self):
        self.assertAlmostEqual(1.2 - 1.0, 0.2)
        
    def test_exact(self):
        self.assertEqual(1.2 - 1.0, 0.2 )

unittest.main(argv=['first-arg-is-ignored'], exit=False)

.F...
======================================================================
FAIL: test_exact (__main__.TestFloatArithmetic)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<ipython-input-11-bc6eae005c7e>", line 7, in test_exact
    self.assertEqual(1.2 - 1.0, 0.2 )
AssertionError: 0.19999999999999996 != 0.2

----------------------------------------------------------------------
Ran 5 tests in 0.004s

FAILED (failures=1)

<unittest.main.TestProgram at 0x7f1ff0023700>

Running from command line¶

The more common way to run unit tests is to have them in a test folder or file test.py. You can then run tests using

python -m unittest test.py

Or use pytest via

pytest test.py

pytest is another Python testing framework - it is compatible with unittest, and has additional funcitonality which we aren’t going to cover.

Test-Driven Development¶

You don’t need to wait to implement everything in order to write your tests. Writing your tests first is called test-driven development. One advantage of test-driven development is that you’ll know when you have succeeded in your implementation, since all your tests will pass.

Let’s consider a suite of tests that would test a power_method function:

import numpy as np
np.ones((5,5))

array([[1., 1., 1., 1., 1.],
       [1., 1., 1., 1., 1.],
       [1., 1., 1., 1., 1.],
       [1., 1., 1., 1., 1.],
       [1., 1., 1., 1., 1.]])

class TestPowerMethod(unittest.TestCase):
    
    def test_eigenpair(self):
        n = 5
        A = np.random.randn(n, n)
        A = A + A.T # make symmetric
        lam, v = powermethod(A)
        v2 = A @ v
        self.assertAlmostEqual(np.linalg.norm(v2 - lam * v), 0)
    
    def test_norm1(self):
        n = 5
        A = np.random.randn(n, n)
        A = A + A.T # make symmetric
        lam, v = powermethod(A)
        self.assertAlmostEqual(np.linalg.norm(v), 1)
    
    def test_rank1(self):
        n = 5
        A = np.ones((n,n))
        lam, v = powermethod(A)
        # check that v is close to constant function.
        self.assertAlmostEqual(np.linalg.norm(v - np.ones(n)/np.sqrt(n)), 0)

Exercise¶

Implement a function powermethod which satisfies the above tests (using the Power method algorithm, of course)

## Your code here

Further Examples¶

Check out test.py in each homework assignment, which is used for autograding.
You can find another example in the repository python-packages

Continuous Integration¶

Continuous Integration, or CI, is the practice of automatically building code and running tests continuously. Continuously in this case generally means any time changes are made, which might be multiple times a day.

The advantage of CI is that when you make changes to your code, you quickly find out if there are problems that need to be solved if your tests fail. You can run these tests before merging branches in your git repository, making sure that checks pass.

GitHub actions is one way of implementing CI. This is what we are using in this class - you can find an example in the python-packages repository.

Another popular option which you may see in open source software is Travis-CI.

Both these platforms are configured using a *.yml file. See the python-packages repository for an example.

You can do more than just run unit tests using CI, such as running integration tests, verifying that data analyses don’t change, etc.