Testing the Untestable
Abstract
Sometimes you come across a code-base which has no unit-tests. This makes contributing to the project risky. If an application has not been written with unit-tests in mind from the beginning, it is often very difficult to add them later on. This article takes one such project and uses the fantastic unittest.mock Python module to add tests without modifying the code.
Preface
From personal experience with old code-base maintainance, projects become hard to unit-test for four main reasons:
- Violations of Demeter's Law.
- Fuzzy borders between the main application code and external I/O (like stdout, databases, network services, ...).
- Hidden global state.
- Function-call side-effects / non-pure functions.
There are other reasons, but these are by far the ones I've ran across the most.
Note
The following article mentions the open-source project munin-influxdb. It is only taken as illustrative example and I do not intend to paint this in a bad light.
As a starting point, we'll look at commit 8b6b4efca of munin-influxdb.
THe project confronts us with several challenges:
Challenge 1 - No Clear Entry-Point
The project contains a shell-script and separate modules in the bin subfolder. Additionally, the setup.py module does not define any entry_points
Issues
- As nothing is packaged probably it becomes non-trivial to install. Neither in an isolated virtual environment, nor on the system.
- Testing the main entry-point becomes more difficult as it is a bash script.
Challenge 2 - Global State
In order to add unit-tests it is important that one can look at a piece (a "unit") of code and can say what it's doing without looking through other parts of the code. The project makes this difficult in it's current state. It makes use of a Settings class containing key/value pairs which are populated at run-time by various methods, and are not known in advance. If there's a function taking a Settings instance as argument it's not trivial to "guess" the structure of that instance to be able to run the function.
