Fuel/How to Test Your Code

Why to write tests?

• Testing helps you find errors in your code.
• Testing helps you write better code.
• Writing test cases will save you time later.
• Unit tests provide immediate feedback on your code.
• Test cases document intent.

These are some guidelines for writing unit tests.

1. A testing unit should focus on one tiny bit of functionality and prove it is correct.
2. Each test unit must be fully independent. Each of them must be able to run alone, and also within the test suite, regardless of the order they are called. The implication of this rule is that each test must be loaded with a fresh dataset and may have to do some cleanup afterwards. This is usually handled by setUp() and tearDown() methods.
3. Write both valid and negative tests
4. Try to avoid mocking and stubbing, favoring test parameters or attributes instead. When resorting to mocking and stubbing, only mock against a small, stable, obvious (or documented) API, so stubs are likely to represent reality after future refactoring.
5. Check the state of the system after test thoroughly. Example: when deployment is finished, notification should be created. The right test will check that the only one notification was created, and all fields of it are exactly which we expect. Bad test will filter DB to find any success notifications and will pass if any was found.
6. Test how the code handles exceptions. Think about how will user be notified about an error. It is bad practice to just show "Server error has occurred".
   • Example: test does PUT request with data which break the system inside (somewhere ValueError is raised). Response status code must not be 500.
   • Another example: Due to temporary network connectivity issue agent fails to access Nailgun API.
7. Test wisely. It perhaps has no sense to test if new notification can be created with default params, but it makes a lot of sense to test if valid notification object (with all attrs set) was created on node discovery. What message will be in notification if in POST request to /api/nodes memory is None? Or 0? Or some number less than 1 Gb, will it be rounded correctly?
8. Unit tests are also code. It must be clear, structured, follow DRY and other code standards.
9. If you changed some code unit, take a piece of paper and write down all the other code units that use this one. Go and find tests for each of these units, and check if there are tests for each unit which test interaction between them. If you discovered that there is no test for one of such code units, you must write it, no matter who missed it before.
10. Take a minute and think about what could possibly fail in your code. Or to what you've not payed attention enough. Make yourself a hacker, what data to provide to break your code? Write tests for all of this.
11. When fixing a bug, write failing unit test first. When the bug is fixed, test should pass.
12. Don't mix a few checks in one unit test: use separated tests. Do not write iterators to generate tests.
13. Try hard to make tests that run fast. If one single test needs more than a few millisecond to run, development will be slowed down or the tests will not be run as often as desirable. In some cases, test can’t be fast because they need a complex data structure to work on, and this data structure must be loaded every time the test runs. Keep these heavier tests in a separate test suite that is run by some scheduled task, and run all other tests as often as needed.

References:

• https://python-guide.readthedocs.org/en/latest/writing/tests/
• http://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests
• https://github.com/howaboutwe/rspec-style-guide