Testing your patience

Marvelous maven Marcelo asks:

"Any recommendation/best practices to organize modules, and the respective test cases for them?"

Great question.

There are two schools of thought, for how to organize test code:

1) Embedded Tests

You have a source tree, with your Python code organized in different directories and sub-directories.

Modules and sub-modules.

At each level, in the exact same directory, you create test module. Containing the unit test cases.

So for a directory (module) tree like:

polls/__init__.py
polls/widgets.py
polls/ab_split_testing/__init__.py
quizzes/__init__.py
quizzes/video.py
quizzes/randomization/__init__.py
quizzes/grading.py

You'll have test files named:

test_polls.py
polls/test_widgets.py
polls/test_ab_split_testing.py
test_quizzes.py
quizzes/test_video.py
quizzes/test_randomization.py
quizzes/test_grading.py

There are other ways to do this, but this is the idea.

Before explaining the proz and conz of doing it this way, let me tell you about the other choice:

2) Parallel Hierarchy

Here, you'll have a top-level directory, holding the application source code. Named, often, "src/". Like this:

src/polls/__init__.py
src/polls/widgets.py
src/polls/ab_split_testing/__init__.py
src/quizzes/__init__.py
src/quizzes/video.py
src/quizzes/randomization/__init__.py
src/quizzes/grading.py

And also at the top level, you'll have a "tests" directory. And it's where you stuff ALL your test code. With a tree of subdirectories that matches what's under src. Like this:

tests/test_polls.py
tests/polls/test_widgets.py
tests/polls/test_ab_split_testing.py
tests/test_quizzes.py
tests/quizzes/test_video.py
tests/quizzes/test_randomization.py
tests/quizzes/test_grading.py

See how this works? These two approaches have their cons and pros:

1) With the "embedded test" approach:

• You can always find the test cases matching a particular (sub)module. No guessing where it should go.
• And: if you rearrange how your code is organized, the tests move to the right place, almost automatically.
• But: When you deploy code to production, or ship it to a customer... you probably don't want to distribute the test code too. It won't do any good, and just might cause a problem, so you have to carefully strip it out. Without breaking anything, hopefully.

2) With the "parallel hierarchy" approach:

• You have a clean separation of application and test code. Deploying and distribution is robust and easy.
• And: you can be more explicit about where you point the test runner, and where tests are stored, which has some advantages for keeping code organized.
• But: you're in a constant, labor-intensive battle to maintain a test code organization that matches whatever the current layout is for the code being tested. Drop your guard for one second, and things can get confusingly out of sync, increasing the maintenance burden.

I've worked on plenty of projects where I've used both approaches, and both will work fine. They just have different trade-offs, and you may prefer one or the other.

Just don't MIX them. For the love of Guido, pick one of these two, and stick with it.