Every day development and unit testing
What this post is NOT about: This post does not discuss how to do unit testing or any tools that help with unit testing code.
This is about the experience i had in regular development activities and it may help others facing the same kind of problems.
In our RoR app, we found a serious bug, which probably appeared because of some thing that a plugin called AttachmentFu does, which didn’t play well with some active record enhancements we have done to accomplish lazy persistence(keeping active record from pushing state to the DB).
While code-pairing on this bug, my pair and myself had a major difference in our opinion. Wether to take the ‘Test First’ approach or not.
My pair did not have any exposure to functional languages and i failed to explain her why i favored taking pain to use logging and debugging and not spend time writing a failing test. Finally i decided to let her write a failing test to help her realize the approach does not work when the problematic area is not identified. She tried a lot to mimic the scenario in a test but the test kept passing. Eventually she couldn’t write a failing test, but probably ended up wasting half and hour if not more.
In functional world she had a decent probability of getting the test right…. but with imperative language and all the side effects… she really didn’t stand a chance. It was actually a pity to see her attempt.
Deciding between Test First and Test Last:
Of-course Test First…. Well.. NOT always…. Most of the time it makes sense to write a failing test to fix a bug/add a feature and let the test drive the development. But there are times when we see problems with framework code, way to deep in the stack or corner cases… I don’t think it makes sense to write a failing test for that….
When the problem is somewhere deep down the stack and the reason are not fully understood, holding on to our TDD guns just make us waste more time… For less understood corner cases like these, its extremely hard to be able to imagine and take care of all the possible side effects of all the methods in the call stack and get the test to fail for the exact same reason that is causing the bug. To get the failing test in scenarios like this one is almost like searching a needle in a hay stack. The probability of finding one is so bleak and the time needed for such an exercise is so much, that it makes sense to give up upfront.
Stop write tests????
NO… in these cases it makes more sense to use logging and debugging to figure out where the real problem is and write a failing test for the broken piece of code(which specifically tests that piece). It saves the trouble of writing a unit test for the outer most method(which may be some 30 calls away from the problematic code) and trying to permute all possible scenarios to fail it.
The thirty calls away approach may work with a pure functional language, but definitely not with imperative languages.