Posts Tagged ‘Test-driven development’

21 Comments Test-driven development with code generation - 03/5/09

As I’ve mentioned lately, I’ve been involved with some code generation lately. The software factory generates, on one side, production code, and on the other side, tests to verify the generated code. So that’s all good and well, but…

The software factory also generates a set of tests for each layer, that are there to give you a starting point. I’m having trouble with this, because of several reasons. Since the generated code is fully data-driven (based on stored procedures), the datalayer and its test are ok. They don’t need much adjustment. The tests are testing the way they should, and they are passing, and thus verifying the generation went well.

But the data layer is not the only one with generated tests. The Business layer and presentation layer also have generated tests. As I’ve already said, the generated tests provide developers with a starting point, but all the tests pass when they are generated (while they don’t test any logic). And that bothers me a lot. They should all fail! This may be a good system for developers doing TDD, but if you’ve got developers on your team that don’t do TDD, you’re doomed to have passing tests that don’t test anything useful or are empty. That’s why, in my opinion, all tests should fail after generation. That way the developer will be forced to fix them. And if I see an Assert.IsTrue(true); then I’ll just have a valid reason to kill ;-) .

It you have your SF generating failing tests (I mean only the tests that aren’t complete), it has it downsides too. For starters, it will take you a while to be completly green again. And as Beck says, the time you’re red should be minimal, for motivation reasons. Still, I prefer failing generated tests, since I’ve seen a lot of misuse (useless tests) when they’re passing. And if you can’t live with that, then don’t generate tests that are not finished. You could generate parts, for example: generate the test class itsself, containing a setup and a teardown method. If it’s obvious this code will need some mocking or stubbing, you can initialize a mockrepository in the setup method. But avoid tests that are just half-tests or no tests at all. They should be written by the developers. At least, you’ll have usefull tests.

Having passing useless tests just leads to a maintenance problem after a while, and you’re completely missing the safety net that your tests should provide. The test-safety-net can contain wholes, which you’ll find in bugs, but these are fixed, because you fix a bug by writing a new test. But the net should still be a net, and not some spider web that isn’t going to hold your weight when you fall.

What do you think about test generation? Let me know!

20 Comments Starting with Test-driven development - 02/26/09

I received some great how-to-start-with-TDD-advice from Davy a few months ago. I’m still a TDD-newbie, that’s for sure, but I sure have learned a lot during these past months. Since I think Davy’s advice gave me a great head start, I thought I’d share it with all the developers that want to start with TDD. Thanks to Davy!

Start small, VERY small

My problem when I began, was that I started waaaay too big. I was worrying about how to test complex scenario’s, even before I wrote the very simple tests, test-first.

When you’re just starting, try not to think about these problems just yet. Start out very simple, with code that’s easy to test and doesn’t even require more advanced techniques such as test doubles. Once you get to the more complex ones, you’ll already be a step further.
One rule I always keep in mind: if the code I want to write is so complex that I don’t even know how to test it, it’s time to split it up in smaller tests.

Read about it

You’ll find lots and lots of information on blogs out there, but I think you should start with Kent Beck’s Test-driven development by example (I’ll talk about this book later). I’m not saying the info on blogs isn’t valuable, I’m just saying there are better resources out there to learn the basics. Reading blogs about TDD without knowing the basics well, caused me a lot of confusion because of several reasons:
First, because TDD-posts mostly assume some level of TDD-knowledge, and usually more than knowing the red-green-refactor cycle. Second, because at the end of the road, posts are personal opinions and personal perceptions, which aren’t always completely correct, so you’ll find contradictory opinions and won’t know which one is correct. Once you understand the TDD-basics, you’ll get much more value out of all the information TDD’ers are sharing with us on the net.

TDD by example is a great book to get started. It’s about 200 pages long and it’s extremely easy to read. Once you’ve read it, you’ll be able to start practicing it, talking about it and understanding it’s value. What I also enjoyed a lot, is that Kent writes the book just like you’re thinking now (a not TDD-mentality) => “Let’s implement this. We should call this method, and then perform a calculation… Shame on me! I should start test-first!”. I loved it!

After this one, you can read xUnit Test patterns, which is a more advanced book that describes the patterns you’ll commonly use during testing. I’ve just started to read it (finally) and I’m already enjoying it. This is not going to be a quick read (900 pages), but I know it’ll be very valuable.

Apart from the books, I’d also recommend to read Martin Fowler’s paper about the difference between mocks and stubs, which you can find here. It’s a quick read and it’s very clarifying.

Don’t give up

It won’t be easy, it’s a complete mind-shift. I’ve been in the situation a lot of times that I began test-first, and ended up writing everything test-after. The clue is not giving up, or as Davy might say, be stubborn! The advantage I’ve had is, that I felt guilty whenever I was testing-after, so whenever I got to new functionality, I began test-first again.
Even now, when I’ve just implemented something I’ve tested and I’m green again, I find myself starting to type some new method somewhere. At that point, I try to be very strict with myself, delete what I just typed, and get back to my test suite and write a new test. Remember red-green-refactor :) !

Apart from discipline, I think the easiest way to hold on when learning TDD, is having an experienced TDD’er in your direct environment. But that’s just not always possible. When I got into TDD, I was working with people that don’t saw any value in testing at all (not even in tests-after-coding), so that shouldn’t stop you!

A final word

In my limited and humble experience, I can say that I’ve only experienced advantages from the TDD-approach. It may be the hard way in the beginning, when you’re learning it (and I still havn’t completely passed that stage), but the value you get out of it (less bugs, reliability, clear design, more focused and clean code) isn’t representative to the time you invest in it.

10 Comments How do you know your tests are good? - 01/26/09

I keep asking myself, how do you know your test suite is healthy? I mean:
- How do you know your tests are testing what they should be testing?
- How do you know your tests aren’t testing what they shouldn’t be testing?

You could use code coverage as a metric. But that’s not going to get you all the way. Code coverage is especially handy for people that are doing the testing, but not the TDD-way. Just run with NCover, and you’ll immediately know what you forgot to test.
But still, code coverage isn’t smart enough to know if what you’re testing is actually meaningfull. Let me simplify this with an example.

We have a very simple method, that calculates the age of a person, based on his/her birthday, and today’s date (I’m not going to involve correct calculation, nor validation, since it’s not the scope of this example):

   1: public class AgeCalculator
   2: {
   3:     /// <summary>
   4:     /// Calculates the age based on a given birthday and the current date
   5:     /// </summary>
   6:     /// <param name="birthday">Birthday to base age-calculation on</param>
   7:     /// <returns></returns>
   8:     public int CalculateAge(DateTime birthday)
   9:     {
  10:         return DateTime.Now.Year - birthday.Year;
  11:     }
  12: }

Then we have useless test number one:

   1: [Test]
   2: public void TestThatCoversAgeCalculationButIsNotTestingIt()
   3: {
   4:     AgeCalculator calculator = new AgeCalculator();
   5:     int age = calculator.CalculateAge(DateTime.Now.AddYears(-5));
   6:     CanDriveSpecification canDriveSpecification = new CanDriveSpecification();
   7:     Assert.IsFalse(canDriveSpecification.IsSatisfiedBy(age));
   8: }

Here’s it’s coverage:


NCover is telling me that this test (I used TestDriven.Net to run it with coverage) has 100% coverage. But taking a second look at the test, you’ll see (if you haven’t already) that this test is really testing something else, it’s testing if someone with age 5 may drive or not, which doesn’t ensure that our age calculation is correct. If that’s the only test you have covering your age calculation, your tests havn’t functionally covered it.

The second useless test:

   1: [Test]
   2: public void TestThatCoversAgeCalculationsButTestsItWrong()
   3: {
   4:     AgeCalculator calculator = new AgeCalculator();
   5:     DateTime birthDay = new DateTime(1984, 10, 31);
   6:     Assert.AreEqual(24, calculator.CalculateAge(birthDay));
   7: }

This is a test dedicated only to the age calculation, so we’re getting closer. Still, this test is just wrong! This will run today, tomorrow, even next month, but forget about it in november! And I’m still getting my 100% coverage.

The two tests above will run. They will even provide you with a 100% code coverage for the CalculateAge method, but they are not at all representative. So code coverage is not even close to measuring the quality of our test suite.

How can you ensure the tests are good?

You will always need to keep a human eye on the effectiveness of your tests. It’s the developer’s responsibility to be testing something usefull.

That brings me to the following conclusion: we can distinguish two different kind of developers that write tests:
1) developers that write tests as part of their duty (because management says so)
2) developers that write tests to ensure quality of their work, even without management asking

You’ll be thinking, that the code I showed you above is just plain stupidity, and can’t be written by any developer. Well, I’m sorry to dissapoint you. Maybe in this example I exagerated a bit, but I’m sorry to say, that this type of test-code is actually very common. Developers that don’t care about testing, write this type of code, and not because they’re too dumb to see what’s wrong with it, but just because they don’t care about the tests.

That’s what leads me to test-driven development.

Test-driven development is a technique that requires developers to first write their tests, and then write code to make the tests pass. This all is done is an iteration known as Red-Green-Refactor.

1) Think => how should you write your test?
2) Red => write your test, and see it fail (since there is no implementation code)
3) Green => write production code to make your test run
4) Refactor => refactor both test- and production code
5) Repeat => Do it all again

James Shore does a great job covering this topic in more depth, so don’t forget to take a look!

Revisiting the AgeCalculator example

How could I best write my test?

I need to calculate the age of a person, based on his/her birthday, and the current date. So it would be nice to call a method CalculateAge on a calculation class named AgeCalculator.



I took a screenshot, just to show you that this won’t even build (since this class and method don’t even exist)!


Create an interface IAgeCalculator with a CalculateAge method. This method should accept a DateTime parameter and return an int. Create a class AgeCalculator that implements the interface.

   1: public interface IAgeCalculator
   2: {
   3:     /// <summary>
   4:     /// Calculates the age based on a given birthday and the current date
   5:     /// </summary>
   6:     /// <param name="birthday">Birthday to base age-calculation on</param>
   7:     /// <returns></returns>
   8:     int CalculateAge(DateTime birthday);
   9: }
  11: public class AgeCalculator : IAgeCalculator
  12: {
  13:     /// <summary>
  14:     /// Calculates the age based on a given birthday and the current date
  15:     /// </summary>
  16:     /// <param name="birthday">Birthday to base age-calculation on</param>
  17:     /// <returns></returns>
  18:     public int CalculateAge(DateTime birthday)
  19:     {
  20:         return DateTime.Now.Year - birthday.Year;
  21:     }
  22: }


Maybe in this simple case, you wouldn’t want to refactor your code. But imagine, your AgeCalculator should be able to block invalid birthdays. An invalid birthday could be a date in the future. First you’d write a test that should throw an exception when you’re passing a future date. Then adjust the AgeCalculator class. We’re instantiating the AgeCalculator two times now. You could now choose to instantiate your AgeCalculator class in your test-setup. Then, we’re asked to add a new feature, in which you should be able to pass a date to calculate the age against (in stead of DateTime.Now).
We implement it the RGR-way and it looks as follows:

   1: /// <summary>
   2: /// Calculates the age given a birthday and a date to calculate the actual age against
   3: /// </summary>
   4: /// <param name="birthDay">Birthday to base age-calculation on</param>
   5: /// <param name="compareDate">Date to calculate age against</param>
   6: /// <returns></returns>
   7: public int CalculateAge(DateTime birthDay, DateTime compareDate)
   8: {
   9:     return compareDate.Year - birthDay.Year;
  10: }

Then you’ll notice you could adjust the CalculateAge method that only accepts the birthdate. You could have that method call the CalculateAge(DateTime birthDate, DateTime compareDate) and pass it DateTime.Now, to avoid code duplication (remember the DRY principle?). So, as you can already see, the refactoring step never ends. We’ve already added 2 new features, and we’re still refactoring feature number one :-) .


I’m not at all a TDD-expert (I wish!), this post only reflects how I got interested in TDD. Right now, I try to practice it whenever I can, but I have a very long way to go. I’m currently also reading the TDD-starter book: Test-driven development by example.

I think that in this little example, I proved that TDD helps to keep your tests focused, clear and meaningfull. It also follows the YAGNI principle, since you’ll never be writing code you havn’t written a test for, thus you won’t be writing code you don’t need, unless you’re writing tests you don’t need, and then you’re doing TDD all wrong :) .

TDD can only be done by developers that actually have interest in building qualitative software (if you don’t care, I don’t think you can find the discipline to do it). Just imagine the first useless test I wrote above failing because the calculation was changed and the developer introduced a bug. That will keep the team looking for bugs in the specification, while it’s the calculation that went wrong. And what about the second useless test? That one just won’t run in a few months. And you can start looking for the bug again. In such a case, it’s better not to have the tests at all.

Last but not least

James Shore already linked to the rules to follow in TDD in his Red-Green-Refactor post, but if you havn’t read it yet, then do it now. I’m just repeating the rules here because I think they are very valuable.

Quoting Michael Feathers:
A test is not a unit test if:
1) It talks to the database
2) It communicates across the network
3) It touches the file system
4) It can’t run correctly at the same time as any of your other unit tests
5) You have to do special things to your environment (such as editing config files) to run it.