Testing shortcuts

Writing test takes time, so we might want to cut the corners a bit to go faster. But this can lead to completely changing the behaviour of the test.

Let's say we're writing a class to manage a collection of characters in a movie. We want an add method to add characters to the collection so we can retrieve them later (with a get method maybe). First things first, we write the test:

Looks great! After a character is added, the cast's size would indeed be 3, so if it's 3, we're good. We move on to implementing the class:

Test passes... Except the class, implemented as it is, doesn't really 'add a character to the cast'. It just increments the cast's size. Which means our tests are OK, but our app won't behave as intended. So much for this test your code mantra :p

This happens because we're trying to use a short-cut in our test. Testing that the list actually contains the new character looks like a big thing. So we go for something smaller we think has the same meaning: checking that the size of the list has incremented.

And there are plenty of short-cuts we can take in tests:

• just testing the size of a list instead of its content;
• just testing something is undefined or null;
• just testing the type of an object;
• just testing the status and content-type of an HTTP response;
• ...

We need to become aware of when we use such short-cuts. When we realize we do, ask ourselves if they are really necessary (which they might be). Because when we use those short-cuts, we leave gaps in our tests, openings for bugs to show up.