9 May 2006 titus   » (Journeyer)

For future reference

The slamd distributed load generation engine.

mocks considered harmful?

robertc

asks if mock objects lead to interface skew.

I could probably go off on a long polemic here, but let's see if I can keep it short... Basically, if all you're doing is testing with mocks, then the answer is probably that you're going to screw yourself up.

But let's generalize the question, to see if perhaps there's a deeper truth:

Dear Mr. Touchstone,

I'm using technique X for testing, and I'm only doing unit testing (...or functional testing, or acceptance testing, or integration testing, or regression testing, or UI testing, or smoke testing, or fizzlebar testing...) I'm worried that this is going to cause problems in the future. What should I do?

-- Anxious in Albuquerque

...and my answer:

Dear Anxious,

I think you should consider testing at additional levels. Generally unit testing is not sufficient, because it purposely tests only small, largely independent units of the code. Functional and smoke tests (as in "push button -- does smoke appear?") should be de rigeur for any project; IMO they're as or more important than unit tests.

So that's my answer: there are bigger problems lying in wait! Mock objects are a specific solution to a fairly general problem with unit tests: you can mimic external interfaces that need to interact with your code in fairly stereotyped ways. That stereotyping is a specific weakness of the approach, as well as being the strength that led you there in the first place.

More generally, any time you find yourself relying completely on your unit tests, you're going to be in trouble. Unit tests are a great programmer tool, and they are very useful, but they are not everything.

Someone interviewed me

Pythonthreads just posted a lengthy interview with me. It's been a month or so since I answered the questions, and I'm still fairly happy with my answers!

Bugs

I read an interesting post a few days ago that discussed the dismal situation with bugs in Linux packages: basically, there's been a proliferation of bugzillas, and you can often no longer have the conversation necessary to fix a bug in any one of the bugzilla sites.

It occurs to me that one possible solution is to build a test case that demonstrates the bug; that's sort of the maximally portable bit of problem documentation necessary, no? Then just post that to the most upstream bugzilla...

--titus

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!