I just finished watching Writing Code That Doesn't Suck by Yehuda Katz. I think he also could have called it "Writing Tests That Don't Suck". I'm going to write a summary of some of his points (along with some of my own notes). He covers a lot of the things I've been thinking about over the last couple months as a Python coder coding in Rails using behavioral driven development (BDD).
Most Rails programmers agree that tests are good. However, we don't spend enough time considering which tests are the most valuable. In fact, Yehuda points out that a lot of tests are quite useless.
Unit tested code doesn't mean bug free code. In my own experience, I'm often so tired after writing automated tests that I don't have enough mental energy to do solid exploratory testing. Having someone other than the programmer do some serious exploratory testing is necessary.
alias_method_chain is bad. Having a chain of modules monkeypatching the same method leads to a brittle modularity nightmare that is unmaintainable and unreausable. It's also a sign of a poorly designed API. Just because Ruby lets you do things like that doesn't mean we should be content with poorly designed APIs.
Matz specifically doesn't want to add interfaces to Ruby. However, Yehuda argues that interface-oriented design is a very good thing. Yehuda reminds us that everywhere other than Ruby, coders know that good design means coding to an interface, not to an implementation.
You might have code and unit tests that test that code. You might refactor them both together, and all your tests might pass. However, if you break the API to your code, you've broken everyone else's code that is reliant on that interface. That's why regression testing is important. It doesn't matter how the implementation works--the only thing that matters is that your interface continues to behave as promised.
That's why testing the external interface is so much more important than unit testing the internal implementation. Heavily mocked and stubbed unit tests just aren't as useful as full-stack testing (with, say, Cucumber and Webrat). I personally think there's nothing worse than a controller, model, and a view that are tested individually using mocks and stubs, but which blow up when combined. That's not to say that unit testing isn't useful--it's just that what I would call integration testing or what Yehuda would call regression testing is even more useful. Test what you actually care about--i.e. the external behavior.
"some_object.should respond_to(:some_method)" is not a useful test. If you're not testing what some_method actually does, then why bother, and if you are testing what some_method actually does, then you don't need to test for its existence. I think that "some_object.should respond_to(:some_method)" is sort of like sunbathing--it makes you feel better about yourself and (arguably) makes you look good, but it's not helpful for your overall health and well being.
At the risk of beating a dead horse, when I see tests like "some_object.should respond_to(:some_method)", it really makes me wish I were coding in Haskell. Haskell's type system can enforce interfaces like this without writing boring, do nothing tests. Perhaps that's why so many Ruby coders have switched to Scala.
Yehuta does think that TDD with unit tests is helpful for designing your code, but those tests aren't good as regression tests. After your code is working, then you should write regression tests that cover the things that you actually care about, not the irrelevant implementation details.
Yehuda doesn't have a computer science background. Hence, he is relearning a lot of the lessons that MIT's Structure and Interpretation of Computer Programs has been teaching for decades, and I think that's good. It seems like much of the Rails world has forgotten a lot of the things we've learned about modularity, encapsulation, and good design over the last 30 years. Hopefully, thanks to Yehuda and Merb, that will improve.
It will be interesting to watch the synthesis of good design as taught by SICP with a strong commitment to testing that is popular in the Rails world.