Tuesday, November 17, 2009

Why I don't test

Three words and one contraction and the entire development community probably hates me. Well, almost the entire community (I hope to find a few developers that share the my sentiment).



I work for a (not-so-much-anymore) early-stage startup and we have more abandoned ideas than successful ones. This is indicative of many companies. You have to throw a lot of spaghetti against the wall to see what sticks. This is rarely the environment for BDD, TDD, TAFT or perfectionists. This is where you write a couple of functional tests and release your code into the wild. The words "Alpha" & "Beta" are dear to my heart.



Startups are the entrepreneurial spirit at its best. Even if you are smart, motivated and hard working, this environment is largely a numbers game. The more ideas you actualize, the more are given a chance to take hold. The 80/20 rule is in full effect and great code doesn't matter if no one uses it.



As a developer, ask yourself, "How many cool side projects never made it past the first couple days of coding?". I guarantee all of you have at least one and most of you have a lot more. Now, ask yourself, "What is the main reason you didn't get further in your project?". I also guarntee that the most common reason is getting bogged down in the minutia (like testing). If you would have taken the quickest path to throwing up something that worked most of the time, you could have discovered if your idea was successful.



Not testing isn't as crazy as it sounds. If an idea sticks, then you need to build out a test suite as you handle edge cases and add further functionality. On the other hand, odds are, what you are coding right now isn't going to be around 6 months from now. So, don't waste your time.



I once had to write an IMAP client coded to rfc spec in less than a week. Their is no way I could have done this while writing tests the whole way through. The requirements changed constantly and I would have had to mock an entire IMAP server first. (Fortunately, this idea gained in popularity and now has great test coverage.)



Code is a means to an end and not an end in itself. Tests support the code we write, proving it works well. If your tests don't allow you to code faster then they are failing (if you work for a startup). Here, we write code to prove ideas work. This is why Ruby on Rails has become so popular; it's ability to rapidly prototype (aka quickly birth a concept).




Their are many environments where this attitude doesn't work and wouldn't even make sense (contractors, opensource contributions, large companies, proven models, etc, etc), but TAFT is wrong. Their are no hard and fast rules, but if their is one, I would suggest it is this: T.O.S. (Test On Success).



3 comments:

  1. boo!

    First of all it is TATFT :)

    And second, test first isn't just something you can do. It is a skill like riding a bike or rubbing one out with your off hand. What I'm saying is you have to practice. The Katas that have been circulating around our community lately are a great place to start. (http://www.viddler.com/explore/GreggPollack/videos/29) After you get good at those, then you'll realize something... test first is actually easier than the other way around.

    As for your imap example, you didn't really have to mock. If you were short on time, why didn't you just set up a dovecot server locally and write code against that. As time permitted you could have worked on a test suite for an imap server.

    Or you could just ignore what I said.

    Testing is hard. Lets go shopping!

    ReplyDelete
  2. First, thanks for the feedback! I was expecting some replies like this and am glad to see them come in.

    Second, oops, yepp, it is TATFT, I guess I should have written a unit test for that. ;)

    Thirdly, can you substantiate your TDD claims?

    Fourthly, requiring all developers to set up a dovecot server just to be able to run the test suite? That sounds like a quick path to having a bloated unmaintainable tests. Especially if it had gone the other route and had been silently abandoned 3 months later.

    Lastly, if you go shopping, my birthday is November 29th http://bit.ly/WX3P

    ReplyDelete
  3. I agree with your sentiment.

    A minor deviation I like is: If it takes a Mock-object, it's too complicated to test right now. If it can be tested using just simple types, it's worth AT LEAST a few basic tests to prove it isn't whack (with the idea of fleshing out later if, as you say, the code sticks).

    Reason: These types of test take very little time to write, but often catch dumb little things that might otherwise take MORE time to debug on the spot -- even "debug" in the sense of "just get it basically up and running."

    I also admit that the number of methods which are so basic isn't large! So this still means "don't test a lot."

    Great examples of this: Custom validation, regexes, string manipulation, parsing.

    ReplyDelete