Just finished reading Bil's excellent post on his frustrations with Continuous Integration (CI) in the VS 2005 world. My feelings about CI with VS 2003 and the nAnt, nUnit, Subversion, CC.Net combo are the same as his:
Visual Studio 2003 and Cruise Control.NET. Simple and elegant. A basic NAnt script to build the solution and you're good to go. Run NUnit, output goes to a format CC can understand and Bob's yer uncle. Let me quantify this. Our cruise server has a subversion client (command line) and the .NET 1.1 SDK. Visual Studio isn't installed because, duh, it's a server and cruise just needs something to build the system with.
CI in VS 2003 with CC.Net and friends was great. Everything ran smoothly for the most part. When you went to setup a project the choices of what to use for unit testing, build automation, and CI were pretty clear. Now we have more choices on how to run our projects and it isn't always clear which choice is the best. I've done projects using both sets of tools: VSTS for testing, source control, and CI as well as the old school nUnit, nAnt, subversion, CC.Net combo. The old school method seemed much simpler and easy to use/understand. But I figured maybe it was just because it was what I was used to. However:
Continuous Integration does not need to be this hard. CruiseControl.NET is an excellent tool and very flexible with new tools and integrating output from those tools. However when those tools require a million registry settings and even more DLLs (put in very specific places, trust me, you can't just toss them in the GAC and call it a day) and dump gobs of XML that no mere mortal (well maybe DonXml could) would be able to translate, it's just wrong. And as for the built-in Team Build that you could us, that's equally as useless as a) you can't schedule builds to trigger off of code checkins and b) again it requires a full Team Suite client to be installed on the server.
I feel Bil's pain. However, you can schedule builds to trigger off code checkins, but configuring these things isn't as straightforward as it could be. I didn't setup our last build environment (Jesse did), which had check-in triggered builds with email notifications, but I seem to remember it was pretty complicated and you didn't get the nice CC.Net type emails which showed you exactly what was going on with the build. You had to go look at the build on the server to see what happened.
So for now, if it is a small project, I tend toward the Subversion, nAnt, nUnit, CC.Net path, using MSBuild to build projects as needed (MSBuild is a must for things like building/deploying VSTS Database Pro projects). The big question for me will be what to use when my next big project fires up. I really want to use the MS tools and to become proficient with them, but I also don't want to be constantly fighting with my CI toolset.