So, one of my consistent gigs is working on “AnteSpam”:http://antespam.com/ for “Ironic Design”:http://ironicdesign.com/. We use “SpamAssassin”:http://spamassassin.apache.org/ as our engine, but we (well, mostly I) have built a bunch of infrastructure around it that allows us to do high-volume, redundant, high-availability deployment for domain customers, present held mail through a web interface, so on and so forth.
For the last 18 months or so, I’ve been embarked on a big rewrite, taking everything we’ve learned from having this system in production for the last three-and-a-half years and synthesizing it into a system that will run more accurately, more smoothly and with less maintenance and upkeep.
The rewrite is vastly superior in any number of ways. It increases performance by finding clever ways to avoid doing unnecessary work. It doesn’t move data around unnecessarily. The interface is largely ajax-based, making for better responsiveness. It changes its message handling to make it possible to do statistical learning on pristine copies of messages for better accuracy. It changes the way it represented various entities in terms of the data they stored for better accuracy. Really, tons of things are different, and it is truly kick-ass.
And about two weeks ago, I finally admitted that there was no way that the rewrite was going to see the light of day.
You see, the problem with starting from a clean slate is simply that of, “So, how do you make a transition.” And it was becoming increasingly clear that making a transition was going to be very, very hard. Nigh on impossible to manage in any reasonable time-frame. And some of the changes way down at the core were ones that we had no way to test under real loads, so it would only be as we started making transitions that we would know if they were going to work. So, really, I started having some real concerns months ago. And they just kept building and building.
And so I had my Fred Brooks moment, where I finally was able to admit that continuing down this path was going to be a mistake. Instead, the rewrite will, effectively, be declared a research project, and I’ll spend the next however long incorporating ideas–and probably even code–from it into the production system. So the grand new features will be introduced incrementally (and even doing that is scary enough in some ways), and we’re much less likely to end up getting ourselves up the creek without a paddle.
I’ve been sleeping a bit easier, even though there’s a fair bit of pressure to get this stuff rolled into the existing system. It’s just much more doable.