Essential Versus Accidental Tasks

posted on 10/13/09 at 08:00:00 pm by Joel Ross

I finished reading Mythical Man Month a while ago. [Actually, it was a LONG while ago, but this post has been in my draft list for well over a year!] Overall, I thought it was a good book, but not as good as I expected. I think part of it was the amount of praise the book receives, and part was how good the first few chapters were - they left an expectation for the rest of the book, and it just didn't seem to live up.

Anyway, one of the chapters (No Silver Bullets, I think) talks about whether software development techniques can get an order of magnitude increase in efficiency, and lays out development tasks in two categories:

  1. Essential Tasks: The tasks that are directly related to building the software. Basically, writing code.
  2. Accidental Tasks: The tasks that you have to do to build software, but aren't technically part of the software.

The essay goes on to talk about how accidental tasks would have to be 90% of development time in order to get an order of magnitude reduction development time if you could remove them all, and how that's unreasonable. As a result, you have to attack the essence of software development if you want to achieve those types of results.

It's a logical argument. And while eliminating (or greatly reducing) the accidental tasks won't yield an order of magnitude speed up in development, it doesn't take a huge effort to reduce the accidental tasks either. Before you can determine if it's worth reducing the accidental tasks though, you need to know how much time they actually take. If they only take 5% of your time, any effort towards reduction will seem wasteful with respect to the return. But if the accidental tasks start to approach 1/4 to 1/2 of your time, then reducing that will yield much larger returns.

Before we can answer how much time accidental tasks take, we should probably define what the accidental tasks are. Some of these may seem controversial, but remember, accidental tasks are anything that's not essential to the production software.

  • The build process: On a lot of projects, this can be a major time sink. I've seen projects that do a weekly integration cycle and it can take one person a few hours every week just to get the software built and deployed. Even automated builds can be a time sink, both with the time to set up and maintain the build, but also the ongoing build tax - think "Check-in dance."
  • Environment setup: Creating a development environment takes time. This includes setting up source control, getting the right people access, installing the necessary software and tools, setting up a QA/Test environment, etc. All of the "infrastructure"-type tasks.
  • Testing: This'll probably be the most controversial task on the list. But the reality is that manual testing takes time. Writing automated tests takes time, and while they have great value in creating correct code (and solid design), the tests themselves aren't production code. Plainly, tests aren't deployed.
  • Project Administration: Status meetings, time tracking, billing, code reviews, etc. Important pieces, to be sure. But not essential to the software.
  • Bug Fixes: This one's a bit different, because this code actually does make it into production. But I'm including it here because bug fixes aren't essential to the advancement of new features - they're flaws in features already completed. Had it been done correctly to start with, this time wouldn't be necessary.

I'm sure there are other components I'm missing, but those would be the major "accidental" tasks I see. The first two are basically setup costs. They are essentially a "one time fee". And they're roughly the same for any new projects, big or small. That causes a bit of a conundrum. On large or long running projects, the savings (percentage-wise) is small, so the incentive to figure out a way to reduce it is small. On smaller projects, the savings (percentage-wise) is bigger, and so is the incentive to automate it away. What isn't there is the extra time needed to figure out how to automate those tasks away. If you rarely start new projects, then reducing setup costs isn't a big deal.

So, that really leaves the last three as ways to get significant gains by reducing accidental tasks. They're all repeating tasks, and could easily account for 25% of a project team's time, and that doesn't include the time lost to context switching. Moving to a leaner project management approach will help reduce project administration, but for most organizations, this is a fixed item without a lot of flexibility. If you're consulting, you need to track hours and project status so you can get paid. You can optimize it, but most likely, it's been optimized if the company has been around for any significant time.

So that leaves testing and bug fixes. Ironically, increasing one will reduce the other. The more time you spend testing, the less bugs you'll have. But manual testing takes a lot of time, so the most efficient use of a team's time is looking for ways to automate the testing process. Things like unit tests, integration tests, functional tests, UI tests - anything that can be automatically run in a repeatable fashion - this is where you can gain some real benefits. The payoff: you'll spend a lot less time fixing bugs, which means your team is moving forward, not looking back. Also of note, investing time into automated testing will make your software easier and safer to modify over time, and will actually reduce the time it takes to do the essential tasks.

I took the long way to get here, but the bottom line is this: Increasing one aspect of an accidental task will decrease other aspects of accidental tasks, as well as streamlining essential tasks. Sounds good to me!

Categories: Development, Software

3 comments »


 

3 comments

Comment from: Dan Hounshell [Visitor] Email · http://danhounshell.com
Nice post Joel, but I think you simplified one piece of the puzzle a little too much: "The more time you spend testing, the less bugs you'll have."

I can tell you from personal experience that this statement may not be correct in every situation, it certainly is not correct in our case, and it may be completely incorrect in all cases.

We recently invested a lot of time and resources into revamping our testing procedures, both automated and manual. I can tell you that in our case it is more like "The more time you spend testing, the more bugs you'll have. Way more. Way, way more." Like maybe a 200% increase in your bug backlog size for each 100% increase in time spent testing.

Now these are "good" bugs not "bad" ones - they were found before release and were fixed as part of the development cycle rather than being found in production and fixed later in the maintenance cycle.

It depends on where you want to take the hit for bugs found in production. If you count the hit vs. your development timeline (I assume as most consultants/service companies would - i.e. the job is not finished until the client is happy) then yes, you may end up with a shorter overall timeline with more/better testing. It is much easier/quicker/less expensive to fix a pre-release bug than the same one found afterward. Think about all the time spent by support, reporting, trying to replicate it, finding the cause, fixing it, redeploying, etc.

But if maintenance and post-release bugs are not considered part of the development cycle then you can greatly decrease the project timeline by doing less (or no) testing. Less testing = less bugs (in development). Of course you'll end up shipping crappy, buggy software.

My experience does support the point that I think you were trying to make, though. More testing will decrease the amount of bugs you have to fix in maintenance. However, I don't think more/better testing necessarily decreases the development cycle time - it's probably a wash at best.

Short story - it takes time and resources to build good quality software, there are no shortcuts. There are steps that you can take to increase quality. There are steps that you can take to decrease the timeline, but as you mentioned they all require some type of time/resource investment somewhere. Some of them may be worth it for some projects, in other projects they may not be. Implement the ones that are appropriate for your project and your team. There are no magic bullets.
10/14/09 @ 08:41
Comment from: Joel Ross [Member] Email · http://www.rosscode.com
Dan,

Thanks for the thoughtful reply. You make some great points.

First, I agree with you - going back and adding in tests after the software is built will definitely increase the amount of bugs you find. And you'll waste a lot of cycles fixing those bugs. Or, worse, your users will start reporting the bugs for you, which will take more time. I'll ignore the argument about not testing at all, because I don't see how you can't count the time spent fixing bugs for released software as part of the development cycle. For consultants, getting the project complete typically includes a warranty period. For software product companies, fixing bugs prevents the team from moving forward.

But my point, which obviously didn't come through clearly, is that writing automated tests as you write the software will reduce the bugs found later. Maybe this is more applicable to new proejcts, but once you get into the routine of testing as you write the software, the number of bugs found will go down. Or rather, the number of bugs found by someone other than the developer as they're writing the software will go down. Bugs found by the developer as they're developing a feature are far cheaper and faster to fix than bugs found by a QA team or a user. Over time, when you have a solid set of unit and functional tests, the number of bugs found in production will go down.
10/14/09 @ 15:22
Comment from: yves saint laurent [Visitor] Email · http://www.bagsus.com/yves-saint-laurent.html
http://www.bagsus.com
09/08/10 @ 03:00

Leave a comment


Your email address will not be revealed on this site.

Your URL will be displayed.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)