Estimating project time

In my last blog entry I mentioned a couple tools I use for tracking time. Today a forum posting mentioned the difficulty of doing accurate project estimates. I’ve made mistakes on both sides of the estimation game, and documented a couple of them here. I’ve been doing this for years but I confess I still need to work on some of these points.

There are books, magazines, newsletters, and websites dedicated to project management and tips for consultants and employees who need to account for time. I won’t pretend to summarize the expertise from those media but I can point to areas that I know are often overlooked.

Do’s and Don’ts

1) Detail all time spent because at any minute activity can digress down a dark path and people will be asking how we got there.

2) Advise the project manager (client) of all progress and any activity that detracted from the goal – especially if this wasn’t included in the estimate. Keep communication channels open for frequent updates and everyone should watch for problems.

3) Bill early and bill often. People don’t like getting invoices for effort invested weeks or months in the past, and some people will not pay for some or all of it.

4) Don’t under-estimate the time consumed in normal communications like meetings, phone calls, and especially emails. It takes a couple minutes to digest an email, and "some amount of time" to respond to inquiries or provide detailed updates. You can estimate anywhere from an average of 3 minutes per email to 7, with most responses being quick but some being more detailed and consuming anywhere from 15 to 30 minutes to get "right". There is no one-size-fits-all estimate for communications time. There are different kinds of communications which can be broken down into possibly three categories; Yes/No or response not required, short explanation, detailed information. But the average of all types will probably come down to 3-7 minutes. When the client is shocked about a timesheet that includes 10+ hours of communications for an intense week, you better have detail for it – and then discuss ways to keep essential information flowing without consuming so much time. Make better use of meetings and don’t hesitate to try to move meetings along to keep them quick and efficient, compared to some that drag on and little is accomplished. Life is too short to spend in unproductive meetings.

5) We are usually called upon to work on projects that are mostly within our realm of expertise, but many projects require some piece that is just outside of our comfort zone. These efforts require research. Research takes time. Working with new technology takes time. Understanding the cause and remedy for mysterious errors takes time. The time spent on such activity can easily go to hours or days, yet once we are familiar with the material it may take minutes to accomplish the same task. Unfortunately our estimates often lump everything together and assume that we are familiar with most/all pieces required. Again, there are different kinds of time that need to be estimated, it’s not one-size-fits-all. Just one of these research events can put a project days behind schedule, and depending on the proficiency of the developer, the reality is that some projects will include a series of these, completely throwing the estimate and actual project time out of alignment.

6) I was asked to work on a project that involved using technology X, which is one of my specialties. As the project went on it became apparent that the client was expecting technology X to do things that I knew it wasn’t suited to do, but since all of the discussion was about this technology I didn’t step up to say X was the wrong technology for the long-term effort. I spent a couple hundred hours trying to bully X into doing what was much better suited for Y. The client would have been much happier with Y in the long run. Don’t let people tell you what technology you’re going to use to accomplish specific tasks, because they frequently don’t know what other technologies exist or that something else might be better for a specific task. The "we want X" request might lead back to something they read in a magazine and just because you know X doesn’t mean it should be used whenever someone asks for it.

Quotes vs Estimates

I never give quotes because they always come back to "you said it would take this much time, and that was the agreement so that’s what we’re going to pay you." People confuse the words "quote" and "estimate". I’ve had discussions where everytime I used the word "estimate" it was echoed back as "quote" and I needed to clarify the vocabulary before moving on with the discussion. For the reasons mentioned above it’s sometimes suicidal to quote a project. When working with people or technology for the first time we are subject to many of the issues above. The more familiar we are with people, and application, and the technology in use, the better we can estimate projects – even perhaps to a point of providing a reasonable quote. I have a two-page document that I wrote for new clients that explains why we estimate and why we don’t quote. A couple of our colleagues have adopted this document for their own use and I’ll be happy to provide it to anyone who asks for their own use.

If you’re going to quote, quote small segments of a project because that allows you some successes among some failures, rather than flipping a coin to see if an overall project will be a success of failure in terms of the time vs original quote.

Tracking the target and progress

I won’t get into this too far because people use all sorts of tools from simple calendars to complex software, but there are some common tools that should be considered for every project. Look into tools that allow you to break down projects into tasks so that you can estimate and track activity at a very detailed level. One tool I use is open source freeware called ToDoList. I highly recommend it. I’ve coupled that with the Work Break Timer and Gantt Project, see my last blog for links. I’ll probably look for something soon for PERT charts too. (Here is an interesting page on project management that I found recently – it discusses PERT and Gantt and other related concepts, and echoes some of the points mentioned here.)

One of the nice things about a Gantt chart for internal or personal purposes is that you may be able to visually map all of your open projects to see how activity on one project will affect or be affected by other completely unrelated projects. In this weird "feast or famine" business I find myself trying to cram a lot of projects into a short amount of time because everyone assumes that a project will be started "right now" and finished "real soon". People assume that just because you’re talking about a project now that you’re ready to start on it tomorrow. Well, sometimes you need to rise to the occasion and a Gantt chart helps to see how projects can be juggled on short notice.

Another good thing about Gantt charts is that they clearly show the difference between "it will take a couple days to do this" and "you’ll have results in a couple days". It may take 3 weeks to put in 8 hours on a project. If you tell someone a project will take 8 hours they expect it tomorrow. If it’s Friday, they may expect it Monday. If you work project time into actual time which includes other projects and activities, you may find that it’s difficult to deliver any given 8 hour project in less than 3 days. Right now I’m struggling with the concept of telling a prospect something like "it will take 4 hours to do that and you will have it in a week". That’s the reality of business but too many people assume project start time equals project approval time.

The truth hurts

I was asked recently to estimate a project where the prospect "just wanted a few web forms" to mimic their existing character interface. Using the points above I created an estimate that included the fact that I had never worked with these people before, that I was unfamiliar with their application, and I provided detailed estimates for every task that we had discussed. Then I doubled the time I thought it would take to accommodate unexpected events. The estimate came out to be over 400 hours. I gave the prospect detail for every task which included creating of files and definitions and testing and QA with end-users, and I suggested that they should try to whittle down or cut out any line item that seemed unreasonable. Every task item was less than 12 hours, most 1-4 hours. (Nothing takes 15 minutes!) I couldn’t see a single place where I could eliminate a task or cut back on the time. The only place where we agreed that the estimate was off (so far) was where I was accounting for learning about file structures, and they have people who are intimately familiar with the structures. As long as their people are available to answer questions, yes, I can cut some tasks significantly – but they need to know that they can’t approve vacation time for these people while we’re working on the project. The estimate scared the prospect away – only temporarily I hope. 400 hours was too much for "just a few web forms". Hey, I completely agree, but unless we agree on the details of the tasks I would much rather scare someone before the project than have a "it’ll only take a few days" project turn into 400 hours. (Yes, I’ve done that too.)

My policy, and I’ve reiterated this for prospects a number of times, is that I would much rather give a realistic estimate and take the chance of losing a new client, than to give a really bad estimate and take the chance of having a very unhappy client. Honesty is the policy even if it may cost us business. Most of the time however, the gory details of a project estimate uncover previously unknown realities of a project, like things that need to be done that no one thought of, or things that still need to be done that aren’t included, or the sheer scope of a project that someone thought was a few minutes of effort – and sometimes I find out that I’m making a mountain out of a molehill.

We’re all human and I’m not afraid to make a mistake in over or under-estimating a project in a detailed estimate. What’s important, and this is a tough concept to get across sometimes, is that the numbers are open for everyone to question before the project is started. Again, I’d much rather be wrong in the estimate than after the project has started. Tell me where the estimate is wrong and we’ll both learn something. Let’s turn the estimate into something more concrete that people can debate, rather than a nebulous (no pun intended) guess that everyone needs to accept on faith – or reject perhaps based on misunderstandings.

Anyone who knows me will tell you that I’m still working on my project estimation and time keeping skills – but at least I know what needs to be done. Are you as lucky?

Ahem – and it occurs to me that I still need to document my time for recent projects, update clients on project status, update my Gantt charts so that everyone can see where we are, and get back to work on projects. So what the heck am I doing here?