This Is Why Your Projects Always Take Longer Than You Expect
A contractor friend of mine once told me that no matter how well you plan, any home renovation project will always take longer than you think. In fact, he has come up with a formula for figuring out a more realistic time frame: Double the number and go to the next unit of time for your estimate. For instance, if you believe your kitchen renovation will take two weeks, according to my friend, it will actually take four months. (See also: Is DIY Home Renovating for You?)
This phenomenon is called the planning fallacy, and it happens to all of us when we plan any kind of project. (Full disclosure: I was supposed to have this piece written back in April.)
Economists Daniel Kahneman and Amos Tversky coined the term in 1979 in order to describe our tendency to underestimate the amount of time it takes to complete complex tasks — even when we have experience with similar tasks taking longer than our estimates.
The interesting thing about the planning fallacy is that it is a nearly universal human quirk. There are very few people and organizations that are able to overcome it. Here's what you need to know about the planning fallacy and some strategies you can use to combat its costly influence.
Why We Underestimate
Behavioral economists and psychologists tend to agree on the reasons why we fall victim to the planning fallacy: We are just too optimistic.
For instance, if you are planning a cross-country move, you might think about each of the necessary steps to take to go from one state to another. You'll think through each step, estimate the typical amount of time each will take, and add them all together. But, as Julia Galef points out on bigthink.com,
"The more steps you have in whatever project or task you're working on, the greater the chance that in one of those steps you're going to hit a snag and it's going to turn out to be atypical."
People have enough trouble recognizing the probability of single events. Add in compound probabilities, and we are generally going to plan for everything being exactly typical. This is why studies have shown that people who are asked for a best-case scenario estimate and a realistic estimate provide the exact same time estimate.
Additionally, there can be a self-serving aspect to the planning fallacy. Not only might you purposefully underestimate the time of a project in order to snag a waffling customer, but you might also unconsciously take credit for previous tasks that went well while blaming delays on outside influences, which will make you discount the time evidence of past projects. Even if you are absolutely correct that you are a superstar and the last delay was the distributor's fault, that does not change the fact that distribution might be a problem with the next project, too.
Pulitzer Prize-winning author Douglas Hofstadter (for whom "The Big Bang Theory's" Leonard Hofstadter was named) coined the following law:
It always takes longer than you expect, even when you take into account Hofstadter's Law.
This crystallizes the big problem with the planning fallacy. Even when you recognize that we all have a tendency to underestimate how long something will take, it's not enough to simply add an additional 20% or 40% to your estimate. It will still take longer than you expect.
Overcoming the Planning Fallacy
Unfortunately, even if we have the information necessary to take an outside view of our project, we're still likely to fall victim to the planning fallacy. However, there are a couple of strategies you can use to reduce the effect of the planning fallacy on your projects. (See also: Don't Panic! How to Meet a Deadline)
Take the Outside View
We fall victim to Hofstadter's Law due to what Daniel Kahneman describes as the "inside view" to look at our projects. From the inside, we see our own project as something over which we have a unique level of control. However, if we take an "outside view" and look at our project as one of a group of similar projects, we can much more accurately predict how long the project will take based upon the evidence of others like it.
Systematically Increase Your Estimate
This is basically the advice that my contractor friend gave to me. When planning a project, increase the amount of time that you estimate it will take by doubling the number and going up to the next time unit. This is safer than simply adding additional days (or weeks, or months) to the estimate you come up with because it leaves time for seriously disruptive delays.
The benefit of this strategy is that it doesn't require a great deal of additional thought. However, it is still possible to fall victim to Hofstadter's Law with this strategy. And having used it myself when dealing with various renovation projects around my house, I have found myself coming up with the revised estimate, and refusing to believe it will take that long. (It will.)
Ask an Expert
One of the reasons it can be so difficult to take the outside view of a time estimate is because you are intimately acquainted with all of the specific details of a project, which will lead you to believe that this one is different. Even if you have personal knowledge of other, similar projects, you're likely to underestimate the length of time yours will take.
So, one of the easiest ways to get an unbiased time estimate is to ask an outside expert how long similar projects have taken. That said, it might be difficult to believe the estimate they give. As the Less Wrong blog puts it,
"You'll get back an answer that sounds hideously long, and clearly reflects no understanding of the special reasons why this particular task will take less time. This answer is true. Deal with it."
In her book Time Management From the Inside Out, organization guru Julie Morgenstern outlines a simple but difficult plan for improving your ability to estimate the time it takes to complete tasks: Estimate how long it takes you to complete various tasks, and then time yourself when you do them. This strategy will force you to take an outside view of your tasks and projects, rather than rely on your optimistic inside view.
For long-term projects, Morgenstern recommends breaking down the project into each of its component parts and estimating the amount of time each step will take. If you also record the actual time each step takes during these longer projects, you'll be giving yourself valuable evidence for planning your next project.
Write a Pre-Mortem
Research psychologist Gary Klein created the pre-mortem strategy for dealing with the planning fallacy. In this strategy, just prior to committing to a project, you imagine that you have committed to it, and it is a year later and the project was a disaster. You then spend about fifteen minutes writing out the history of what went wrong. This will allow you to pinpoint ahead of time where problems may arise in your plan.
Klein originally proposed this strategy for organizations, where doubts about a proposed plan of action can often be suppressed. The pre-mortem legitimizes those doubts.
However, a pre-mortem is also a great exercise for an individual making a plan. It allows you to think through ways your plan could be derailed, which will allow you to decide ahead of time how to handle those derailments. You cannot do that kind of pre-planning if you haven't thought through the likely obstacles you might be facing.
The real problem with the planning fallacy is that it leads to overpromising and under delivering. Not only does that cause you stress, but it can strain both work and personal relationships. These strategies can help you to combat the effects of the planning fallacy, and give you the gift of unstressed productivity.
Have you ever had a project go disastrously, miraculously way beyond schedule? Please share in comments!