Over the last three years, I've painted five rooms in my house. The first room I painted was my bedroom, and I was convinced that I'd have the task finished in a weekend.
A month later, I applied the finishing touches.
So naturally, when it was time to paint the second room, I estimated that the task would take a month, right?
Wrong.
"Now that I have more experience painting," I thought, "I'll definitely knock this room out in a weekend—two at most." The second room also took a month. So did the third and fourth. The fifth, my kitchen, actually took a little over a month.
But each time I prepared to paint a room, I was convinced that I'd be finished in a week or two. Even though I knew I'd never finished painting a room in less than a month, I couldn't let go of my confidence that the next room would be easier and faster to paint.
The confidence that led to my unrealistic estimates actually has a name—the planning fallacy—and it's something we all occasionally suffer from when estimating how long tasks and projects will take us to complete.
What is the Planning Fallacy?
The planning fallacy is a term used by psychologists to describe our tendency to underestimate the amount of time it will take to complete a task. The term was first coined in 1977 by psychologists Daniel Kahneman and Amos Tversky.
Kahneman and Tversky explained that people have a tendency to disregard historical data when making predictions. Instead of forming estimates based on historical evidence (it always takes a month to paint a room), we focus solely on the upcoming task (this room is small, so it won't take long to paint).
Kahneman later expanded on the original idea in his 2011 book Thinking Fast and Slow. In it, he argues that estimation mistakes can usually be attributed to two key factors:
Failing to consider how long it's taken us to complete similar tasks in the past
Assuming that we won't run into any complications that will cause delays
The second mistake can be attributed to another psychological term, optimism bias, that's used to describe our tendency to believe that the future will be better than the past. Or, as it relates to the planning fallacy, the belief that things you do in the future won't take as long as things you've already completed.
In other words, our optimism leads us to believe that it's unlikely that we'll run into issues that cause delays. When we estimate how long something will take, we tend to estimate for the best-case scenario. And we also tend to disregard any historical data we have on hand that proves that the best-case scenario is unlikely.
Unfortunately, it's possible that we can't help feeling optimistic when we imagine future tasks. As Tali Sharot, professor of cognitive neuroscience and author of The Optimism Bias, writes: "A growing body of scientific evidence points to the conclusion that optimism may be hardwired by evolution into the human brain."
But it's not all bad news. If you understand the reasons why we tend to underestimate how long tasks will take, you can take steps to avoid the pitfalls of the planning fallacy and start forming more realistic estimates.
How to Overcome the Planning Fallacy
It's one thing to underestimate the amount of time it will take you to paint a room in your house. That's just an inconvenience. But when it comes to underestimating tasks and projects at work, the consequences can be much more severe.
At best, underestimating at work can lead to you—or your team—having to work overtime. At worst, it can lead to insufficient budgets, inadequate profits, and/or disappointed stakeholders, leaders, or customers.
And according to the Project Management Institute's 2018 "Pulse of the Profession" report, inaccurate time estimates are a primary cause in 25 percent of failed projects.
To avoid the negative consequences of underestimating—at both work and in life—you have to make an effort to stop estimating using intuition only. And the best way to do that is to use an estimation technique: an exercise designed to help you form more realistic estimates.
The following six estimation techniques are all designed to help you avoid underestimating when you're trying to determine how much time a task will take.
1. Use Historical Data
In their 1977 technical paper, Kahneman and Tversky recommend "asking the external question 'how long do such projects usually last?', and not merely the internal question 'what are the specific factors and difficulties that operate in the particular problem?'"
If you're coding a new feature for your company's app, look at how long it took your team to build and release a similar feature in the past. If you're writing a 4,000-word blog-post, review your data showing how many hours/days it took you to write a similar piece previously. Then, base your estimates off of that data.
Of course, to do this, you have to first collect the data you'll use to create estimates in the future.
If you work alone, a simple way to do this is by using a time tracking app. Use the app to track how long it takes you to complete different types of tasks over time, then use the app's built-in reports to reference that data later when it's time to estimate.
If you're working as part of a team, you might want to consider adopting project management software. Many of these tools provide different ways—like time-tracking and Gantt charts—to record how long teams take to complete different types of tasks and projects.
Considering historical data doesn't guarantee accurate estimates. But as Kahneman and Tversky write: "It does provide some protection against completely unrealistic predictions."
2. Have Someone Else Estimate for You
In 1994, the American Psychological Association's Journal of Personality and Social Psychology printed the results of five research studies conducted by Roger Buehler, Dale Griffin, and Michael Ross.
The results of these studies not only confirmed that people are prone to the pitfalls of the planning fallacy as described by Kahneman and Tversky, but they also discovered something unique: While we're bad at estimating how long tasks will take us to complete, we're actually pretty good at estimating how long it will take someone else to complete a task.
When research participants were asked to estimate how much time it would take someone else to complete a task, they were much more likely to refer to historical data to form a basis for their estimates. And even when there wasn't historical data to draw from—when participants got all of their information directly from the person who would complete the task—their estimates were much more conservative than those of the people who actually planned to complete the task.
So instead of trying to estimate tasks yourself, ask a friend or coworker to come up with an estimate for you. While we tend to be optimistic about our own abilities to complete tasks quickly, we're much more pragmatic when it comes to figuring out how long it will take someone else to complete a task.
3. Estimate in Ranges, or Build in Time for Delays
There's a Donald Rumsfeld quote that's referenced a lot in project management:
There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know.
When estimating any task or project, you have to take all of these things into account. There are things you know will happen, things you know could happen, and things you never once considered might happen.
To account for this ambiguity and form better estimates, project managers sometimes use the Cone of Uncertainty:
The Cone of Uncertainty is designed to represent the range of variability an estimate can have based on when you provide the estimate. At the beginning of a project when you know very little, the actual time it takes to complete the project might be anywhere from one-fourth to four times your estimate. So if you think a project will take two days, it might realistically take half a day or eight days.
But as you start working on the project, you learn more, and that range shrinks. Still, it's not until the very end—when you've completed the project—that you can say with certainty exactly how long it will take.
The Cone of Uncertainty provides another couple of options for developing more realistic and accurate estimates.
First, by plotting the stage you feel you're in on the cone, you can develop a ranged estimate. If you know very little about what the project entails, divide your estimate by four to get the low end of your range, and multiply it by four to get the high end of your range. Then, provide a ranged estimate (e.g., I believe it will take between one and 16 days).
If you don't think a range will be acceptable, you can use the high end of the range (e.g., I think it will take 16 days) to account for unknowns. After all, your original estimate was probably optimistic in the first place.
4. Use Three-Point Estimations
The three-point estimation technique forces you to confront your possible optimism by asking you to identify three different pieces of data:
A best-case scenario estimate
A worst-case scenario estimate
A most likely scenario estimate
Knowing that we tend to be optimistic when estimating, the best-case scenario estimate is likely the same as your initial estimate. The most likely scenario might be based on the historical data you have for completing similar tasks. And the worst-case scenario should consider how much time something might take if everything goes wrong.
Once you have your three numbers, calculate the average of the three points of data.
To calculate the average, simply add all three numbers together and divide them by three. For example, if your best case scenario is three days, your most likely scenario is five days, and your worst case scenario is nine days, simply add 3+5+9. Then, take the sum, 17, and divide it by three. The average, 5.67 days, becomes your estimate.
And if you really don't want to do the math, type your three points into this free Omni Calculator to get the average instantly.
5. Calculate Your Fudge Ratio
Steve Pavlina, author of Personal Development for Smart People, recommends calculating a fudge ratio that you can apply to all of your estimates.
You start this process by estimating—via intuition—several tasks you need to complete in the near future. Record your estimates. Then, track how much time you actually spend completing the tasks you estimated.
When you're finished, add all of the estimate values together, and add all of the actual time values together. Then, divide the total actual time values by the total estimated hours to calculate your fudge ratio.
In the image above, I estimated that several tasks would take me five days to complete. In reality, they took seven days to complete. To calculate my fudge ratio, then, I need to divide 7 by 5, making my fudge ratio 1.40. That means I'm generally estimating 40 percent lower than the actual time I'm spending to complete tasks.
Then, to improve your estimates, multiply your initial estimates by your fudge percentage.
If I initially anticipate that a blog post will take me two days to write, I can multiply that by my fudge ratio (2 x 1.40) to come up with a more realistic estimate: 2.8 days.
And if you're having trouble figuring out how to add a percentage value to your original estimate, use this free percentage calculator to calculate your final estimate easily.
6. Estimate During the Low Point of Your Day
In his book When: The Scientific Secrets of Perfect Timing, Daniel H. Pink takes a deep dive into all of the research surrounding how our chronotypes—internal clocks—impact how we feel across the day. What he found is that our chronotypes not only control when we're feeling most energetic and focused, but also at what point in the day we're most likely to feel creative, positive, or negative.
He cites a study by Scott A. Golder and Michael W. Macy that analyzed people's moods using data from millions of Twitter messages. What they found is that people's messages tend to be less positive when they hit their low point in the day—what Pink calls the trough.
Most people hit their trough mid-day, right after lunch. And when you're in your trough, according to the Golder and Macy study, you're less likely to feel positive. Your good mood is at its lowest point in the day. In that case, you may be feeling less optimistic, which could help you create more realistic estimates.
So our final estimation technique is to create estimates when you're in your trough. Most people hit their trough about six hours after waking up, but you can also just wait until you're feeling unfocused and tired. Then, take advantage of your low energy and waning mood to create your estimates.
Finally, measure the estimates you created during your trough against actual time values to see if they're more realistic.
Understand the Planning Fallacy to Get Better at Estimating
The best part about understanding the planning fallacy is that just knowing it's a thing can help you create better estimates. For example, the next time I paint a room, I'm going to expect it to take a month. I now understand why my earlier estimates were inadequate.
In the end, you may not be able to control your optimistic inclinations. But if you understand them and take steps to avoid the trappings of your innate optimism, you can avoid the negative consequences of the planning fallacy and start managing your time much better.
Related reading:
Watch image by Mitchell Hollander via Unsplash. Project status chart by Project Management Institute via Pulse of the Profession 2018.
