The optimist looks at the glass and sees it half-full; the pessimist, half-empty. In a project environment that is adapting to changes in how we lead, resource, and deliver projects, all too often, optimism outflanks pragmatism in successful project delivery.
Best practice frameworks, PMOs, governance forums, stage gates, milestone review, and Agile adoption – these are all capabilities designed to help us deliver better projects.
While we often see the ‘good news’ stories abound in the media, there is a reasonable expectation that with all of these systems and methodologies that we should have evolved beyond project failure. So why is it that projects continue to run late and over budget?
Regularly high-profile projects show us how easy it is to blow the schedule and budget. Think The Sydney Opera House, the NBN roll-out, the Channel Tunnel, WestConnex, and many others.
And it’s really not just an issue for major projects; it happens across all layers of project delivery. Stephen Dubner’s classic Freakonomics recently explored the issue on the podcast of the same name and it got us thinking: Why do organisations still fall prey to the planning fallacy in project delivery?
The Planning Fallacy: A Definition
Daniel Kahneman and Amos Tversky coined the concept of the Planning Fallacy back in 1979, a concept in which predictions about how much time is needed to complete a future task usually show an optimism bias and underestimate the time actually needed to perform the task.
Despite knowing how long a similar task may have taken in the past – including that it may have taken longer than previously predicted – the concept posits that we still have a tendency to be overly optimistic that the current task can be done in less time. The optimism bias affects predictions about our own capabilities and capacities in performing certain tasks, however, when an outside perspective views the same tasks, a pessimistic bias tends to occur, overestimating the time that is required to deliver those tasks.
An expanded definition was offered by Kahneman and Lovallo in 2003, noting that we also display a tendency to underestimate time, cost, and risk in future actions while at the same time overestimating the benefit of the same actions.
“Most of us view the world as more benign than it really is, our own attributes as more favourable than they truly are, and the goals we adopt as more achievable than they likely to be.”
— Daniel Kahneman
Optimism and even over-confidence can produce positive outcomes: They can help us to step outside our comfort zone to attempt larger and more difficult tasks, while protecting us against the natural aversion to loss.
It’s a very primal human trait and one that is difficult to shift. But what impacts do these traits and drivers have when it comes to delivering projects?
The Characteristics of the Planning Fallacy
Optimism Bias
Neuroscientist, Tali Sharot, writes in her book The Optimism Bias, that the belief that the future will be much better than the past and present that forms the optimism bias occurs irrespective of race, region, and socio-economic bracket.
If the bias exists, then it’s natural to expect that at some level a project team is going to be overly confident in the capability of its ability, the impact of managing relationships, and the interactions within and outwith the team. It’s also reasonable to expect that they will underestimate the impacts of procrastination by organisations or individuals.
You might expect that with the growing evidence of not being as successful in delivering projects, however, we often still continue to believe in the illusion of success. Sharot points to the scientific evidence that it may be down to optimism being hardwired into the human brain. Optimism has clear benefits. Hope after all keeps our minds at ease, lowers stress and improves physical health.
In a project environment, however, it almost becomes essential to invoke an optimistic perspective. Presenting an executive committee with a business case that shows the true cost of projects against actual benefits might often stop the project in its tracks. If optimism wasn’t a feature in project delivery, however, would we have the Sydney Opera House?
In fact most infrastructure projects would not proceed without an optimism bias.
Overconfidence
In the Freakonomics podcast, Katherine Millkman, PHD in computer science and business, found that when groups work together, several factors come into play in contributing to the planning fallacy we fall prey to. Overconfidence and thinking we will do better than we actually will is part of it.
“We are overconfident for many, many reasons. One is that it makes us feel better about ourselves and we’re often rewarded for it. Most organisations respond more positively to the person who says, “I’m going to be great and I will deliver great things.”
— Katherine Millkman, PHD
In the era of delivering ‘good news’, overconfidence will have its rewards in the form of greenlighting projects that perhaps require more scrutiny.
Coordination Neglect
Millkman calls coordination neglect the failure to think about how hard it is to put ‘stuff’ together when other people are involved. In a project environment, it’s a dangerous oversight. When there are teams of people coming together to produce a project, not considering the impacts on – or because of – people trying to finish work on time is a risky proposition.
When a bigger team is on a project, there is a tendency to focus on all the benefits associated with specialisation. What is often neglected is how challenging it is to get that work all back together focussed on a single goal.
An example is when an engineer has to talk to another engineer about how to combine their outputs into one integrated system. The effort required to do this seemingly simple task is almost always underestimated. Replicate that several times over in a project, and it’s not hard to see why people are the one part of project management that should not be overlooked.
Strategic Misrepresentation
The Freakonomics podcast also touches on a point that many of us have borne witness to: strategic misrepresentation. Calling it out for what it is:
“Planners were interviewed who said that they did this deliberately, that they actually were incentivised to misrepresent the business cases for the projects in their benefit-cost analysis. They wanted their projects to look good on paper, to increase their chances of getting funded and getting approval for their projects. “We do this by underestimating the cost and overestimating the benefits, because that gives us a nice high benefit-cost ratio so that we actually get chosen”’.
Although seemingly dishonest, this strategy is endorsed by Danny Kahneman himself, who won a Nobel Prize for economics:
“If you realistically present to people what can be achieved in solving a problem, they will find that completely uninteresting. You can’t get anywhere without some degree of over-promising”.
So How do we Overcome Planning Fallacy?
If the tendency of people is to be overly optimistic on capability to deliver (and the time it takes) and the reality is that we don’t see that in delivery, what are some of the steps that can be taken to overcome the challenges that arise with planning fallacy?
Reference-class Forecasting
Yael Grushka-Cockayne teaches project management and decision-making at the University of Virginia’s Darden School of Business. She states that when people estimate how long a project will take, they focus too much on the individual quirks of that project and not enough on how long similar projects took. This approach is called reference-class forecasting.
In Europe and the US, more government bodies are publishing some planned and some actual deadlines and budgets. Tracking performance and tracking historical plans and actuals is the fundamental first step in overcoming the planning fallacy.
By tracking performance, even just starting with that and nothing more sophisticated, organisations will see the profile of the issue as a performance issue within the organisation, which enables them to improve. Once the data is available, organisations can then factor the gap Actual Vs Estimated. This in turn can be built into future planning and start to be more accurate. The continuous feedback of this information will reduce the likelihood of the planning fallacy.
Automation/AI and Algorithms
Algorithms. Machine learning. Artificial intelligence. These are ways of exorcising planning fallacy and a high visibility example of this is online retailer, Amazon.
Amazon relies on an algorithm to make forecasts based on substantial amounts of data going into that algorithm. As an organisation they have solved this problem billions of times before. Using data and computing power instead of human judgment to make forecasts is key if it’s possible to collect and interrogate the data. Organisations investing in AI, machine learning, blockchain and other similar solutions are now permeating many different industries to support better planning and outcomes, and project management will be no exception.
Greenbook
In the United Kingdom, governments have been implementing the Greenbook approach to major infrastructure projects since the early 2000s. The Greenbook approach requires that the cost estimate and schedule estimate is produced for the project, then on the basis of empirical evidence, a database documents how much the budgets are usually underestimated for the category of the project.
The numbers from previous, similar projects are then used for the new project estimate. For example, if projects typically go forty percent over budget, then that amount is added to the budget for the planned project. Project planning teams are required to have an incentivised commercial structure in place to ensure that any vendors or contractors are onboard.
Evidence from the UK to date states that this is helping improve estimating and hence confidence in project budgets, which helps with planning and public sentiment.
Can We Ever Escape the Planning Fallacy?
Planning Fallacy is a constant and is embedded as a foundational human trait. Human optimism may always factor into our ability to properly estimate accurate project delivery, however one message is clear: It’s important to manage optimism and overly high confidence.
Managing it and the impact it has on your next project could mean the difference between success and failure.
As project specialists, we develop fit-for-purpose strategy and project assurance. Contact us here to find out more about how we work with your teams or call 02 9098 6300.
We believe that quality thought leadership is worth investing in. Please share our content with your colleagues via any of the links below.
Interested in republishing our content? Find out how.