Flyvbjerg and Gardner’s How Big Things Get Done

 Projects Minor, Major and Mega

 

 

 

These days we live and work in a world of projects, with everything from planning a holiday, a product launch or a political campaign seen as a project. Organisations have project teams and use project-based management systems. There is a project management book of knowledge, known as the PMBoK, taught in the many PM courses now available.

 

Major projects, and public projects in particular, are frequently associated with cost blowouts and schedule slippages. Some projects become notorious for cost overruns, like the Sydney Opera House (1,400 percent), the Scottish parliament building (1,000 percent) or Boston’s Big Dig tunnel (600 percent). In fact, the great majority of large projects like airports, pipelines, tunnels, railways and roads are not delivered on time or within budget. Software projects are rarely delivered on budget. However, about 20 percent of major projects are delivered on time and on or below estimated cost, so although that may be difficult and unusual it is not impossible.

 

The 20 percent success rate comes from a database of major projects built by Bent Flyvbjerg, a Danish researcher now at Oxford University. Starting in the early 1990s he began collecting project cost and time data, initially for transport (roads, rail and bridges) then extended to include water, power, oil and gas, IT, and aerospace projects. That data became the basis for many journal papers on project performance and the 2003 book by Flyvbjerg, Bruzelius and Rothengatter Megaprojects and Risk: An Anatomy of Ambition. 

 

Flyvbjerg and his colleagues coined the phrase ‘Survival of the unfittest’ to describe projects that get approved and built despite their poor economic and financial characteristics and outcomes. Their key characteristics of projects are:  

• They are inherently risky due to planning horizons and complex interfaces between the project and its context, and between different parts of the project;
• Costs and benefits are many years in future, and are large enough to change their economic environment with unintended consequences;
• Stakeholder action creates a dynamic context with the escalation of commitment driven by post hoc justification of earlier decisions;
• Decision making and planning are processes with conflicting interests;
• Often the project’s scope or ambition changes significantly after starting work;
• No allowance is made for unplanned events (known as ‘black swans’) so budget and time contingencies are inadequate;
• Misinformation about costs, benefits, and risks is the norm, and in some cases is strategically misrepresented to get a project started and ensure commitment;
• The result is cost overruns and/or benefit shortfalls with a majority of projects.

 

There is also the 2023 book from Flyvbjerg and Dan Gardner, How Big Things Get Done: The Surprising factors Behind Every Successful Project, From Home Renovations to Space Exploration. Thanks to Gardner’s contribution this is a brisk, readable book, not another academic tome, and it made its way onto the best business books lists of the Economist, Financial Times and McKinseys. Although it covers the factors and issues in the 2003 book there is less data and analysis, and this book has more examples of different types of projects (buildings, films, tunnels, railways etc.) and the people responsible. Each of the nine chapters addresses a specific issue, illustrated by interesting stories about the people and projects featured, and presents a key concept, the ‘universal drivers that make the difference between success and failure’. 

 

The first chapter is ‘Think slow, act fast’, meaning plan thoroughly and as completely as possible before staring work, or ‘think first, then do.’ Once started a project should be delivered as quickly as possible, to reduce the risk of something going wrong. The chapter has data from Flyvbjerg’s database of 16,000 projects: 91.5% go over their time and budget; 99.5% go over cost and time and under-deliver on benefits. The typical project has underestimated costs and overestimated benefits, and the risk of a project going disastrously wrong (not 10%, but 100% or 400% over budget) is surprisingly high. 

 

Chapter 2 is ‘The commitment fallacy’, where projects are approved before alternatives are explored and/or continued after money has been spent (the start digging a hole strategy). Strategic misrepresentation is an organisational and institutional explanation where project promoters produce biased appraisals at the approvals stage (underestimated costs + overestimated benefits = approval) and projects that get funded are ones that look best on paper (i.e. have largest errors) not the best projects. Another explanation is psychological, from Daniel Kahneman’s ‘planning fallacy’ for decisions based on delusional optimism about the time needed to complete tasks. Premature commitment leads to poor outcomes because people assume What You See Is All There Is – the WYSIATI fallacy – focusing exclusively on what is in front of them and not exploring alternatives. 

 

In chapter 3 a kitchen renovation is the example, a project that expanded and grew after starting and blew its budget. Although planned well it did not start by asking ‘why are you doing this?’ The point is to decide what the project is for first, before thinking about how to achieve that goal. The first requirement for a successful project is to select the right one, and whether or not to proceed. Chapter 4 is ‘Pixar planning’, which is spending a lot of time exploring an idea with many iterations to get to proof of concept stage. Often repeated advice is the three words ‘Try. Fail. Again.’ The authors say we are good at learning by tinkering, ‘which is fortunate because we’re terrible at getting things right the first time.’ Chapter 5 argues for the importance of experience and tacit knowledge, and shows how common it is for such a basic insight to be ignored. 

 

Chapter 6 introduces Reference Class Forecasting, a solution to optimism bias and the illusion that a project is unique. This involves three steps: Identification of a relevant reference class of past, similar projects; establishing a probability distribution for the reference class; and comparing the specific project with the reference class distribution. From the comparison reliable forecasts of a project’s budget and schedule can be made. In chapter 7 the idea that ingenuity and creative chaos leads to great outcomes is refuted, it is the occasional success, which is an exception, that makes this such a good story. 

 

The importance of getting the team right is Chapter 8, with British Airport Authority’s 2007 Heathrow Terminal 5 the example project. This was a famously successful megaproject. The delivery of T5 on time and on budget, with a remarkable safety record, was due to the three inter-related factors of risk management, integrated teams, and the alliance contract. BAA held all the risk and an incentive contract meant suppliers and contractors were motivated to find solutions and opportunities. BAA used in-house project management teams where traditional boundaries were broken down and replaced by colocation, so people from different firms worked in integrated teams in BAA offices under BAA management. The focus was on solving problems before they caused delays.

 

Chapter 9 argues modularity delivers projects, faster, cheaper and better because it allows repetition, and repetition allows learning by doing. Rather than building one huge thing the Lego approach is to make modules that can be assembled into buildings, cars, cakes, satellites and subway stations. In the database the most successful projects (i.e. least likely to have cost overruns) are energy projects for solar, wind and thermal generators that are inherently modular. At the other extreme are nuclear power plants and waste storage, hosting Olympic Games, and hydroelectric dams, ‘all classic ‘one huge thing’ projects’. The chapter closes with an appeal to address climate change through building out the energy transition as quickly as possible. 

 

The book ends with eleven heuristics for better project leadership that collect the book’s key points. These are ‘rules of thumb used to simplify complex decisions’ such as: Hire a masterbuilder; Get the team right; Take the outside view (i.e. use a reference class); Build with Lego; Think slow but act fast; Think right to left (i.e. start with your goal, then identify the steps to get there); and Say no and walk away. Although these may seem obvious, the point is how often they are not followed and how many projects go over time and budget, and areled by people with only partial competence with no provision made for black swan events.

 

Flyvbjerg and Gardner argue a significant reason for poor decisions on projects is unwarranted optimism about outcomes, the planning fallacy. Planners underestimate the time, costs, and risks due to size, gestation and time for delivery, and overestimate the benefits of projects. In some cases there is strategic misrepresentation of costs and benefits. After a project has started there are the risks of escalated commitment and lock-in, scope changes, and conflicting interests. None of these risks are unknown or mysterious, which raises the question of why so many projects have such poor outcomes. 

 

The answer is often the quality and competence of project managers. A 2016 infrastructure report from the McKinsey Global Institute, the think tank for the management consultancy, found ‘Cost overruns for large projects average 20 to 45%. We often see cost differences of 50 to 100% in similar projects carried out by different countries, even those in similar income levels. If countries apply the best practices that have already been proven effective elsewhere, they can achieve remarkable results.’ McKinsey argued a key factor was the quality of the project manager, as their research ‘across thousands of projects indicates that top quartile project managers consistently deliver projects ahead of time and below cost, whereas the opposite is true for the bottom quartile’. 

 

That said, how likely is it that project managers will read How Big Things Get Done? Probably not enough, if McKinsey is right about how little best practices are copied. Although the book is about projects, it does not specifically include or refer to the PMBoK toolkit of processes and knowledge areas, that project management qualifications are based on. Also, while the examples used of architects like Jørn Utzon and Frank Gehry, Pixar movies, iPods, the Empire State Building and Heathrow Terminal 5 are interesting and revealing, because they are unusual and exceptional projects many project managers might not accept that the lessons taken from those projects are widely applicable.

 

It may be the real audience for the book is clients and owners rather than project managers. The client ultimately has responsibility for a project, even if they try to unload this onto a project manager. Much of the advice, on project selection, planning, iteration, contingencies and modularity for example, is about the development stage of a project when the client is or should be in control, not the delivery stage after work commences when the project manager is responsible. And the important message the book sends is that the success or failure of the great majority of projects is determined early on, during planning and development.  

 

 

                                                                      *

 

 

Bent Flyvbjerg and Dan Gardner, 2023. How Big Things Get Done: The Surprising factors Behind Every Successful Project, From Home Renovations to Space Exploration. New York, Currency Press. 

 

Flyvbjerg, B., Bruzelius, N. and Rothengatter, W. 2003. Megaprojects and Risk: An Anatomy of Ambition, Cambridge, Cambridge University Press.

 

Review of Ed Merrow's book Industrial Megaprojects

 

It is well known that the future is uncertain, where uncertainty is an unmeasurable or truly unknown outcome, often unique. This can be clearly seen on large infrastructure projects, which often bring into focus the issues around project selection. A remarkable number of these projects are unsuccessful, by exceeding their time and cost estimates, or inefficient because their returns and/or benefits are well below forecasts.

Major infrastructure projects and other megaproject costs and benefits are many years into the future, and any estimates of them will depend on the assumptions and type of model used. They change their economic environment, generate unintended consequences, and always have the possibility of escalation of commitment driven by earlier decisions.

Ed Merrow did the first published study on major projects costing over US$1 billion (known as megaprojects) for the US military think tank RAND Corporation in 1988, on 52 private sector projects – refineries, oil, transport, and nuclear. It looked at time and cost performance and the factors that drive the outcomes on these projects. Most met performance and schedule goals, but only four came in on budget with an average cost growth of 88%. He concluded “The larger the project, the more important is the accuracy of early estimates.” (1988: 80). This remains the key issue.

Merrow set up Independent Project Analysis to provide project research for heavy industry and the process and extraction industries. Depending on the project, between 2,000 and 5,000 data points are collected over the initiation, development and delivery stages. From this database companies can compare their project with other, similar projects, across a wide range of performance indicators. The data gives estimates on approval, design and documentation, and delivery times for the type of project, and allows for factors like location, access and complexity in costs. When Merrow published his book Industrial Megaprojects in 2011 the IPA database had 318 megaprojects, out of about 11,000 projects in total, from industries like oil and gas, petroleum, minerals and metals, chemicals, and power, LNG and pipelines.

In his 2011 book Merrow recommended a process he called front-end loading, and his best examples of project-definition reduced project timelines and cost by roughly 20 percent. He saw projects having three stages, the first evaluates the business case, the second is scope selection and development, and the third is detailed design. His argument was that there needs to be gates between the stages that prevents less viable projects from getting to authorisation. He emphasises the ‘period prior to sanction of the project.’

Using evidence from the 11,000 projects in his database Merrow argued the best form of project delivery is what he called ‘mixed’: hiring engineering design contractors on a reimbursable contract then construction contractors on a separate fixed price contract. His view was this is the most effective form of project organization, basically traditional construction procurement where consultants are appointed to do the design and a competitive tender is held for one or more contractors to execute the works against a complete design.

Merrow also argues the owner’s job is to select the right project and the contractor’s job is to deliver the project as specified, on time and on budget. In his view contractual relationships are more tactics than strategy, and cannot address any fundamental weaknesses in the client’s management of the project, in particular the client ultimately has to own the design. This crucial point became widely recognised by the private sector clients/owners of large engineering projects that Merrow studied, because they understood that significant risk transfer from clients to contractors is structurally impossible on the projects they undertake.

Design and delivery of major projects can be contracted separately to reduce project costs and risks so that, as far as possible, design and documentation is complete or nearly complete before tendering. The ‘nearly complete’ qualifier is important. A simple project can be fully specified just because it is simple. However, there is a limit to how much design can be completed in the initial stages of a major project, because the specification of a major project develop over time as the project details are refined and defined. Therefore, it is unreasonable to expect a major project to be fully specified at tender, and in most cases this would not be possible. On the other hand, it is not unreasonable for tenderers to expect the documentation they receive to be sufficient, because the extent and clarity of the design determines their project time and cost plans.

There are a number of advantages of this strategy of unbundling design and construction, particularly for major projects. Breaking a project into smaller, sequential contracts spreads the cost out over time, and does not incur interest costs if a loan is not used for design work. It makes quality control easier and more effective, by being focused on each stage, which is an important risk management tool. Separating the design stage from tendering and construction will also improve opportunities for consultation with the community and stakeholders. Most importantly, completion of design and documentation before tendering reduces contractor risk and therefore total project cost.

This argument is for design and construction of projects to be contracted separately, because this will reduce project costs and risks. As far as possible, design and documentation should be complete or nearly complete before tendering or starting the works. The key factor is therefore the extent of the specifications, on some projects there may be a limit to how much design should or could be completed upfront. For many major projects these develop over time as the project details are refined and defined. It is unreasonable to expect a complex project to be fully specified at tender, and in most cases this would not be possible. It may also be advantageous to look for innovative ideas or design options, so for these projects an incremental approach would allow contractors and suppliers the opportunity for input during the development of the design. This also has the advantage of reducing uncertainty from poor tender documentation, thus lowering risk and cost for tenderers.

To deliver better results in on-time and on-budget delivery, Merrow argues project developers or sponsors should spend 3 to 5 percent of the cost of the project on early-stage engineering and design. This is because the design process will often raise challenges that can to be resolved before construction starts, saving time and money.

If more realistic, and therefore more accurate, time and cost estimates were given for major infrastructure projects before they are approved, and during the design and development stages, there would be fewer recriminations about project performance and less incentive to find scapegoats on completion, which is typically over budget and schedule. There would be fewer of the common accusations of poor productivity, management failures or poor planning, thus lessening the atmosphere of acrimony that often surrounds major projects in their later stages

Merrow, E.W. 1988. Understanding the Outcomes of Megaprojects, RAND Corporation, Santa Monica.

Merrow. E.W. 2011. Industrial Megaprojects: Concepts, Strategies and Practices for Success, Wiley, Hoboken, NJ. Second edition 2024. 

Catch 22: Construction Innovation and Procurement

 

Source: https://www.statista.com/statistics/270233/percentage-of-global-rundd-spending-by-industry/

Construction is in a catch-22 situation, where neither industry incumbents nor its clients can rationally commit to significant, expensive investments in innovation for the vast majority of construction projects. Procurement has a significant effect on technological opportunity and innovation because the effects of appropriability of intellectual property (IP), substitutability between suppliers, and risks associated with innovation for clients are mutually reinforcing factors that have worked against innovation in construction. 

 

The development of new technology and increasing productivity requires investment in R&D and IP. If firms cannot capture the benefits of innovation and IP for some reason, because of imitation, piracy or secure supply of materials for example, they will not invest in innovation. Because the traditional tender method does not allow capture of IP and knowledge externalities by contractors, there is a perverse disincentive to innovate. Tendering rules or codes have been developed to maintain the integrity of the bidding process, not to encourage innovation, and a successful tenderer’s scope to be innovative is limited. There is opportunity to maximise profits within the tender price by novel ways of organising work or driving down subcontract prices, but bidders are not asked to put forward design suggestions, there are no criteria for evaluation of novel proposals, and tenderers cannot be treated equally if one is preferred on an alternative tender, which is non-conforming in terms of the original invitation. 

 

The answer often proposed is that the best way to increase innovation lies in changing the methods and systems used to procure building and construction projects. If contractors can make novel proposals to owners, productivity can improve, and society benefits from innovation. With non-traditional procurement methods such as design and construct (D&C), build, own, operate (BOO) or build and maintain (B&M), this disincentive is reduced because contractors can appropriate benefits of innovation and R&D through improved performance. 

 

It may not be that simple. If all firms have access to the same technology, and compete through continual, but gradual, improvement, they are subject to a ‘ratchet effect’. First identified in the 1930s by sociologists studying workers subject to performance pay, they found workers choose to restrict their output because they rationally anticipate that employers will respond to higher output by raising output requirements by cutting piecework pay or worker incentives within firms. It was also an unintended consequence of Soviet planning. If a factory met or exceeded its planned target, the target for subsequent years was increased, thus reducing incentives and effort for the factory manager. 

 

In construction the ratchet effect can be seen in bidding for projects, where tenderers will typically not deviate far from a client’s expected cost for the project, and all tenderers have access to the same information. Because of the ratchet effect, a firm avoids revealing a significant cost advantage on one project that might jeopardise margins on future projects. Importantly, it allows for innovations that improve productivity and efficiency, that are neither disruptive nor expensive to contractors but will deliver a windfall gain if a project comes in well under budget, which will be hidden from the client and competitors as much as possible. This suggests that there will be cost-reducing innovations available to contractors if they decide to invest, but the pressure to find them will be affected by client demands, upfront costs, market conditions and a competitor’s likelihood of using them. [1]

 

Also, clients avoid risk associated with innovation on their projects and do not include it in their budgets. Clients can act as a significant barrier to innovation because they are concerned about both construction costs and operating costs, and do not think they individually will benefit significantly from a successful innovation. Further, clients carry a significant share of innovation risk and as a result do not take on the risks of budget and time overruns or poor building performance, and other costs associated with innovation. This risk minimisation objective also applies to financiers and insurers of construction projects.

 

While this argument might be generally true, exceptions prove the rule. An example is the Heathrow Terminal 5 (T5) project. This project demonstrates the effect a determined client with a clear strategy to encourage innovation in order to improve performance can have. In its role as the client BAA took on all the risk for the ₤4.3 billion project, under the unique T5 Agreement that the 60 first tier contractors signed. In total, 15,000 suppliers were involved. The overall project was divided into 147 sub-projects, with an integrated team led by BAA responsible for each one. Unlike the majority of megaprojects, T5 was delivered on time and on budget. 

 

The key relevant point about T5 was that innovations were actively sought out and rewarded. These included product innovations in offsite fabrication such as the roof structure, technological innovations such as the tunnelling process and equipment, process innovations such as the two logistics centres and management innovations in the industrial relations, insurance provisions and supplier incentives built into the T5 Agreement. 

 

The risk associated with large, complex projects can provide the motivation for clients to pursue and reward innovation by major contractors and suppliers, who on T5 demonstrated a capability for innovation that is left unused under traditional tendering and procurement methods. However, most construction projects are less complex, many are standardized and repetitive, and clients have no reason to support innovations that might marginally affect their project’s delivery or performance but increase the risk of cost overruns. Construction is in a catch-22 situation, where neither industry incumbents nor its clients can rationally commit to significant, expensive investments in innovation for the vast majority of construction projects. 

 

The traditional procurement method does not allow capture of IP and knowledge externalities by tenderers. Therefore, many believe the best way to increase innovation lies in changing the methods and systems used to procure building and construction projects, but while there will be cost-reducing innovations available to contractors if they decide to invest, the pressure to find them will be affected by client demands, upfront costs, market conditions and a competitor’s likelihood of using them. As a result, innovation is difficult, though not absent. 

 

 

[1] Given a variety of locations with different relative prices, there will be a best location for supply of the most productive factor. Therefore, firms can raise productivity by moving to a site with a larger supply and lower relative price of the most productive factor, so for any one location there will be a better technology in use somewhere else (but with different relative prices). However, firms face search and switching costs when looking for new technology, and sunk costs in adopting one.  

 

 

More Data on Australian Contractors

 Grattan Institute Transport Infrastructure Report

 

 

The Grattan Institute, a Melbourne based think tank for public policy, released an important report into procurement of Australian transport infrastructure projects. Their Megabucks for Megaprojects report has four chapters and makes 12 recommendations. The chapters contain a lot of carefully compiled and useful information, while the recommendations are all worthy and, despite their careful phrasing, make a strong case for greater client involvement in the design, documentation and management of large public sector projects. 

 

Chapter 3 of the report is ‘Competition is fundamental’, addressing the issue of the dominance of tier one contractors. The chapter collects data on projects and contractors that is not readily available, with the sources and methodology detailed in the appendices. Their key point is the increase in size of projects since 2014, as shown in Figure 3.3 below.

For the last few years the quarterly value of work done on these large transport projects has been over $5 billion. In 2020 Australian governments spent a record $120 billion on road and rail transport projects. 

The argument is that it is increasingly difficult for mid-tier contractors to win work on these very large projects, of the 11 projects above $3 billion 8 were ‘contracts involving multiple tier one firms’. These firms are ‘few and well-known’ in Australia and their Figure 3.10 shows how few, and how they consolidated their position through M&A over the last couple of decades. 

The two sources of potential competition for the three tier one contractors are domestic rivals that might scale up sufficiently or new international entrants to the Australian market. In chapter 4 the report argues strongly for breaking up large contracts to allow greater participation from domestic firms, Recommendation 10 is: State governments should develop and use a systematic approach to determining an optimal bundling of work packages for large projects, including when to disaggregate bundles that include both complex and straightforward activities. While not a new idea it is still important because public clients do not generally do this, and often do not have the resources required to manage multiple contracts. 

 

That leaves international entrants, which the report argues have been playing an important role since 2005: ‘International entrants add to local competition, and it’s very helpful to governments if there are a variety of market players willing and able to take on work. In particular, when tier one firms form a joint venture to bid on a large contract, the only source of genuine competition may be from international firms’. Their Figure 3.5 shows the distribution of contracts between new international entrants and firms that were already here in 2006. Of those firms, Bouygues won 4 contracts, Lang O’Rourke and Acciona 5 each. 

There are barriers to entry when bidding for these contracts, on top of the high bid costs. These are the lack of transparency in the weighting given to selection criteria and the emphasis on local experience. The report’s Recommendation 8: In selecting a successful bidder, governments should not weight local experience any more heavily than is justified to provide infrastructure at the lowest long-term cost. Governments should publish weightings of the criteria used to select the winning bid for a contract. The Grattan Institute is strongly opposed to ‘market-led proposals’ from contractors, and strongly in favour of open tendering. The state with the highest transparency rating is NSW, also the state with the most contracts with new international entrants. 

The report collects data on 51 projects over $1 billion in Australia since 2006. Their dataset of transport infrastructure projects includes 177 contracts worth more than $180 billion (in December 2020 dollars). That data makes this an important contribution to the debate about construction industry policy in Australia, to the limited extent that there is such a debate. A couple of decades ago this data would have been published by the Commonwealth, by the Department of Industry or similar organization, and incorporated into the procurement guides being developed by the Australian Procurement and Construction Council and related State agencies. The report concludes “these guidelines leave a great deal of room for subjectivity in the choice of contract type. Although some of the state guidelines and decision-support documents are quite detailed, none go so far as to prescribe a rigorous and systematic methodology for procurement strategy selection.”

 

This raises the awkward question of who the report is addressing. The fundamental problem is the politicization of the project selection process not the cost of delivery, Australian construction is not expensive by international standards. The recommendations address the problem obliquely by highlighting improvements in procedures and processes, all of which have merit, but not considering alternatives such as the role an independent authority could play or national coordination of procurement and other regulatory systems. In this it was something of a missed opportunity

 

 State and Federal budgets have billions in unallocated funds for projects at all levels (community sports grants, local and regional infrastructure) and for major projects the Commonwealth has Snowy Hydro, the NAIF, the Murray-Darling Basin Plan etc etc. These projects, large and small, shovel money out the door with little or no accountability and there is no evidence that politicians are interested in change at this time.  

Procurement, Contracts and Information