Delusion and Deception in Project Selection

Two Basic Methods for Prevention and Avoidance

It is well known that the future is uncertain, where uncertainty is an unmeasurable or truly unknown outcome, often unique. This can be clearly seen on large infrastructure projects, which often bring into focus the issues around project selection. A remarkable number of these projects are unsuccessful, by exceeding their time and cost estimates, or inefficient because their returns and/or benefits are well below forecasts.

Major infrastructure projects are typically selected under conditions of uncertainty, not risk. Risk is identifiable and measurable, uncertainty is not. There are three main reasons:

1. Costs and benefits are many years into the future, and the estimates depend on the assumptions and type of model used;
2. These projects are often large enough to change their economic environment, hence generate unintended consequences, with the Oresund Bridge between Sweden and Denmark the prime example; and
3.  Stakeholder action creates a dynamic context, with the possibility of escalation of commitment driven by earlier decisions.

In their 2009 paper ‘Delusion and deception in large infrastructure projects’ Flyvbjerg, Garbuto and Lovallo argued project planners are often far too optimistic in their estimates (delusion) or ‘strategically misrepresent’ their project to approving and funding organisations (deception). Clearly, one path to better project selection that would address these issues is better information about the proposed project.

One source of such information can be found in the performance of previous similar projects. Although it seems obvious, this has only recently become common practice by some experienced private sector clients’ when considering major projects, as the example of IPA shows.

Independent Project Analysis was established by Edward Merrow in 1987, after a stint at RAND where he did the first published study on megaprojects, those costing over US$1 billion. The company provides a project research capability for heavy industry and the process and extraction industries. Their database in 2011 had 318 megaprojects, of about 11,000 projects in total, from industries like oil and gas, petroleum, minerals and metals, chemicals, and power, LNG and pipelines. In his book on megaprojects Merrow found that the best examples of project-definition work reduce both project timelines and costs by roughly 20 percent.

Depending on the project, between 2,000 and 5,000 data points are collected over the initiation, development and delivery stages. From this database companies can compare their project with other, similar projects, across a wide range of performance indicators. The data gives estimates on approval, design and documentation, and delivery times for the type of project, and allows for factors like location, access and complexity in costs.

In his 2011 book Industrial Megaprojects Merrow advocates a process he calls front-end loading, the “period prior to sanction of the project”. There are three stages. In summary, the first evaluates the business case, the second is scope selection and development, and the third is detailed design. His argument is that there need to be gates between these stages that prevent less viable projects from getting to authorisation. If there is a problem in the private sector with project selection, even with the managerial structures, capital budgeting and corporate finance constraints found in profit-driven companies, then the problem in the public sector can be reasonably expected to be much worse.

A significant reason for poor decisions on project selection is unwarranted optimism about outcomes, called the planning fallacy by Kahneman and Tversky, or the tendency to underestimate the time needed for a task, even with the experience of similar tasks over-running. Thus, we have a general tendency to underestimate the time, costs, and risks of future actions and overestimate benefits of those same actions.

In their ‘Delusion and deception’ paper’ Flyvbjerg, Garbuto and Lovallo proposed a solution to optimism bias they called Reference Class Forecasting. This works the same way as the IPA database, but their database was mainly composed of public infrastructure projects, many in the transport sector. RFC involves three steps:

1. Identification of a relevant reference class of past, similar projects;
2. Establishing a probability distribution for the reference class;   
3. Comparing the specific project with the reference class distribution.

In decision-making under uncertainty errors of judgment are often systematic and predictable rather than random, manifesting bias rather than confusion. RFC may limit bias just by following a procedure and by gathering relevant data for a panel of projects to be used in the comparisons. RFC may also prevent excessively large projects being preferred to more welfare-efficient projects when the political benefits are large compared to more effective projects.

To deliver better results in on-time and on-budget delivery, Merrow argues project developers or sponsors should spend 3 to 5 percent of the cost of the project on early-stage engineering and design. This is because the design process will often raise challenges that can to be resolved before construction starts, saving time and money.

If more realistic, and therefore more accurate, time and cost estimates were given for major infrastructure projects before they are approved, and during the design and development stages, there would be fewer recriminations about project performance and less incentive to find scapegoats on completion, which is typically over budget and schedule. There would be fewer of the common accusations of poor productivity, management failures or poor planning, thus lessening the atmosphere of acrimony that often surrounds major projects in their latter stages. This would also encourage more transparency about the project’s performance, in both the delivery and operational stages, particularly by public officials.

Merrow argues the owner’s job is to select the right project and the contractor’s job is to deliver the project as specified, on time and on budget. In his view contractual relationships are more tactics than strategy, and cannot address any fundamental weaknesses in the client’s management of the project, in particular the client ultimately has to own the design. This crucial point is now widely recognised by the private sector clients/owners of large engineering projects that Merrow studies.

For example, both Shell and BP established project academies in 2005 because they understood that significant risk transfer from clients to contractors is structurally impossible on the oil and gas projects they undertake. In the public sector, the UK Cabinet Office started a Major Projects Leadership Academy with the aim of reducing reliance on consultants, and in Australia a similar Leadership Academy was announced in 2013, and six MBA-type courses on procurement developed with government departments are now running at Australian universities.

A great deal is already known about the requirements for large infrastructure to be successful, based on the performance of projects over the last two decades and the many studies and reports that have been done on those projects. Better use of data from previous projects in the evaluation and definition stages of new projects would be a transformative innovation in procurement management, and a more empirical approach by clients in collecting and using data is necessary if better decisions are to be made.

 

Flyvbjerg, B., Garbuto, M. and Lovallo, D. 2009. Delusion and deception in large infrastructure projects: Two models for explaining and preventing executive disaster, California Management Review.

Kahneman, D. and Tversky, A. 1979. Intuitive prediction: biases and corrective procedures. TIMS Studies in Management Science.

Merrow. E.W. 2011. Industrial Megaprojects: Concepts, Strategies and Practices for Success, Hoboken, N.J.: Wiley.

Sunlight is the Best Disinfectant

Collusion and Corruption in Canadian Construction

By coincidence, around the same time as Dyson Heydon was delivering his Royal Commission’s report to the Australian Government, in Canada another Commission had also found collusion and corrupt behaviour in the construction industry in Quebec. The problems of collusion and corruption are of course not unique to Australia and Quebec, as the Netherlands twelve years ago and Spain in 2014 demonstrate.

The findings from both inquiries on organised crime and a culture of lawlessness in the construction industry are disturbingly similar. Both Commissions identified various causes for the illegal activities they found to be pervasive in construction. How many recommendations get implemented by their respective Governments remains to be seen, although the Commissions’ findings and recommendations provide all the justification needed.

The Report of the Commission of Inquiry on the Awarding and Management of Public Contracts in the Construction Industry, headed by Justice France Charbonneau, was released in November 2015. The Report (4 volumes and 1,741 pages) made 60 recommendations to prevent collusion and corruption in the construction industry, and links with organized crime. Unfortunately, the report is in French so this post is drawn from English language coverage, not the original documents (including here for the recommendations, here for the main witnesses and their testimony, and here for context).

Following a 2009 Radio-Canada broadcast into corruption inside Quebec’s biggest construction union, a joint investigation was carried out by Canada’s Competition Bureau and Unité Permanente Anticorruption, an anti-corruption unit established by the Government of Quebec. The investigation uncovered evidence of a scheme giving preferential treatment to a group of contractors in municipal contracts, mainly for infrastructure projects.

As a result of that investigation, 77 criminal charges were laid against nine companies and 11 individuals in 2012. These charges included 20 counts of bid-rigging under the Act against nine companies and 24 counts of bid-rigging against six individuals. Other criminal charges included corruption in municipal affairs, breach of trust, improperly influencing municipal officials, fraud, production and use of counterfeit documents, secret commissions, misrepresentations or false statements, extortion and conspiracy.

The Charbonneau Commission was set up by the Government of Quebec in 2011 to look at cases of collusion and corruption in Montreal over 15 years between 1996 and 2011. The terms of reference included financing of political parties and organized crime in the construction industry, and recommendations for remedial measures to identify, eliminate and prevent collusion and corruption. Another 37 people, including two mayors, were arrested in 2012.

The commission shone a light on politicians’ and public officials’ relationship with construction and related service companies over more than two decades. It concluded that a link existed between political party financing and public contracts, calling the practice deep rooted and systemic. Several recommendations focused on reforms to political financing rules. Quebec’s strict laws governing political donations mean parties are unable to raise enough money for campaigns from individuals but are prohibited from receiving corporate donations. What happens instead is corporate donations funneled through proxies. This is illegal.

The scale of the system, however, extended much further. It linked construction firms, many with mafia ties, and a cabal of consulting engineers to politicians and managers in Montreal. Corrupt practices included kickbacks, illegal campaign contributions, rigged contracts with mafia payoffs, limiting competition for projects to a favoured few with the right political and/or mafia connections, and using threats to discourage competition by persuading potential bidders to withdraw. The Commission differentiated this kind of tacit collusion, where reputable firms withdraw from the market, from political corruption, where companies illegally contribute to political parties or candidates.

The Commission investigated the consulting engineering industry in Quebec and made a number of significant recommendations. Quebec is the only jurisdiction in Canada that does not allow a professional order to impose disciplinary sanctions on a partnership or a corporation that provides professional engineering services. This limits its supervisory powers to engineers as individuals.

Corrupt public authorities imposed unreasonably short deadlines for tenders, to benefit one bidder. Substantial and late alterations to a project, often described in addenda, without adjusting the tender filing deadlines, had the same effect. The case of Montreal's procurement of water meters was cited as an example of this.

Commissioner Charbonneau concluded corruption and collusion are "far more widespread than originally believed" and found that organized crime had indeed infiltrated the industry. The Italian Mafia was deeply involved in the industry and significant sections of the report were blacked out because of criminal cases before the courts.

Commissioner Charbonneau found several causes, connected with the public procurement of construction projects, for the various schemes uncovered during the inquiry. Major failings in procurement strategies, in particular the lack of expertise, resulted in public bodies being exposed to a high risk of collusive agreements. For example, the non-negotiable tariff contracts, called “rate-setting contracts”, set a price for manufacturing and laying asphalt a geographical area. The effect is that the contracts allow the asphalt plants owners to divide up territories.

There were also transparency issues. Examples were the release on request of the list of contractors who have the specifications or tendering documents, and the release of cost estimates for projects. The Commission noted that it was well-known the bid bonds Montréal requires for tenders are 10% of the estimated cost of the project. By disclosing the amount of the bond security required, the City was publishing the amount that it expected to pay.

A small group of union halls were allowed to dictate who is hired for each job, although around 70 percent of Canadian construction workers are not unionized. The testimony at Charbonneau was often salacious, with tales of the Province's largest union being infiltrated by the Mafia and Hells Angels, death threats for dissenters, million dollar investments of union funds in biker-owned strip clubs and massive financial fraud involving some labour leaders.

The Commission was asked to make recommendations, and the first recommendation was a Procurement Authority, a centre of expertise for procurement processes to support and oversee public bodies. In the words of the Commission: “The lack of personnel and the loss of expertise may have created fertile soil for collusion.” The Commission also recommended the various public authorities consolidate their internal expertise in construction.

The second recommendation was to vary the rule requiring contracts to be awarded to the lowest conforming bidder, arguing the strict rule of winning public contracts as lowest tenderer facilitated collusive agreements, based on the cartel's ability to control the results of calls for tenders.  Also the mandated 15 day tender period should allow adjustment for project complexity.

The Commission recognized long payment delays restrict competition in the industry, a factor in the creation and continuation of collusive agreements. Contractors bear the cost of payment delays and the lack of funds limits the numbers and growth of contractors. In 2013, more than three-quarters of contractors were said to have not responded to at least one call for tenders, because they viewed the payment clauses as abusive or they anticipated payment problems.

The other recommendations covered whistleblower protection, financial transparency, and relationships between officials, politicians, unions and contractors. There was some criticism that the recommendations were too moderate and lacked bite.

In its details the operation of this system was unique to Quebec and Montreal, due to cultural and institutional factors. However, the incentives inherent in the transactions between construction contractors and governments inevitably provide opportunities for side deals, collusion and corruption. And in Quebec these had become entrenched. Despite critics claims the Charbonneau Commission was too expensive and ineffective it did what any good anti-corruption inquiry should do, shine a light on the institutions and people involved. Sunlight is the best disinfectant.