How transport projects are evaluated has always been of interest to me. I believe that although the standard cost benefit analysis approach that lies behind the NZTA economic evaluation manual has its flaws, the resulting BCR is still an important factor in determining whether a project, or a particular project option, should proceed. I don’t really buy the argument that a project with an unfavorable BCR should be trumped by “strategic” reasons to enable it to proceed. If the strategic reason is any good it will probably be reflected in the BCR, particularly if wider economic benefits (WEBs) are taken into account. I put the Puhoi to Warkworth business case in this category (BCR 0.92) , along with the eye-wateringly expensive Additional Waitemata Harbour Crossing for cars and trucks (BCR 0.4).
Recently I took a look at a number of documents on the East-West Connections project – formerly called the East-West Link, released by the NZTA . At this stage they’ve completed an “Indicative Business Case” (IBC) – essentially, an initial investigation of the options for improving connectivity in the area. They’ve published the IBC alongside a number of technical appendices.
This is a welcome step as this is the first time that the wider public is getting a decent look at the project, including all of the options on the table. NZTA’s decision to release the full documentation, without redacting large sections of the analysis, is really good for transparency.
So let’s take a high-level look at some of the specific trade-offs between costs and benefits of the different options. To jump right to it, NZTA’s conclusion is that Option F should be progressed to a Detailed Business Case. Here’s a picture of Option F, which involves a new highway along the Onehunga foreshore:
For context, Options C, D and E also involved building new roads (part of the way) along the foreshore, while Options A and B entailed upgrades to existing roads, including freight lanes. A summary assessment of this “short list” is available in Appendix O of the business case.
NZTA conducted a cost-benefit analysis (CBA) of these six options. CBA for transport projects typically compares:
- The financial costs to build and operate the project, and
- The monetised economic benefits of the project, including user benefits such as travel time savings, vehicle operating cost savings, and reliability improvements and other benefits such as vehicle emissions reductions (or increases) and effects on economic productivity (“agglomeration”).
While CBA does have some weaknesses, largely due to shortcomings in the modelling tools available to us, it’s a conceptually robust way to assess project options. Especially in the case of road projects, NZTA’s approach should capture the majority of the economic benefits arising from projects, including productivity improvements for freight users.
This is the summary table:
You can see the net present value of the total benefits exceed the total costs for each option – i.e. the total benefit-cost ratio (BCR) for each option is above 1.
However, there are two problems.
The first is that the NZTA has made somewhat arbitrary assumptions about agglomeration benefits, which in theory reflect the productivity gains arising from improving connectivity between businesses. Rather than formally modelling it using the procedure specified in Appendix A10 of NZTA’s Economic Evaluation Manual, they’ve simply assumed that agglomeration impacts will add 25% on top of transport user benefits for each option.
As Stu previously highlighted in the case of the Mill Road highway, which included a similar “fudge factor” for agglomeration benefits, there is no real reason to do this (other than the rather circular argument that the same thing is being done for other projects).
In the case of East-West Connections, there is a stronger argument to be made for agglomeration benefits, as this project will serve a busy commercial/industrial area. However, it’s still necessary to do the analysis to establish their existence and magnitude! I (perhaps cynically) suspect that the 25% figure has simply been used to make the BCRs all appear higher than they otherwise would be. For public relations purposes it is preferable to have a higher BCR than a low one, even though the purpose of the exercise is an option evaluation rather than an assessment of the absolute economic worth of the project.
The second problem with this table is that it is not consistent with NZTA’s own requirements. Section 2.8 of the EEM sets out requirements for calculating and reporting BCRs. That section requires an incremental analysis of costs and benefits:
In other words, if you are choosing between two options, one of which is considerably more expensive than the other, it’s not enough to say that the more costly option has a BCR above 1. It’s actually necessary to show that the added (incremental) benefits of the costly option exceed the added (incremental) costs.
This is an important step in cost-benefit analysis as it shows you whether spending that extra bit of money for a more expensive solution is justified. Failing to do an incremental CBA is basically an invitation for gold-plating and overspending – i.e. find a worthwhile project, and then jack up the costs as high as possible.
So let’s take a look at an incremental BCR analysis of the East-West options. For simplicity I’ve focused only on Options A, B, and F – the two cheapest options, and NZTA’s preferred option. (Options C, D, and E are fairly similar to F in terms of total costs and total benefits – including them wouldn’t get a different result.)
Here’s a picture of Option A, which is an upgrade of SH20 and the existing Nielson St route to SH1:
And here’s Option B, which is pretty similar but also adds a south-facing ramp to SH1:
I’ve ranked the options from least to most expensive:
- Option A has total costs of $200 million and total benefits of $850 million. Consequently, it has an incremental BCR (relative to spending nothing) of 4.3. In other words, Option A seems like a good project.
- Option B has total costs of $500m and total benefits of $1650m. This means that it has incremental costs of $300m (i.e. $500m-$200m) and incremental benefits of $800m (i.e. $1650m-$850m). Its incremental BCR, compared to Option A, is therefore 2.7. This suggests that it’s well worth spending the extra money for Option B.
- Option F has total costs of $800m and total benefits of $1550m. Relative to Option B, its incremental costs are $300m and its incremental benefits are -$100m. Its incremental BCR is therefore -0.3.
In other words, if the NZTA were to follow their own economic evaluation manual it shows that Option F is not great value for money. It costs a lot more while actually delivering fewer economic benefits than Option B. Negative BCRs are generally not a positive sign that a project is a good idea.
This isn’t to say that we shouldn’t build Option F, or that we should build Option B. There may be some significant positive or negative effects that aren’t captured in this analysis and that may tip things in a different direction. For example, existing traffic modelling tools may not capture travel time reliability benefits very well. Similarly, we haven’t taken a look at environmental costs – on the one hand, Option F paves over the remainder of the Onehunga foreshore, which is negative; on the other, it potentially moves trucks away from the town centre and residential areas, which might be a good thing.
Admittedly I’m not a professional economist, but to me the incremental BCR analysis does highlight several questions that need to be answered:
- Given the fact that Option F costs more than Option B while delivering fewer quantified economic benefits, is there evidence that other unquantified benefits, such as travel time reliability, are sufficiently large to justify the added costs?
- Given that the project is primarily intended to improve convenience for freight users, has the government asked freight companies and shippers in the area if they would be willing to invest their own money to pay for Option F?
- Given the results of the Basin Reserve Flyover hearings, in which a Board of Inquiry found that the incremental economic benefits of the Flyover weren’t sufficient to outweigh the added environmental/amenity costs, is there a risk that approval for Option F won’t be forthcoming?
And finally, given the results of an incremental BCR analysis, isn’t there a case to also progress Option B for a more detailed assessment in the next stage of the work, given that it maximises economic benefits at a lower cost?