Visit the NZ Transport Agency website
www.nzta.govt.nz
Travel behaviour change evaluation procedures
Technical report
Print version: Travel behaviour change evaluation procedures (PDF, 1.57 MB, 98 pages)
5. Proposed evaluation procedure
- 5.1 Objective
- 5.2 Overview
- 5.3 Evaluation procedure components and assumptions
- 5.4 Worked examples
- 5.5 Sensitivity tests
- 5.6 Evaluation stages
- 5.7 Evaluation of packages
- 5.8 Other Transfund allocation procedure reporting requirements
5.1 Objective
A key output of this project is to produce evaluation procedures that are appropriate for the scale of individual TBhC initiatives as well as promote consistency in the specification and evaluation of these projects.
Transfund and EECA’s requirements are for procedures that:
- Are consistent with the Transfund Allocation Process
- Are not inconsistent with Transfund’s other evaluation procedures
- Are simple to understand and apply
- Use inputs that are readily available
- Are evidence based
- Reflect the uncertainty in some of the input variables
- Encourage the identification of costs/benefits that are not quantified/valued in the analysis
- Require an analysis effort that is in proportion to the scale of the projects.
TBhC projects tend to result in small impacts to a large number of people. They are much more difficult to evaluate than conventional projects because the impact tends to be different for each participant whereas with normal road projects most users tend to be attributed the same benefit. This leads to a conflict between procedures that accurately reflect all of the different individual responses to TBhC projects (but which may cost more to actually apply than the cost of the project being evaluated) and procedures that are cost effective to use but that may involve significant approximations and averaging of the effects on different participants. This averaging and approximation is an inevitable consequence of the requirement for simplified procedures and the relative newness of TBhC projects, which means that the database of project impacts is still relatively small and subject to considerable variation.
Nevertheless, in keeping with the project requirement for simplified procedures we have developed an evaluation procedure that covers the main types of TBhC projects and can be extended to other types if necessary. This procedure is considered suitable for adoption as an interim procedure that can be refined in future following more experience.
5.2 Overview
Transfund’s simplified procedures for walking and cycling projects, and public transport funding policies were considered to provide good models for simplified procedures for TBhC projects and would satisfy the criteria of being easy to use and suitable for scale of project. In particular the approach of adopting average values for the aggregate effects of the projects was considered essential to make TBhC procedures practical (subject to conditions that screen out projects types/situations that are known not to deliver the averages).
A single evaluation procedure for TBhC projects is proposed that is similar to the existing Transfund simplified procedure for cycling and walking. However instead of the benefit value being embedded in the procedure, a set of tables is provided giving composite benefit values for the common TBhC types and situations expected to be encountered in New Zealand in the next three to four years. Analysts select the TBhC project description and situation that is closest to their proposed project and use the corresponding diversion rate and composite benefit value in the evaluation procedure. The proposed interim evaluation procedure is shown in Appendix B of the Guidance Handbook.
In terms of the information requirements of the Transfund Allocation Process the evaluation procedure determines the economic efficiency of TBhC projects. Transfund also requires other information on proposed projects and activities as discussed in Section 2.2. This includes information on seriousness and urgency, and effectiveness of the project. If Transfund decides to categorise TBhC projects as generic activities it will provide standard assessments aaginst these other information categories and proponents will not need to complete detailed assessments of these aspects for each project. This is discussed further in Section 5.8 and the Guidance handbook that has been prepared in conjunction with this report.
5.3 Evaluation procedure components and assumptions
This section considers the key components and assumptions of the evaluation procedure. Aspects covered include:
- Value for money measure
- Calculation of composite benefit value
- Trend of benefits in future years
- Evaluation period.
5.3.1 Value for money measure
It is proposed that a benefit cost ratio is used as the value for money measure for TBhC projects. Although some TBhC evaluations have been undertaken using the alternatives to roading procedures to date, which could imply that a measure such as the Efficiency Ratio may be appropriate, this is unnecessary in the case of TBhC projects because of the absence of service provider revenue changes. There may be such changes but these are incidental to the project rather than a central feature of it. The benefit/cost ratio is able to reflect all of the benefits and dis-benefits associated with mode switching and trip change discussed in Section 3.0.
It is proposed that the TBhC value for money ratio comprises:
- Numerator — net perceived and indirect benefits and disbenefits to all TBh changers, other transport system users affected by the project, and all other external effects
- Denominator — net costs of the TBhC project to government (ie Transfund and other government entities, including councils).
Such a formulation will ensure comparability with other Transfund value for money measures.
Consideration needs to be given to excluding the costs of monitoring from the evaluation even though this might be a significant cost for TBhC projects and is likely to be included as part of funding requests. This is because monitoring is not specifically included in the costs of other types of projects and should not be an additional hurdle for TBhC projects. Rather, the additional monitoring costs in early years should possibly be regarded the same as research and development or demonstration project expenditure.
It is understood that Transfund will hold discussions with approved organisations to complete an assessment statement for each proposal. This will provide organisations with the opportunity to highlight where particular activities make additional contributions to the requirements or convey other relevant information in order for this to be taken into account in the prioritisation stage of the Transfund Allocation Process.
5.3.2 Calculation of composite benefit value
The composite benefit values used in the simplified procedure are calculated as a total annual benefit of the proposed TBhC project. The procedure for calculating this value is described below.
Unit benefits
In Table 6 we summarised the total distance related and costs and benefits, and per trip costs, associated with trips by each of the main modes.
- Car as driver
- Car as passenger
- Public transport
- Cycle
- Walk.
Average trip lengths
Average trip lengths for trips for a range of different situations were extracted from New Zealand Travel Survey data. These are shown in Appendix B. Overall the average trip lengths are approximately:
|
|
The above ranges of trip lengths are for different locations, time periods, and purposes. The average trip lengths for school related trips are slightly shorter than the above overall averages.
Cost and benefit values and average trip lengths were calculated for trips:
- In peak and off-peak periods
- In Auckland, Wellington, Christchurch and other urban centres
- To CBD or other destinations
- For commuting, other, primary and secondary school trip purposes.
Average cost (benefit) per trip
The per kilometre and per trip costs and benefits, and average trip lengths, were combined to calculate total average cost per trip for each mode in each of the above situations. These are shown in Appendix B. These figures are the total costs associated with a trip except for the net TBh changer benefit which is included at thenext step. A negative value indicates that an additional trip is a benefit. For example cycle and walking trips have a negative value.
Select diversion rates
A range of default diversion rates for the percentage point change in mode share from car as driver to other main modes in the above situations were determined as discussed in Section 4.0. The appropriate set of diversion rates for the TBhC project situation is selected.
Calculate composite benefit value per year
A composite benefit value for each situation is then determined as follows:
- Multiply mode change (diversion) for each mode by an appropriate average trip cost (or benefit)
- Calculate TBh changer net benefit and multiply by the relevant mode change
- Calculate total benefit per person for target population
- Multiply by relevant days per year to obtain overall composite benefit value per year.
The diversion rates for car as driver are always a negative value (ie percentage point reduction) so when these are combined with the cost per trip (which are always positive) the result is a negative value representing the benefit of avoided trips. Diversion rates for public transport, cycle, and walk are positive values so when these are combined with the cost of a trip the result is positive (ie a cost) if the additional trips involve net costs, or negative (ie a benefit) if the trips involve net benefits (eg due to health benefits of cycling and walking).
It would not be correct to base the benefit of car trips avoided on the average trip lengths for car as driver from the New Zealand Travel Survey and then the cost or benefit of the additional trips be environmentally friendly mode on the average lengths of these modes. This would imply that a 10km car as driver trip could be replaced by, say, a 5km cycle trip. Therefore we have calculated costs for car as driver trips based on the average trip lengths of the modes that people divert to rather than the average for all “car as driver” trips. This recognises that walk and cycle trips are more likely to replace shorter car trips. The result is that the benefit of avoided car as driver trips is based on a weighted average of the trip lengths of the “to” modes.
The net benefit to TBh changers from car driver to public transport and car driver to cycle/walk is then calculated based on the values in Table 3. The benefit per TBh changer is determined by the percentage point change from car to the relevant mode and then converted to a weighted average benefit for all people changing from car to other modes. This benefit value is then multiplied by the “car as driver” diversion rate to obtain the total of his benefit. In cases where diversion rates indicate that some people divert from “car as driver” to “car as passenger”, the weighted average benefit for all people changing from car to other modes is multiplied by the sum of the diversion rates to public transport, cycle, and walk.
The total benefit is calculated as the total of avoided “from” trips plus the total of “to” trips plus the total perceived benefit to TBh changers.
This is then multiplied by the relevant days per year to obtain the overall composite benefit value per year. The following days per year assumptions were used in this calculation:
|
|
The composite benefit calculation also includes an assumption about the proportion of travel behaviour change that occurs in the peak and off peak periods. The assumptions are:
|
|
The assumption of 15% of changed trips in household/community TBhC projects being in the peak period is based on overseas experience that most trip change is off-peak. Estimates from studies indicate a range of 0-20% being peak trips. The assumption of 15% peak trips also captures the fact that there are only 230 workdays per person.
The composite benefit values calculated by the above method represent an average benefit per head of target population and allow for the fact that the majority do not make substantial change and that there are also a proportion who do not participate in the programme at all for one reason or another.
In the evaluation procedure itself the composite benefit value is multiplied by the total of the target population toobtain the TBhC project benefits for the first year after implementation.
5.3.3 Trend of benefits in future years
Effort to date has focussed on estimating the first year benefits after the implementation of a TBhC project. The evaluation also requires an understanding of the likely trend of benefits in future years. There are two inrelated aspects to this that need to be decided for the evaluation procedure:
- Whether benefits of TBhC projects persist or decay over time
- Whether travel by all modes grows at the same rate in future years.
Sustainability of benefits
An issue for the evaluation procedure is whether benefits of TBhC projects persist or decay after the completion of the programme.
The following four possibilities have been suggested in various references:
- Benefits decay over time
- Benefits can be maintained by ongoing “maintenance” expenditure
- Benefits are durable without maintenance
- Benefits increase over time.
Benefits decay over time
The following points are from Ker and James (1999) which is an example of an evaluation in which benefits were assumed to decay over time.
Ker and James noted that the behaviour change brought about through individualised marketing is a social, economic, and environmental asset. Most assets require ongoing maintenance if they are to continue to deliver the benefits for which they were created. There is no experience with maintenance of individualised marketing, nor have techniques been developed specifically to address it. The evaluation therefore adopts a conservative approach to durability of impacts rather than attempting to make an assessment of “asset maintenance” requirements or the extent to which maintenance might increase the level of future benefits.
For sensitivity purposes, three effectiveness decay rates were used:
- A central case in which the residual effectiveness fell to 15% after 10 years
- An optimistic case with half the progressive rate of decay (residual effectiveness approx 42% after 10 years), and
- A pessimistic case with twice the progressive rate of decay (residual effectiveness zero after 8 years).
A further assessment was made of changes in travel behaviour being self-reinforcing and maintained at their initial level.
Benefits can be maintained by ongoing “maintenance” expenditure
A paper by Tisato and Robinson which evaluated a travel blending proposal in Adelaide included an assumption that benefits would decay without maintenance. To maintain the level of benefits they included cost estimates for annual reinforcement programmes and periodic (five year) repeats of the travel diary. The recent Combined Options Assessment and Business Case for a TravelSMART Community Program in Melbourne also ongoing project costs each year equal to approximately 20% of the cost in the first two years although some of this is likely to be related to monitoring.
Benefits are durable without maintenance
Papers by Ker and Brog and John gave examples of TBhC programmes where benefits appeared to have been sustained for up to four years after the programme.
Ker reported as follows. Community/household-based travel behaviour change initiatives are relatively new, but a number of applications have been monitored to establish whether there is any loss of effectiveness over time - ie, whether people have a tendency to revert to their previous travel behaviour.
In Kassel, Germany, only a small reduction in public transport mode share, after the more than doubling immediately post-implementation, was evident after four years, although within this there was a slight shift away from public transport for shopping and leisure journeys with a corresponding increase in work and education.
In Nurenberg, Germany, similar durability of impact was found up to two years after the travel behaviour change intervention, although any conclusions are less robust than for Kassel because of the lack of a control group.
In South Perth, Western Australia, the impact of the pilot project was monitored from 1997 through to February 2000, with the conclusion that, over that period, mode choice stayed constant at the immediate post-implementation levels, with the gains in public transport, walking and cycling mode share being maintained.
Brog and John stated that the effectiveness of learning declines over time unless the message is continually reinforced. With individualised marketing to change travel behaviour, the experience of changed travel behaviour is itself an effective reinforcing mechanism, provided the quality of the experience does not deteriorate. They note that there is limited evidence on the durability of behaviour change. However, a two and a half-year follow-up survey in South Perth, using both experimental and control groups has shown that the extent and form of travel behaviour change has been maintained.
It could be considered that the follow-up surveys themselves help to maintain the visibility of the programme and may be regarded as a form of maintenance.
Benefits increase over time
The UK Department for Transport Smarter Choices report considered the possibility that effects of TBhC projects may build up over time. It noted that this type of effect is common in pricing studies. The report notes that theory, logic and intuition, but little evidence, suggests that this build-up process could also apply to the similar behavioural responses involved in some TBhC measures, and, if so, those studies with a short period will underestimate the impacts of such measures, after allowing also for the effects in the longer term of other factors (eg income, car ownership etc) which may be operating in the opposite direction.
The report goes on to say that other TBhC measures, may have the effect of shortening the behavioural response period, by making immediate information available, and alternatives worth considering, which would otherwise only filter through to some travellers much more slowly, or not at all. Further, some TBhC initiatives seem to need reinforcement or refreshment after a period.
Smarter Choices concluded that it is most common not to assume any delay in behavioural response - ie that full response is achieved within first year (and is fully reflected in diversion rates from after studies undertaken soon after implementation).
It appears that the paper’s suggestion of a build up over time related more to people only becoming aware of a system change gradually. With toll roads for example this is known as the ramp up period. However TBhC projects inherently put as much information in peoples’ hands at the outset so any “ramp up” occurs almost immediately and Smarter Choices did not appear to argue that there was likely to be an ongoing further growth in benefits.
Conclusions on sustainability of benefits
Earlier evaluations assumed that benefits would decay or that maintenance expenditure would be required. The more recent papers have found evidence that benefits appeared to be self-sustaining without specific maintenance. There did not appear to be any evidence for an ongoing growth in benefits in future years (without further TBhC investment).
It is intuitively plausible that if TBhC programmes provide information that corrects misperceptions about alternative travel options and modes that people were not aware of, many of the people making changes will find the new option to be an improvement and will not have an incentive to revert. Reversion is more likely in cases where people were persuaded to change to a less convenient option because this was more environmentally sustainable. People in this situation may be more likely to revert without occasional reinforcement. If TBhC projects include infrastructure changes this may also help to increase the durability of benefits.
Further analysis of previous experience indicates that for household/community initiatives there appears to be some reversion to previous travel choices over the first nine months following the TBhC project but that people who have not reverted by this time tend to stay with their new travel choice. Experience from Perth over a four to five year period indicates stable mode shares at the same proportions as they settled at 9 – 12 months after the TBhC project. There is no experience yet over longer periods but it may be assumed that there is little reason for reversion after four or five years. Therefore evaluations of house/hold community projects could generally assume that benefits will be retained in future years with little or no maintenance expenditure, subject to adopting a suitable evaluation period.
Workplace travel plans and particularly school travel plans are more likely to require ongoing maintenance expenditure due to staff and student turnover. In the case of workplace travel plans some of this will become part of the companies’ cost of business but in the case of school travel plans this may require ongoing council or Transfund expenditure which would need to be estimated and included in the evaluation.
Trip growth rate in future years
A related issue is whether travel by all modes grows at the same rate in future years. In other words if underlying car traffic growth is 2.5% per annum is it reasonable to assume the same growth rate for trips by all modes, and is the TBhC project likely to change the relative future year growth rates in addition to the initial shift in mode shares.
Possible assumptions that could be adopted are that trips on all modes grow at the same rate as underlying traffic growth, or that after the initial response there is no further growth in the changed trips - all subsequent growth in each mode is what would have occurred anyway in the absence of the TBhC project.
The preferred option in this case is to assume that benefits of the TBhC project do not grow in line with traffic growth in future years. This is not to say that trips by alternative modes will not grow but rather that this is underlying growth that would have occurred anyway in the base case and is not attributable to the TBhC project. It could only be attributed to the TBhC project if the project was being repeated each year and “diverting” the usual proportion of the growth in car trips. The implication of this assumption is that we can use a uniform series discounting factor when calculating the present value of future years benefits rather than adiscounting factor that incorporates growth rate assumptions.
5.3.4 Evaluation period
Existing TBhC evaluations used evaluation periods between one year and 30 years with most evaluations using evaluation periods of 5 or 10 years. The evaluation period did not appear to be related to the total project cost but rather to consideration of the likely longevity of the TBhC impacts. It does seem appropriate that TBhC projects should generally use a shorter evaluation period than the standard period of 25 years when they are mostly “soft” measures that do necessarily generate a permanent change.
A 10 year evaluation period is considered the maximum appropriate length for TBhC projects at this stage, given the recommendation above to assume that benefits are sustainable without maintenance, and the absence of experience of the durability of benefits beyond about five years. This could be reviewed in future in the light of ongoing monitoring of projects undertaken in early years.
The main exception would be if a project included items of infrastructure that were expected to have a service life longer than 10 years. However this may be uncommon for most of the infrastructure changes likely to bemade as part of TBhC measures, particularly in urban areas so can be ignored. Significant items of infrastructure likely to have longer lives than 10 years will require separate evaluation in any case.
5.4 Worked examples
A worked example illustrating the application of the interim evaluation procedure is included as an appendix in the Guidance handbook.
Some further examples of TBhC evaluation results using the interim procedure are shown in Table 14:
Table 14: Examples of TBhC project evaluation results
| Workplace | School | Household | |
|---|---|---|---|
| Target population | 500 | 1,000 | 5,000 |
| Composite benefit ratio | $137.9 | $92.4 | $65.7 |
| Discount factor | 5.49 | 5.49 | 5.49 |
| PV of benefit | $378,500 | $507,500 | $1,802,000 |
| PV of cost | $150,000 | $150,000 | $750,000 |
| Benefit/cost ratio | 2.5 | 3.4 | 2.4 |
These results are towards the lower end of the range of results obtained in ex-ante TBhC project evaluations overseas. For household based programmes Winn obtained results ranging from 2.9 to 10 (based on a 15 year evaluation period) in Melbourne, The Department of Infrastructure results were 5.0 to 7.5. Ker and James reported a range of 4 to 33 in Perth and Tisato and Robinson obtained a typical benefit/cost ratio of 5.7 for travel blending in Adelaide. In the case of household programmes most of the differences is considered to be due to higher assumed diversion rates in the other evaluations. Diversion rates recommended for New Zealand are based on averaging and other analysis of post implementation surveys of project results and so avoid the optimism in some overseas evaluations. Some projects will achieve the higher diversion rates but experience shows that a number will be less effective than anticipated.
5.5 Sensitivity tests
As indicated in Section 3.0 sensitivity tests were conducted to assess the effect of different assumptions in relation to some benefit values on the evaluation results.
Three scenarios were tested for each type of TBhC project since some benefit values affect certain scenarios more than others. The results of the sensitivity tests are shown in Table 15.
Table 15: Sensitivity test results
Table 15 shows the total annual benefit value per head of target population for each sensitivity test scenario and the percentage difference between this result and the result obtained with the recommended benefit values incorporated in the interim evaluation procedure. The table shows that changes to the benefit values and underlying assumptions have different impacts on different scenarios.
With workplace travel plans the results for Auckland and Wellington with the medium diversion rate are generally less sensitive to the variations than Christchurch/other. This is because the benefit values are dominated by the decongestion benefits of the avoided peak period car trips which are not affected by the sensitivity tests. The exception is the sensitivity test with higher car parking resource cost correction for peak period trips with CBD destination in Auckland. In this case the alternative resource cost correction value from the passenger transport funding report is considerably higher than the value recommended for TBhC evaluations and as this affects all diverted trips it has a significant effect. The Christchurch/other sensitivity test results are more affected by the changes in all of the tests because the decongestion benefits make up a smaller proportion of the benefits in this scenario so variations in other factors have a correspondingly greater influence. This does not mean that the results for Christchurch are any less certain than those for other locations. The variations tested in the sensitivity tests are to show the effect of different assumptions on the results, not because it is considered that the assumptions could actually be that different in all cases. Also the decongestion benefits adopted are those derived for the Transfund passenger transport funding policy and may themsleves be subject to uncertainty and variation from the regional averages in particular cases.
The results for school travel plans are sensitive to the assumptions in relation to the vehicle operating cost resource cost correction, health benefit resource cost correction and the cycle/walk accident resource cost correction. The result is not sensitive to the TBh changer benefit because we do not include a value for car passengers changing to other modes. It could possibly be argued that in this case where the behaviour change may be more the parent’s choice than the student’s that differences in resource costs rather than resource cost corrections should be used to determine net benefits. If this approach was adopted, a sensitivity test has indicated that the net benefit value could be up to 80% higher than the current proposed value in the simplified procedure. This is mainly due to the calculation counting the full resource cost of the avoided car trips (16 cents/km) as a benefit rather than just the unperceived proportion of costs.
The results show that the benefits for household/community-based projects are generally slightly more sensitive to the different variations tested, in all cities. This is because travel behaviour change resulting from such projects mainly affects off peak trips and so does not produce the decongestion benefits that occur with projects that specifically target peak period travel such as workplace travel plans. Therefore benefit categories other than decongestion have a greater influence on the results.
The sensitivity tests show that the approach and benefit values are reasonably robust (and consistent with other Transfund procedures such as passenger transport funding policy) for workplace travel plans and household based projects but that there is somewhat greater uncertainty with school travel plans. This occurs due to the additional complexity introduced by the fact that the subjects of the behaviour change may not be voluntary participants in the same way that they are for other TBhC projects, and the fact that we do not have mode choice relationships between car passenger and slow mode or public transport, particularly when the passengers are children travelling with parents.
The composite benefit values in the tables in the interim evaluation procedures show the sensitivity of the results to different default diversion rates. These also give an indication of the likely sensitivity to alternative project specific diversion rates that analysts might wish to adopt for non-standard projects if Transfund decides that it is also necessary to have a more detailed full evaluation procedure.
Tests were also undertaken of the benefits of diverting primary school car as passenger trips to walk and cyclebased on the unit benefit values and trip lengths used in the derivation of composite values. Results were comparable with benefit values in Transfund’s simplified evaluation procedure for walking and cycling.
If Transfund does decide to have a full evaluation procedure (probably computer based) for non-standard TBhC projects, it should include a requirement for sensitivity testing given the uncertainty of diversion rates andsome other variables in relation to TBhC projects at present, and the emphasis in the Transfund Allocation Process for reporting to include an assessment of the level of confidence in possible outcomes.
5.6 Evaluation stages
Figure 8 shows the proposed evaluation and funding approval process.
An indicative evaluation would be carried out for workplace or school travel plan projects based on assumedtypical components and costs in order to obtain funding for project development. This could either be for each school or workplace individually or for a number aggregated together. This would then be revised and/orconfirmed when the plan has been developed and implementation funding is sought.
Other types of TBhC projects would be evaluated using the evaluation procedures, probably without needing aproject feasibility stage.
In discussions with Transfund it was decided that the threshold below which a composite evaluation is not required for a TBhC project containing non-TBhC components should be set at $200,000 for the interimevaluation procedures as shown in Figure 8.
The evaluation stages and funding approval process are discussed further in the Guidance Handbook.
Figure 8: Proposed evaluation and funding approval process
5.7 Evaluation of packages
Evaluation of packages is relevant on two levels for TBhC projects. Firstly, it is common for TBhC projects to comprise packages of measures that include infrastructure improvements and other components for which Transfund evaluation procedures already exist. The evaluation procedure needs to be able to deal with the overlaps between TBhC soft measures and the other components in the TBhC package. Secondly the Transfund Allocation Process places an increased emphasis on funding (sizeable) integrated packages of projects and it appears likely that TBhC projects will form part of such packages.
TBhC packages
In the first case, where the TBhC proposal itself is a package of measures, it is proposed that below a certain cost threshold (initially set at $200,000) the individual components of these projects do not need to be evaluated separately. The project cost can include a proportion of infrastructure cost and any benefits and synergistic effects of this are reflected in the default diversion rates and hence benefits. This is because the previous project examples that have been analysed to derive the default diversion rates all had various combinations of infrastructure improvements and other measures as part of the TBhC project package.
If the projected costs of the TBhC package exceed $200,000 and a significant component of these costs is for (roading, cycling or walking) infrastructure or new or improved passenger transport services that will provide wider benefits to other existing road users (including cyclists and pedestrians), a “composite evaluation” incorporating other Transfund evaluation procedures is required to calculate the benefits for the proposal. In recognition of the synergies generated by combining related or complementary projects, the benefits of the different components are generally treated as being additive (i.e. they are added to the benefits calculated for the TBhC components of the package).
Irrespective of the TBhC package composition, the total costs for all components are included in the denominator of the benefit cost ratio. Where a new or improved passenger transport service is involved, the costs included are the “funding gap”, that is the cost that needs to be funded by local and central government if the proposal is to proceed.
For the TBhC components in a package, the appropriate composite benefit value in the TBhC evaluation procedures is used to calculate the “new user” benefits for the TBhC target population/area.
Table 16 summarises the appropriate procedure and methods to calculate benefits from new or improved passenger transport services; walking or cycling facilities; and roading infrastructure of various types for all existing users and for new users from the population outside the TBhC target population/area.
The numerator of the benefit cost ratio for a composite TBhC package is the sum of the TBhC benefits and the non-TBhC benefits.
If future monitoring programmes reveal that total diversion rates associated with implementing the combined TBhC/infrastructure/services project do not occur, then the “treatment of benefits” calculations in Table 16 may need to be adjusted.
The Guidance Handbook contains additional guidance on composite evaluations of TBhC packages.
Table 16: Treatment of non-TBhC benefits in TBhC composite evaluations
| Non-TBhC component | Benefits to existing users and non-TBhC target population new users | Comments |
|---|---|---|
| New or improved passenger transport service | Use Patronage Funding benefits assessment or passenger transport ATR procedures as appropriate to: 1. Calculate existing user benefits (for all existing users whether inside or outside TBhC target population area) 2. Calculate new user and associated externality (remaining road user) benefits for population located outside TBhC target population area |
• Serious potential for double counting of new user benefits – care must be taken not to count TBhC target population twice • If the PT improvements component is small, would normally use PF • Where improvements are more substantial, it may be more appropriate to use ATR procedures |
| Cycle infrastructure | Use Walking and Cycling Simplified Procedure to: 1. Calculate existing user / externality benefits (for all existing users whether inside or outside TBhC target population area) 2. Calculate new user / externality (remaining road user) benefits for population located outside TBhC target population area |
|
| Walking infrastructure | • TBhC procedures should be treated as the primary evaluation • If project proponent thinks more walking trips will be created than is allowed for in TBhC evaluation diversion rates, the Walking and Cycling Simplified Procedure can be used to estimate the additional benefits associated with the extra trips |
• Serious potential for double counting of new user benefits |
| Roading • Bus priority lane, high occupancy vehicle lane • Road capacity improvement • Minor safetty works • Traffic calming (30 or 40 km/h zones) |
Use relevant Project Evaluation Manual procedure to 1. Calculate all benefits associated with roading project |
• Minor road safety works includes such as intersection treatment, parking changes, road crossing • Potential for double counting new user benefits where bus priority lane is proposed - see “improvements to passenger transport services” above for guidance |
Integrated transport packages
The Transfund Allocation Process places an increased emphasis on funding (sizeable) integrated packages of transport projects. When TBhC projects are combined with other types of projects into packages the issue of quantifying the potential synergy benefits arises.
Transfund’s presentation about the new Allocation Process describes packages as follows:
- These are collections of projects
- They should be inter-related and complementary
- They often provide better transport solutions.
The example given is providing intra urban congestion relief through package of constrained road congestion relief projects, alternative transport mode capacity increase and travel demand management measures including TBhC projects.
In terms of submitting packages the presentation slides state that:
- They are submitted as a group, each element project referencing the package
- The package as a whole is described completely, once, and assessed as a whole
- Element projects may be from any approved organisation.
It is not clear whether “assessed as a whole” refers to how Transfund will assess the package or how the evaluation of the project is to be completed by the proponent(s). Transfund has advised that in the evaluation of packages to date the synergy benefits have all been able to be modelled and the package could be evaluated as a whole. However it may not be possible for the impacts of TBhC projects to be modelled as readily as part of evaluations of large transport project packages. It is more likely that TBhC project benefits and costs will need to be added manually to the modelled evaluations of the other parts of the package.
Each TBhC project should be worthwhile in its own right before being added to an integrated transport package. A proportion of any synergistic benefits may already be included within the benefits of the TBhC project. For example if a TBhC project is combined with a road capacity improvement project it could reduce the demand and hence the amount of capacity that needs to be added. However the TBhC evaluation includes benefits based on average congestion levels in the network without the improvement. Although it may enable the capacity and hence cost of the road project to be reduced slightly, this may be offset by the need to assume a lower decongestion benefit in the TBhC project evaluation.
At this stage it is difficult to provide hard and fast rules about these issues. However it needs to be recognised that while there are potential synergies from packages of projects there is also potential for double counting of benefits, particularly if the projects are evaluated separately and then added together, the most likely approach for TBhC projects. The greater uncertainty about achieving the forecast benefits of TBhC projects (as evidenced by the range of outcomes from projects to date) may also be a reason to take a cautious approach. The risk margin may well outweigh the synergistic benefits. It could be possible to include a factor that increases the overall sum of benefits, as an estimate of synergy benefits, when TBhC projects are combined with other transport projects but given the uncertainty and potential double counting it is probably more appropriate at this stage to adopt the position that these positive and negative effects cancel out.
This appears to be an area of the Transfund Allocation Process and project evaluation procedures that is still developing. Evaluation of TBhC packages should as far as possible follow the same procedures as will apply for broader packages under the Allocation Process.
5.8 Other Transfund allocation procedure reporting requirements
Transfund has indicated that in the interim it intends to categorise TBhC projects as standard activities in terms of its Allocation Process requirements. This means that project proposals will need to include reporting with respect to seriousness and urgency and effectiveness in addition to the economic evaluation result.
Table 2 of the September 2004 document describing Transfund's Allocation Process shows that the assessment of effectiveness is to include consideration of:
- Integration
- Safety
- Sustainability
- Responsiveness.
Effectiveness also includes the impact of the project on the New Zealand Transport Strategy objectives of:
- Economic development
- Safety and personal security
- Access and mobility
- Public health
- Environmental sustainability.
Other relevant factors include the contribution of the TBhC project to the National Energy Efficiencyand Conservation Strategy and the relevant regional land transport strategy and other Government directives such as the requirement to assess whether proposals impact on Iwi economic or cultural activities. The TBhC Guidance Handbook, prepared as part of this project, contains more detailed guidance and templates to assist analysts in providing the types of information that are required on these aspects. Transfund's Land Transport Programme on-line reporting system will also provide guidance as to the required categories of information and level of detail.
Other particular reporting requirements that are considered necessary for TBhC projects include:
- A descriptive or quantitative measure of the proponent’s enthusiasm and commitment to make the project a success
- A monitoring plan and proposed funding cash flow - however the Transfund Allocation Process may be adjusted to accept this information elsewhere.
If TBhC projects are eventually categorised as generic projects they will be assigned a standard profile with respect to seriousness and urgency and effectiveness. This means that it would not be necessary for project proponents to provide a detailed assessment and justification of each proposal against the requirements of the Act and relevant strategies since this would be standardised and covered by a standard profile (or profiles) for TBhC projects.
Page created: 29 October 2008