Quiggin, J. (1991), Too many proposals pass the benefitcost test: commentı, American Economic Review 81(5), 14469.
TOO MANY PROPOSALS PASS THE BENEFIT COST TEST - COMMENT
University of Maryland
In a recent paper, Hoehn and Randall have argued that, because of a failure to take into account interactions between programs, 'too many proposals pass the benefit cost test'. In this note it is argued that the first of Hoehn and Randallıs results is unrelated to policy interactions, and simply imposes an upper bound on aggregate benefits. The second result is incorrect as it stands, and may be corrected only with a more restrictive definition of a policy proposal.
*I would like to thank Bob Chambers, Darrell Hueth, Richard Just, Erik Lichtenberg and Ted McConnell and several anonymous referees for helpful comments and criticism. Responsibility for errors remains my own.
TOO MANY PROPOSALS PASS THE BENEFIT COST TEST - COMMENT
In a recent paper, John Hoehn and Alan Randall argue that conventional benefit cost outcomes are systematically biased upwards because of interactions between policy components which are not taken into account in standard benefit cost analysis. Further, they argue that, if the costs of policy change are non-trivial, then the agenda as a whole will produce negative benefits as it becomes large. The purpose of this note is to show that the mathematical model proposed by Hoehn and Randall is essentially unrelated to the supporting verbal argument.
The first result is actually implied by the apparently innocuous technical assumptions of the model and has no necessary connection with policy interaction. In essence, the result is that, if aggregate estimates exceed some upper bound imposed by the finite nature of the economy then they must be over-estimates. The second result is invalid as stated, and can be rescued only by imposing strong assumptions about the nature of the policy program. Policy proposals must be defined to ensure that interaction effects are always of one sign. Thus, contrary to Hoehn and Randall, it is not possible to determine a priori whether too many proposals pass the cost-benefit test, or whether the public sector overprovides non-market services.
The framework adopted by Hoehn and Randall involves a division between market and non-market goods. Government policy is assumed to affect the supply of non-market goods, specified as a vector s = (s1, s2........sK). The independent evaluation and summation (IVS) procedure consists of evaluating changes in the components i = 1...K separately with respect to the initial position, thereby ignoring the effects of policy interaction.
The first result claimed by Hoehn and Randall states that IVS necessarily leads to overvaluation of the policy agenda as it becomes large. This result turns on a simple numerical argument which applies equally in the presence of any source of error in benefit cost evaluation, or indeed in any testing process. In essence, the argument runs as follows. Assume that the evaluation process is such that there always exists at least one additional policy proposal having estimated benefits greater than some positive value e (in the terminology of Hoehn and Randall, the policy environment is e-augmentable). Since the welfare of society is bounded, it cannot be improved by more than some finite number o(a,dfo 3()-). Then, if there are more than n = o(a,dfo 3()-) /e proposals each having estimated benefits greater than e, the benefit cost process must over-estimate the total benefits.
The crucial step in this argument is the assumption of e-augmentability. The symbol e is normally used for an arbitrarily small positive number. Similarly, Hoehn and Randall (p.548) give the following informal definition for e-augmentability:
³A policy environment is described as e-augmentable if it is always possible to find at least one more policy component that appears to be beneficial when evaluated independently²
which implies that e may take any positive value. However the number of separate components of policy, n, is bounded above by K. Hence, for the argument to work it is necessary that e > o(a,dfo 3()-)/K. Since the upper bound o(a,dfo 3()-) is derived from the boundedness of the economy, and is likely to be of the same order of magnitude as the value of market GNP, it is possible that e may be very large even when K is also large.
The theorem simply states that if there are sufficiently many projects yielding sufficiently large benefits, then total benefits must be overstated. Thus, even for large agendas, it is an empirical question whether the IVS procedure leads to an over-estimate of benefits.
In the formal model presented by Hoehn and Randall, K is fixed, but some of their discussion implies that K, and hence n, may be made arbitrarily large. One way of doing this would be to assume that evaluation takes place separately for individual proposals, rather than for individual components of s. In this case, the number of proposals could be unbounded and e could be chosen to be arbitrarily small. However, from the model assumptions, the space of possible outcomes is compact. Thus, no path of infinite length can avoid coming arbitrarily close to itself. That is, it is impossible that an infinite sequence of proposals, all yielding non-trivial net benefits at the initial price level, can be feasible.
A second possibility would be to consider a finer specifications of the goods space, thereby yielding an increase in K and a reduction in the lower bound for e. However, since the benefits associated with any single-component policy would decline commensurately with the reduction in e, this modification would have no substantive effect.
Hoehn and Randall assume that the only source of error in the IVS procedure is the failure to consider interaction effects. Other sources of error are excluded by assumption. However, for the purposes of the mathematical argument, based on e-augmentability, any other source of error could have been specified in place of interaction effects. Assume that interaction effects are taken into account, but costs and benefits are measured with error, due to technological uncertainty or some other source. The error process may be unbiased, or biased towards either overstatement or understatement of net benefits.
The boundedness of total welfare implies that, for e > o(a,dfo 3()-)/K, the set of policy alternatives yielding benefits greater than e is finite, and bounded above by some n* < o(a,dfo 3()-)/e. The assumption of e-augmentability now implies that, after the implementation of any set of g policies, there remains a policy which yields measured benefits greater than e. When g > n*, there must be some policies for which the measured net benefits are greater than e, although the actual net benefits are less than e. The proof of Theorem 1 now goes through without any change.
Theorem 1 is driven entirely by boundedness, and the assumption of e-augmentability. It does not depend on interactions between policy components or on any other specific source of error.
The second Hoehn-Randall result is that, when policy change is non-trivially costly, IVS states the benefits of the policy agenda as positive when in fact they are negative. The argument here is more closely tied to a benefit cost framework. The framework adopted by Hoehn and Randall involves a division between market and non-market goods. Government policy proposals are assumed to change the net supply of non-market goods while consuming at least s in market goods, evaluated at the initial starting point. Convexity in preferences ensures that the combined cost of g proposals is at least as great as the cost of the proposals assessed separately and hence is at least gs . This amount is unbounded as g grows large, and hence must eventually outweigh the bounded benefits of the proposals. A project, that is, a shift from ak-1 to ak, is defined as non-trivially costly if at least one component of ak-1, denoted aj, is increased and this increase requires an expenditure of market resources such that the marginal cost of an increase in aj exceeds some finite d.
It should first be noted that there is a technical error in Hoehn and Randallıs proof. They assert that the sum
(1) Isu(i=1,r, ) -MCj (aj -o(a,dfo 3()~)j)
is unbounded as r grows large. The argument is that since MCj d and aj -o(a,dfo 3()~)j is finite there is a s such that MCj (aj -o(a,dfo 3()~)j) s for all r elements. Hence by choosing r o(a,dfo 3()-)/s, it is possible to make the sum greater than o(a,dfo 3()-) for any o(a,dfo 3()-). That this is incorrect may be seen by considering the case aj -o(a,dfo 3()~)j = 2 -j, MCj = d, " j. The sum is now bounded above by -d. The problem in Hoehn and Randallıs argument is that s depends on r, so it is not possible to choose r o(a,dfo 3()-)/s. This problem may rectified by a change in the definitions of non-trivial cost, directly imposing the requirement that MCj(aj -o(a,dfo 3()~)j) s for some positive s. As with Proposition 1, the mathematical argument depends on the implicit assumption that s > o(a,dfo 3()-)/K.
However, a more critical point in this case is the definition of a proposal. In their presentation of the model framework in Section I, Hoehn and Randall use a very general specification of the notion of policy proposals. In particular, they explicitly (p545) include the case when non-market services are used as inputs into production of market goods. Many government projects, including such classic subjects of cost-benefit analysis as irrigation schemes, fall into this category.
The definition of a non-trivially costly project proposed by Hoehn and Randall does not exclude the possibility that changes in other components of s lead to the release of market resources. In particular it is possible that the increased market output arising from these components of the program more than offsets the non-trivial costs, so that projects have positive net market benefits. However the proof of Theorem 2 breaks down when the possibility of increased market output is taken into account.
Hoehn and Randall partition the benefits of the policy agenda into two parts. First, there are the consumption benefits of changes in the supply of non-market goods and and production impacts with trivial marginal costsı. The latter category includes all components with marginal costs algebraically less than d, and in particular all components for which d is negative (and possibly very large) so that market resources are released. The second part of the benefit evaluation consists of those components with non-trivialı marginal costs. As in the case of e-augmentability, the term trivialı is used in a rather idiosyncratic way.
Hoehn and Randall assert that the first stage benefits are bounded as in Theorem 1. However, the boundedness argument in Theorem 1 relied on the assumption that the shift under consideration was within a compact set of feasible consumption bundles. The first-stage benefits that Hoehn and Randall now evaluate include all increases in the supply of market goods associated with components of projects in the agenda, but not any reductions in the supply of market goods associated with other components of the same projects. There is no reason why these benefits should correspond to feasible values of y and a.
The fact that the proof of Theorem 2 is incorrect as stated may be seen more simply by counterexample. There is nothing in the Hoehn-Randall definition of non-trivial costs to exclude the possibility that every project in the agenda is such as to strictly expand the feasible set of market production possibilities Y(a) and also to increase the consumption benefits from the supply of non-market goods. In this case, the negative second stage benefits summed in (1) are strictly smaller than the benefits from ³production impacts with trivial marginal costs². Since perceived consumption benefits are also positive the policy agenda must have net positive benefits.
In order to make the proof of Theorem 2 valid, it is necessary to redefine the notion of a project so as to exclude the possibility that some components of policy yield increases in the supply of market goods. This can be done, for example, by combining the definition of non-trivial cost with an assumption that each proposal affects only a single component of a. With this restriction imposed, Theorem 2 shows that if all policy proposals expand the production of non-market services at the expense of market goods, then the interaction effects associated with resource scarcity will be negative and, if the agenda is sufficiently large in relation to GDP, the net benefits will also be negative. A dual argument shows that the same negative interaction will hold if all policy proposals produce additional market goods but contract the production of non-market services by some amount s*.
These results could be proved without relying on the assumption of e-augmentability, using fairly weak assumptions on consumers preferences. As long as preferences are smooth and the optimum always includes a positive quantity of both market and non-market services, the required result will follow. The crucial point, however, is that the definition of non-trivial costı includes the assumption that all policies under consideration work in the same direction (in this case, increased non-market services and decreased market commodities). Under this assumption, there are negative interaction effects.
This problem of project definition is relevant more generally to the problem of evaluating the likely impact of policy interactions. When policies which are likely to have significant interaction effects, analyzing them in combination will usually yield improvements in project design as well as project evaluation. Of course, as Hoehn and Randall note (p548) this may be difficult to achieve when the policies are under the control of separate autonomous agencies.
In summary, the analysis of Hoehn and Randall yields conditions under which interactions between projects will be predominantly negative. Theorem 1 shows that if stated benefits from IVS are sufficiently large, then aggregate benefits must be overstated. With the modified proof suggested here, Theorem 2 shows that if all proposals work to increase non-market output and reduce market output (or vice versa), benefits will be overstated and will ultimately become negative. Neither of these results, however, is sufficient warrant for the conclusion that ³too many proposals pass the benefit-cost test.²
Hoehn, John P. and Randall, Alan,Too Many Proposals Pass the Benefit Cost Testı, American Economic Review June 1989, 79, 544-551.
 I am indebted to a referee for suggesting this presentation of the argument.
 As is suggested by the species preservation example given by Hoehn and Randall, some procedures used in benefit cost analysis may yield benefit estimates which are highly dependent on the specification of the goods space. But this is a problem with these procedures, not an inherent feature of the IVS method.
 Any other convergent series will yield a similar counter-example. If correct, the argument used by Hoehn and Randall would prove that all infinite series diverge.