Quiggin, J. (1991), Too many proposals
pass the benefitcost test: commentı, *American Economic Review* 81(5), 14469.

TOO MANY PROPOSALS PASS THE BENEFIT COST TEST - COMMENT

by

John Quiggin

University of Maryland

September 1989

Abstract

*In a recent paper, Hoehn and Randall have argued that, because of
a failure to take into account interactions between programs, 'too many
proposals pass the benefit cost test'*. *In this note it is argued that the
first of Hoehn and Randallıs results is unrelated to policy interactions, and
simply imposes an upper bound on aggregate benefits. The second result is
incorrect as it stands, and may be corrected only with a more restrictive
definition of a policy proposal.*

* *

*I would like to thank Bob Chambers, Darrell Hueth, Richard Just, Erik Lichtenberg and Ted McConnell and several anonymous referees for helpful comments and criticism. Responsibility for errors remains my own.

TOO MANY PROPOSALS PASS THE BENEFIT COST TEST - COMMENT

In a recent paper, John Hoehn and Alan Randall argue that conventional benefit cost outcomes are systematically biased upwards because of interactions between policy components which are not taken into account in standard benefit cost analysis. Further, they argue that, if the costs of policy change are non-trivial, then the agenda as a whole will produce negative benefits as it becomes large. The purpose of this note is to show that the mathematical model proposed by Hoehn and Randall is essentially unrelated to the supporting verbal argument.

The first result is actually implied by the apparently innocuous
technical assumptions of the model and has no necessary connection with policy
interaction. In essence, the
result is that, if aggregate estimates exceed some upper bound imposed by the
finite nature of the economy then they must be over-estimates. The second
result is invalid as stated, and can be rescued only by imposing strong
assumptions about the nature of the policy program. Policy proposals must be defined to ensure that interaction
effects are always of one sign. Thus, contrary to Hoehn and Randall, it is not possible to determine *a
priori* whether too many proposals pass the
cost-benefit test, or whether the public sector overprovides non-market
services.

The framework adopted by Hoehn and Randall involves a division
between market and non-market goods.
Government policy is assumed to affect the supply of non-market goods,
specified as a vector ** s** = (

The first result claimed by Hoehn and Randall states that IVS
necessarily leads to overvaluation of the policy agenda as it becomes
large. This result turns on a
simple numerical argument which applies equally in the presence of any source
of error in benefit cost evaluation, or indeed in any testing process. In essence, the argument runs as
follows. Assume that the evaluation
process is such that there always exists at least one additional policy
proposal having estimated benefits greater than some positive value *e *(in the terminology of Hoehn
and Randall, the policy environment is *e*-augmentable). Since the welfare of society is
bounded, it cannot be improved by more than some finite number *o(a,*dfo 3()* ^{-})*. Then, if there are more than

The crucial step in this argument is the assumption of *e*-augmentability. The symbol *e* is normally used
for an arbitrarily small positive number. Similarly, Hoehn and Randall (p.548) give the following informal
definition for *e*-augmentability:

³A policy environment is described as *e*-augmentable if it
is always possible to find at least one more policy component that appears to
be beneficial when evaluated independently²

which implies that *e* may take any positive value.
However the number of separate components of policy, *n*, is bounded above by *K*. Hence, for the argument to work it is
necessary that *e* > *o(a,*dfo 3()* ^{-})*/

The theorem simply states that if there are sufficiently many projects yielding sufficiently large benefits, then total benefits must be overstated. Thus, even for large agendas, it is an empirical question whether the IVS procedure leads to an over-estimate of benefits.

In the formal model presented by Hoehn and Randall, *K* is fixed, but some of their discussion implies that *K*, and hence *n*,* *may be made arbitrarily large. One way of doing this would be to
assume that evaluation takes place separately for individual proposals, rather
than for individual components of ** s**. In this case, the number of proposals
could be unbounded and

A second possibility would be to consider a finer specifications of the goods space, thereby yielding an increase in K and a reduction in the lower bound for e. However, since the benefits associated with any single-component policy would decline commensurately with the reduction in e, this modification would have no substantive effect[2].

Hoehn and Randall
assume that the only source of error in the IVS procedure is the failure to
consider interaction effects.
Other sources of error are excluded by assumption. However, for the
purposes of the mathematical argument, based on *e*-augmentability, any
other source of error could have been specified in place of interaction
effects. Assume that interaction
effects are taken into account, but costs and benefits are measured with error,
due to technological uncertainty or some other source. The error process may be unbiased, or
biased towards either overstatement or understatement of net benefits.

The boundedness of total welfare implies that, for e > *o(a,*dfo 3()* ^{-})*/

Theorem 1 is driven entirely by boundedness, and the assumption of e-augmentability. It does not depend on interactions between policy components or on any other specific source of error.

The second
Hoehn-Randall result is that, when policy change is non-trivially costly, IVS
states the benefits of the policy agenda as positive when in fact they are
negative. The argument here is
more closely tied to a benefit cost framework. The framework adopted by Hoehn and Randall involves a
division between market and non-market goods. Government policy proposals are assumed to change the net
supply of non-market goods while consuming at least *s *in market goods,
evaluated at the initial starting point. Convexity in preferences ensures that the combined
cost of *g* proposals is at least as great as the cost of the proposals assessed
separately and hence is at least *g**s* . This amount is unbounded as *g* grows large, and hence must eventually outweigh the bounded
benefits of the proposals. A project, that is, a shift from *a*^{k}^{-1} to *a** ^{k}*, is
defined as non-trivially costly if at least one component of

It should first be noted that there is a technical error in Hoehn and Randallıs proof. They assert that the sum

(1) ** I**su(_{i}_{=1},* ^{r}*,
) -MC

is unbounded as *r* grows large. The
argument is that since MC_{j}_{ } *d* and *a*_{j}_{ }-o(*a**,*dfo 3()^{~})* _{j}* is finite there is a

However, a more critical point in this case is the definition of a proposal.[4] In their presentation of the model framework in Section I, Hoehn and Randall use a very general specification of the notion of policy proposals. In particular, they explicitly (p545) include the case when non-market services are used as inputs into production of market goods. Many government projects, including such classic subjects of cost-benefit analysis as irrigation schemes, fall into this category.

The definition of a
non-trivially costly project proposed by Hoehn and Randall does not exclude the
possibility that changes in other components of ** s** lead to the release of market resources. In particular it is possible that the increased market
output arising from these components of the program more than offsets the
non-trivial costs, so that projects have positive net market benefits. However the proof of Theorem 2
breaks down when the possibility of increased market output is taken into
account.

Hoehn and Randall partition the benefits of the policy agenda into
two parts. First, there are the consumption benefits of changes in the supply
of non-market goods and and production impacts with trivial marginal costsı.
The latter category includes all components with marginal costs algebraically
less than *d*, and in particular all components for which *d* is
negative (and possibly very large) so that market resources are released. The
second part of the benefit evaluation consists of those components with
non-trivialı marginal costs. As in the case of *e*-augmentability, the
term trivialı is used in a rather idiosyncratic way.

Hoehn and Randall assert that the first stage benefits are bounded
as in Theorem 1. However, the boundedness argument in Theorem 1 relied on the
assumption that the shift under consideration was within a compact set of
feasible consumption bundles. The first-stage benefits that Hoehn and Randall
now evaluate include all increases in the supply of market goods associated
with components of projects in the agenda, but not any reductions in the supply
of market goods associated with other components of the same projects. There is
no reason why these benefits should correspond to feasible values of ** y** and

The fact that the proof of Theorem 2 is incorrect as stated may be
seen more simply by counterexample. There is nothing in the Hoehn-Randall
definition of non-trivial costs to exclude the possibility that every project
in the agenda is such as to strictly expand the feasible set of market
production possibilities ** Y**(

In order to make the proof of Theorem 2 valid, it is necessary to
redefine the notion of a project so as to exclude the possibility that some
components of policy yield increases in the supply of market goods. This can be done, for example, by
combining the definition of non-trivial cost with an assumption that each
proposal affects only a single component of ** a**. With this restriction imposed, Theorem
2 shows that if all policy proposals expand the production of non-market
services at the expense of market goods, then the interaction effects
associated with resource scarcity will be negative and, if the agenda is
sufficiently large in relation to GDP, the net benefits will also be negative. A dual argument shows that the same
negative interaction will hold if all policy proposals produce additional
market goods but contract the production of non-market services by some amount

These results could be proved without relying on the assumption of *e*-augmentability,
using fairly weak assumptions on consumers preferences. As long as preferences are smooth and
the optimum always includes a positive quantity of both market and non-market
services, the required result will follow. The crucial point, however, is that the definition of
non-trivial costı includes the assumption that all policies under
consideration work in the same direction (in this case, increased non-market
services and decreased market commodities). Under this assumption, there are negative interaction effects.

This problem of project definition is relevant more generally to the problem of evaluating the likely impact of policy interactions. When policies which are likely to have significant interaction effects, analyzing them in combination will usually yield improvements in project design as well as project evaluation. Of course, as Hoehn and Randall note (p548) this may be difficult to achieve when the policies are under the control of separate autonomous agencies.

In summary, the analysis of Hoehn and Randall yields conditions
under which interactions between projects will be predominantly negative. Theorem 1 shows that if stated
benefits from IVS are sufficiently large, then aggregate benefits must be
overstated. With the modified
proof suggested here, Theorem 2 shows that if all proposals work to increase
non-market output and reduce market output (or *vice versa*), benefits will be overstated and will ultimately become
negative. Neither of these
results, however, is sufficient warrant for the conclusion that ³too many
proposals pass the benefit-cost test.²

REFERENCE

**Hoehn, John P. and Randall, Alan**,Too
Many Proposals Pass the Benefit Cost Testı, *American Economic Review* June 1989, *79*, 544-551.

[1] I am indebted to a referee for suggesting this presentation of the argument.

[2] As is suggested by the species preservation example given by Hoehn and Randall, some procedures used in benefit cost analysis may yield benefit estimates which are highly dependent on the specification of the goods space. But this is a problem with these procedures, not an inherent feature of the IVS method.

[3] Any other convergent series will yield a similar counter-example. If correct, the argument used by Hoehn and Randall would prove that all infinite series diverge.

[4] I am indebted to a referee
for suggesting this way of viewing the problem.