Measuring success by how much you spent on the program: A renewable energy example

Michael Giberson

In general, in public policy analysis, you’d like to judge ultimate success or failure of a program by its net results, by actual benefits less the costs involved in achieving those benefits. Admittedly sometimes benefits are hard to measure, but ultimately the point of a policy change is to bring about some improvement in something somewhere. Ultimately it would be nice, once a program is done, to try to find and measure that improvement.

What we often get instead, however, is an attempt to infer a benefit based on the expenditures on the program: how much money was spent, how many people were employed, how many miles of ditches were dug, and so on. This is, more or less, what we see this week from the U.S. Department of Energy in the study it commissioned from the National Renewable Energy Lab on the impact of the Section 1603 Treasury Grant Program.

The Section 1603 grants were payments made to qualifying renewable power projects in lieu of those projects claiming the Investment Tax Credit or Production Tax Credit subsidies for which the projects would have otherwise qualified for. The NREL study looked at the $9.7 billion in program spending up through November 10, 2011; by the time the program ended it’s three-year run in December 31, 2011 over $11 billion in federal funds had be committed.

The DOE asked NREL to estimate the effects of the 1603 program on jobs and economic expenditures. In NREL’s report they explicitly state that their work is an estimate of “gross jobs, earnings, and economic output.” This means that they don’t consider any private sector crowding out, any disincentives from the taxation needed to support the program, any consequences from duplication of other government incentive programs, and so on. They simply treat the federal resources as if it were manna falling from the heavens, and the jobs, capital, and industries that became involved in building renewable power plants would have otherwise sat idle. (Note that I’m not criticizing NREL in performing just a piece of the overall analysis, they just did the work that DOE asked for and paid them to do.)

But note that this is primarily a study which just measures the expenses of the program and a part of what the expenditures bought. So, it is a partial study of the costs of the Section 1603 program, and not any kind of estimate of any of the benefits of the program.

Nonetheless, in the DOE press release accompanying publication of the study, they said the study found “the program has been a huge success.” How does it justify its claim of success? By noting how much was spent, how many people were employed, and how many things were subsidized by the program.

The DOE is not the only one to claim success. At Climate Progress, Stephen Lacey’s assessment is titled, “Grant Program Supported Up To 75,000 Wind And Solar Jobs: Congress Killed It Anyway.” Lacey’s post does mention some of the construction activity might have happened even without the grants, and he observes it estimates just the gross impact (and, by implication, doesn’t reflect any negative effects due to the crowding out of unsubsidized economic activity). But along the way Lacey keeps claiming the program was a success. How does he know? Well, he summarizes from the NREL report: the government spent a lot of money, hired a lot of people, and subsidized the purchase of a lot of things.

Great, but resources consumed is not a measure of success. Any fool can spend money, but spending it well can be a challenge. Is there any evidence in the NREL report that the money was well spent?

If the answer to that question is “no,” then we can’t conclude that the program was a success.

ADDITIONAL LINKS: Reactions to the NREL report from North American Windpower, Solar Industry magazine, and Clean Technica. Rep. Ed Markey (MA) cited the report in calling for Republicans to support “revisions to the tax code that level the playing field for clean energy.”


3 thoughts on “Measuring success by how much you spent on the program: A renewable energy example

  1. Good points, but I don’t think directly relevant to the 1603 program, since that program was the conversion of existing tax credits into cash grants. The net cost to the treasury is zero, so long as you have the same number of projects participating under either structure, so arguing about whether or not the benefits of 1603 were supported by the costs doesn’t parse algebraically. (A point that is also lost on the NREL report, sadly). Doing the math properly would require an assessment of how much additional project volume was driven by the shift to a cash grant, and then doing the cost benefit assessment just on that gap – e.g., you have to come up with some estimate of how much of the $11 billion in funds would have been spent if it remained a tax credit only, and just look at the net.

    I think that would likely show a positive result every time, simply because so few renewable project developers have tax appetites, and therefore the tax-credit-only approach causes some percent (typically, 10 -30%) of the total benefit to flow to banks and other tax equity sponsors rather than project developers. So if X is worth doing, the incentive to do X will flow more efficiently to the actors who cause X to happen with a cash grant rather than tax credit program.

    This isn’t to disagree with your main point of course – far too many public policy efforts (including, but not limited to clean tech) define success by how much they spend rather than the goals they accomplish. Such is the nature of bureaucracy, I suppose, where rank is determined by the size of the budget you control. Just pointing out that 1603 per se isn’t the greatest example, even though NREL has (erroneously, IMO) chosen to frame it that way.

  2. Sean, you are right that the 1603 program is a complicated one to judge, for the reasons mentioned.

    I’m ready and willing to dispose of the full tax credit-based approach to subsidizing renewable power (and pursue a pollution tax and increased R&D spending to address associated externality and public good issues).

    But for folks who are not ready to make that leap, your remarks suggest some interesting ideas. If perhaps as much as 30 percent of the subsidy is flowing to financial groups with a tax appetite, perhaps we can replace the $22 MW/h PTC with a $15 or $16 direct production subsidy. No net harm to renewable developers, just net tax savings due to a more efficient program design.

    Of course it would put the energy production subsidies more directly into the U.S. budget rather than hiding them via tax credits, and that may be a politically difficult move these days.

Comments are closed.