Michael Giberson
Last December I posted a few comments on a study by John Kwoka that examined the methods of 12 studies on electric power restructuring and found them lacking. Among other things, I said, “Kwoka’s study, funded by the American Public Power Association, did not review the recently completed [study by Scott Harvey, Bruce McConihe and Susan Pope], funded by PJM. However, the APPA did issue a news release in which it seeks to counter the claims made by Harvey et al.”
I observed that, by the standards for analysis set out by Kwoka, it was going to take APPA more than anecdotes in a news release to challenge the Harvey et al. report. APPA apparently agreed, because they commissioned Kwoka to review that report.
As it turns out, according to Kwoka’s review, the Harvey, McConihe, and Pope report also has its methodological problems. Kwoka writes:
We begin … with a summary of observations made in the report to the APPA of methodological and other issues in these evaluation studies generally. This provides a standard against which to judge … [W]e conclude that the LECG study ultimately is not a sound and convincing evaluation of electricity restructuring, and hence its conclusions should not be relied upon.
APPA is running an Electric Market Reform Initiative and has supported development of several studies of restructuring issues. While APPA is a trade association with a particular viewpoint and economic interests to defend — so these reports could be no more than lobbying briefs — in general they seem to hiring smart people to do useful work.
Kwoka’s methodological reviews are good examples. The industry and policymakers need carefully done studies to help assess the costs and benefits of electric industry restructuring. Kwoka has examined now a total of 13 “often-cited studies of electricity restructuring” and concluded that all of them are flawed. I suspect, too, that if Kwoka were to apply his exacting standards to the APPA’s own reports, they would also be found lacking. It is not always obvious to the non-specialist when these reports can or can not be relied upon, so it is useful to have a specialist do this kind of review.
It would also be useful to have a few conclusions we can rely on.
It might be helpful now for APPA and Kwoka to survey the wider universe of electricity restructuring studies and produce a list of studies that do meet his standards. Or, finding none, to make one.
These “Market Benefit” studies seem to me as exercises in futility as on of the rationalization given for a Market is “Price Transperancy”. True I understand the need for the stakeholders to assess the benefits, but a framework for reference is subjective at the best.
One of topics of contention is the price change in
Fuel Mix. Now One could use prices from Gas and other Markets to adjust for a Ceteris Paribus study, but isn’t one of the benefits of a Market is the ability to Optimize and dispatch the least cost unit(possibly based on least costly fuel)
So isn’t Lucas Critique in play, even for a post hoc analysis they propose to do.
Another fact which seems to be missing in consideration is general monetary inflation.
One gotta love these studies as muse for a weekend blog post.
Thank you
Arun Eamani
http://devilisinthedata.blogspot.com/
I think it is the case that often such policy studies – for and against – are in effect rationalizations of an author’s or organization’s prior belief, but at by going through the effort of putting out a research report you are at least trying to set out why you reach the conclusions you reach. By publishing your report you expose yourself to examination and criticism (for example, by someone talented in such matters like Kwoka), and maybe the “dialogue” results in better analysis.
Okay, so maybe I’m a bit naive.