Michael Giberson
Last December I posted a few comments on a study by John Kwoka that examined the methods of 12 studies on electric power restructuring and found them lacking. Among other things, I said, “Kwoka’s study, funded by the American Public Power Association, did not review the recently completed [study by Scott Harvey, Bruce McConihe and Susan Pope], funded by PJM. However, the APPA did issue a news release in which it seeks to counter the claims made by Harvey et al.”
I observed that, by the standards for analysis set out by Kwoka, it was going to take APPA more than anecdotes in a news release to challenge the Harvey et al. report. APPA apparently agreed, because they commissioned Kwoka to review that report.
As it turns out, according to Kwoka’s review, the Harvey, McConihe, and Pope report also has its methodological problems. Kwoka writes:
We begin … with a summary of observations made in the report to the APPA of methodological and other issues in these evaluation studies generally. This provides a standard against which to judge … [W]e conclude that the LECG study ultimately is not a sound and convincing evaluation of electricity restructuring, and hence its conclusions should not be relied upon.
APPA is running an Electric Market Reform Initiative and has supported development of several studies of restructuring issues. While APPA is a trade association with a particular viewpoint and economic interests to defend — so these reports could be no more than lobbying briefs — in general they seem to hiring smart people to do useful work.
Kwoka’s methodological reviews are good examples. The industry and policymakers need carefully done studies to help assess the costs and benefits of electric industry restructuring. Kwoka has examined now a total of 13 “often-cited studies of electricity restructuring” and concluded that all of them are flawed. I suspect, too, that if Kwoka were to apply his exacting standards to the APPA’s own reports, they would also be found lacking. It is not always obvious to the non-specialist when these reports can or can not be relied upon, so it is useful to have a specialist do this kind of review.
It would also be useful to have a few conclusions we can rely on.
It might be helpful now for APPA and Kwoka to survey the wider universe of electricity restructuring studies and produce a list of studies that do meet his standards. Or, finding none, to make one.