Cass Sunstein on Regulatory Analysis and the Knowledge Problem

Cass Sunstein begins:

With respect to the past and future of regulation, there are two truly indispensable ideas. Unfortunately, they are in serious tension with one another. Potential solutions lie in three reforms, all connected with democracy itself – but perhaps not quite in the way that most people think.

The first indispensable idea is that it is immensely important to measure, both in advance and on a continuing basis, the effects of regulation of social welfare. As an empirical matter, what are the human consequences of regulatory requirements? That is the right question to ask, but inside and outside of government, it is tempting to focus on other things. […]

At the present time, the right way to answer the right question is to try to identify the costs and benefits of regulations, in order to catalogue and to compare the various consequences, and to help make sensible tradeoffs. To be sure, cost-benefit analysis can create serious challenges, and at the present time, it is hardly a perfect tool. Nonetheless, it is the best one we now have. Some people do not love cost-benefit analysis, but they should. If we want to know about the real-world effects of regulation, that form of analysis deserves a lot of love.

The second idea, attributable above all to Friedrich Hayek, is that knowledge is widely dispersed in society. As Hayek and his followers emphasize, governments planners cannot possibly know what individuals know, simply because they lack that dispersed knowledge. The multiple failures of plans, and the omnipresence of unintended consequences, can be attributed, in large part, to the absence of relevant information. Hayek was particularly concerned about socialist-style planning. He contended that even if socialist planners are well-motivated and if the public interest is their true concern, they will fail, because they will not know enough to succeed. Hayek celebrated the price system as a “marvel,” not for any mystical reason, but because it can aggregate dispersed information, and do so in a way that permits rapid adjustment to changing circumstances, values, and tastes.

In sum, while it is immensely important to measure the effects of regulation, we may lack the necessary information.

Sunstein notes, pragmatically, that regulators have access to substantial amounts of information and methods of cost-benefit analysis are well-established and improving. He wrote, “In the day-to-day life of cost-benefit analysis, regulators are hardly making a stab in the dark.” He continues:

Nonetheless, modern followers of Hayek are correct to emphasize what they call “the knowledge problem,” understood as the government’s potential ignorance, which can be a problem for contemporary regulators of all kinds […]

The tension, in short, is that regulators have to focus on costs and benefits (the first indispensable idea), but they will sometimes lack the information that would enable them to make accurate assessments (the second indispensable idea). … In light of the knowledge problem, can they produce reliable cost-benefit analyses, or any other kind of projection of the human consequences of what they seek to do, and of potential alternatives?

Sunstein identified three reforms he said respond to the first indispensable idea while helping overcome or mitigate the limits imposed by the knowledge problem: First, modern “notice-and-comment” rulemaking processes; second, retrospective analysis of regulations; and third, careful experiments and especially randomized control trials. As pragmatic responses to knowledge problems, each of the three have something to contribute.

Is it enough?

What would Hayek say? Sunstein responded to this question in a footnote:

I am not suggesting that Hayek himself would be satisfied. Consider this remarkable passage:

This is, perhaps, also the point where I should briefly mention the fact that the sort of knowledge with which I have been concerned is knowledge of the kind which by its nature cannot enter into statistics and therefore cannot be conveyed to any central authority in statistical form. The statistics which such a central authority would have to use would have to be arrived at precisely by abstracting from minor differences between the things, by lumping together, as resources of one kind, items which differ as regards location, quality, and other particulars, in a way which may be very significant for the specific decision. It follows from this that central planning based on statistical information by its nature cannot take direct account of these circumstances of time and place and that the central planner will have to find some way or other in which the decisions depending on them can be left to the “man on the spot.”

Hayek [“The Use of Knowledge in Society,” AER, 1946]. In my view, the claim here is a mystification, at least as applied to the regulatory context. Statistical information “by its nature” can indeed “take direct account of these circumstances of time and place.” Of course it is true that for some purposes and activities, statistical knowledge is inadequate.

It is an odd note.

Sunstein quoted Hayek, said Hayek’s point is a mystification, then admitted, “Of course it is true….” I guess Sunstein’s point is that it is a true mystery?

In any case, we should note few things about Sunstein’s reforms. First, his focus is not so much on the doing of regulation itself, but rather on regulatory oversight. His three reforms are not to be applied in day-to-day regulation, but rather serve as external correctives. A related and perhaps more fundamental issue concerns the manner in which regulatory operations can facilitate the production and coordination of knowledge in ways that promote better outcomes.

Second, extending this last point, as we compare government and market processes, we note the relative power of feedback processes in the private sector and the weakness of feedback processes in the public sector. If you are dissatisfied with your service in a restaurant, you can take choose to eat elsewhere next time. If you are dissatisfied with your service at the Department of Motor Vehicles (or the Veterans Administration or the local Zoning Board, etc.), you have many fewer options. Feedback processes are essential to the production and coordination of knowledge. How can regulatory agencies learn in an environment in which feedback processes are so significantly attenuated?

Third, Sunstein is operating with a rather flat view of knowledge. That is to say, in his explanation knowledge and information and data are various names for more or less identical things. But if we take seriously Sunstein’s remark “of course it is true that for some purposes and activities, statistical knowledge is inadequate,” then further questions arise. Sunstein does not explore the point, but it is exactly here, for Hayek, that the force of the knowledge problem emerges.

There is a research agenda here.

Obviously Sunstein endorses further research and development of benefit-cost analysis, expansion of notice-and-comment rulemaking processes, retrospective regulatory analysis, and experimental tests of regulation. Benefit-cost analysis has a dedicated academic society and journal and book level treatments. Sunstein discusses efforts to broaden understanding of regulatory proposals and encourage public engagement (for example, the government’s www.regulations.gov and the university-based outreach project regulationroom.org). Retrospective regulatory analysis and experimental tests get less attention, but a number of academic programs do at least some retrospective work (for example: the Mercatus Center Regulatory Studies Program at GMU, the Penn Program on Regulation, the George Washington University Regulatory Studies Center). As Sunstein notes, a number of federal agencies have committed to using experiments to help understand the impact of regulation.

How far can these efforts get us in the face of the knowledge problem? For which regulatory “purposes and activities” is it the case that “statistical knowledge is inadequate”? Are there patterns in this inadequacy that bias or undermine regulatory action? Assuming Sunstein’s reforms are fully implemented, what residual knowledge problems would continue to trouble regulation?

A good place to start is Lynne Kiesling’s article “Knowledge Problem” (obviously!) which appeared in the 2014 volume Oxford Handbook of Austrian Economics. Mark Pennington’s book Robust Political Economy examines how knowledge problems and incentive problems can frustrate political activity. Obviously, too, Hayek’s own “The Use of Knowledge in Society” and “Economics and Knowledge” are relevant, as is the later “Competition as a Discovery Process.” Don Lavoie’s “The Market as a Procedure for Discovery and Conveyance of Inarticulate Knowledge” condenses Hayek’s statements on the knowledge problem and further explains why the problem cannot be overcome merely by further developments in computer technology.

Going deeper, one might explore James Scott’s book Seeing Like a State, which emphasizes how the top-down processes of measurement and aggregation can strip meaning from knowledge and result in destruction of value. Then perhaps a work like Hans Christian Garmann Johnsen’s new book The New Natural Resource: Knowledge Development, Society and Economics might have something to say. (I’ve only just begun looking at Johnsen’s ambitious book, so it is too soon to judge.) Complementary to work on institutional frameworks and knowledge would be close studies of government agencies, like the Pressman and Wildavsky classic Implementation: How Great Expectations in Washington are Dashed in Oakland… and broader surveys of policy history such as Michael Graetz’s The End of Energy or Peter Grossman’s U.S. Energy Policy and the Pursuit of Failure.

And more.

Here I have focused just on those parts of the Sunstein article where he bumps up against the knowledge problem most explicitly. He explores each of the three reforms in more detail, providing much more to think about.

Recommended reading for regulatory analysts.

NOTE:

Sunstein’s paper is “Cost-Benefit Analysis and the Knowledge Problem.” Regulatory Policy Program Working Paper RPP-2015-03. Cambridge, MA: Mossavar-Rahmani Center for Business and Government, Harvard Kennedy School, Harvard University.