Posts Tagged ‘knowledge problem’

h1

RTO forward capacity markets are unlikely to succeed

April 6, 2012

Michael Giberson

The Gulf Coast Power Association meetings earlier this week included a debate over the future of resource adequacy within the ERCOT power system. Debate moderator Eric Schubert, BP Energy Company, introduced the issue with a critique of capacity market structures that is heavy on its reliance on Hayek’s knowledge problem. It is a topic dear to our heart here at the Knowledge Problem blog, so I thought we’d share a bit of it.

Here’s Schubert:

Hayek’s “Knowledge Problem” and its optimal solution – decentralized commercial markets – provide the best lens for regulators to see the fundamental issue in electricity market design in response to rapid technological change and increasingly diverse groups of buyers and sellers. As the procurement and use of electricity cross a complexity threshold, as a few customer classes are transformed into a multitude of individual market participants, the electricity market design needs to move away from centralized planning to a decentralized procurement of resources, to be both sustainable and efficient.

… In trying to adapt the centralized forward capacity mechanism to changing market and technological conditions, regulators and RTOs play a never-ending game of “whack-a-mole” because they can never overcome the “Knowledge Problem.” Even worse, under centralized procurement or any type of explicit “top-down” procurement of new resources mandated by regulators, unintended consequences of centralized procurements arise at the speed of markets and are corrected at the speed of administrative law.

Both long-standing economic theory and recent economic practice suggest that centralized forward capacity mechanisms are very unlikely to succeed. If they fail, state and federal regulators in the US will be forced to choose between the two known solutions to the “Knowledge Problem”:

  • A return to full integrated resource planning conducted by regulators, or
  • A move to fully decentralized wholesale and retail markets where individual customers make their own choices.

… Integrated resource planning, however, solves the “Knowledge Problem” by suppressing it. Having regulators in charge of integrating 21st century technologies would prevent consumers, retailers, and other market participants from using their local knowledge and ingenuity to find the next killer app or great idea that would provide all of us cleaner, more efficient and thoughtful use of energy. Or put another way, would we even have I-Phones today if regulators had never broken up Ma Bell?

[The alternative approach is that] with the proper price signals, buyers and sellers in ERCOT’s energy-only market will procure and manage sufficient resources to meet their individual needs and preferences while keeping the market resource adequate. Such decentralization of decision-making is the most efficient solution to the “Knowledge Problem.” The challenge of this path, however, is keeping the lights on during the transition; none of us can fully understand at this moment how the integration of the new technologies will happen, and what new ways of doing business and managing electricity use will spontaneously emerge over time.

By the way, in that first ellipsis I excised a reference to and quote from a great paper by Kenneth Rose that takes a detailed look at RTO capacity market structures.

Schubert’s full introduction is publicly available on the GCPA website, but only for the next month or so. Get it while it’s hot (and available).

h1

New paper: Knowledge Problem

February 9, 2012

Lynne Kiesling

I have a new paper that may be of interest to KP readers, since the subject of the paper is the same as the name of this site: Knowledge Problem. I am honored to have been invited to contribute this paper to the forthcoming Oxford Encyclopedia of Austrian Economics (Peter Boettke and Chris Coyne, eds.). Here’s the abstract:

Hayek’s (1945) elaboration of the difficulty of aggregating diffuse private knowledge is the best-known articulation of the knowledge problem, and is an example of the difficulty of coordinating individual plans and choices in the ubiquitous and unavoidable presence of dispersed, private, subjective knowledge; prices communicate some of this private knowledge and thus serve as knowledge surrogates. The knowledge problem has a deep provenance in economics and epistemology. Subsequent scholars have also developed the knowledge problem in various directions, and have applied it to areas such as robust political economy. In fact, the knowledge problem is a deep epistemological challenge, one with which several scholars in the Austrian tradition have grappled. This essay analyzes the development of the knowledge problem in its two main categories: the complexity knowledge problem (coordination in the face of diffuse private knowledge) and the contextual knowledge problem (some knowledge relevant to such coordination does not exist outside of the market context). It also provides an overview of the development of the knowledge problem as a concept that has both complexity and epistemic dimensions, the knowledge problemʼs relation to and differences from modern game theory and mechanism design, and its implications for institutional design and robust political economy.

In this paper I analyze the development of the two categories of the knowledge problem — the complexity knowledge problem and the contextual knowledge problem — and explore both the history of the development of these concepts and their application in robust political economy and new institutional economics. As is the hallmark of a good research project, I think on balance I learned more than I created in the process of writing this paper.

One other thing I made sure to include was a discussion of how the knowledge problem and its development relates to game theory and mechanism design, through the work of Oskar Morgenstern (and then through some of the work of Herb Simon and Vernon Smith, among others).

Tying together economics, institutional design, history of thought, and epistemology, I hope you find this paper informative and useful! I’ll also make sure to update when the full volume is available.


h1

Penn Jillette and Hayek: “I don’t know”

August 17, 2011

Lynne Kiesling

Penn Jillette, the taller and more vocal half of the magic performance duo Penn & Teller, has written a lovely and thoughtful essay as a companion to his appearance last night on Piers Morgan’s CNN show. It defies excerpting, so I encourage you to click through and read it in its entirety.

His theme: “I don’t know”, particularly with respect to religion and to helping the poor, leading him to conclude that he is an atheist libertarian. For example, about helping the poor he writes

Then he asked me what we could do to help poor people. I said I donated money, food, medical care, and services and he said, “No,” he meant, what could society do to solve the problem of poor people. Again, I was stumped.

He said the government had to do it, which I interpreted as another way of saying he didn’t know, but he thought that made me look mean … even though I do care and do try to help. …

And I don’t think anyone really knows how to help everyone. I don’t even know what’s best for me. Take my uncertainty about what’s best for me and multiply that by every combination of the over 300 million people in the United States and I have no idea what the government should do.

In this essay Jillette is channeling some of the most important ideas about the knowledge problem developed by Hayek and others, ideas that are the foundation of what we do here at KP — the limits of individual knowledge, the necessary limits of collective knowledge, and the humility that should arise as a consequence, both individually and in collective action/policy situations. Jillette is also channeling a lot of David Hume’s skepticism, not just in terms of religion but also in terms of empiricism and the limits of human reason.

Jillette’s essay is also charming in tone, reflecting respect for those who disagree with him and those with different life experiences and abilities. A very thought-provoking read.

h1

I cringe when I see Hayek’s knowledge problem wielded as a rhetorical club

April 4, 2010

Michael Giberson

The knowledge problem made the newspaper today – that’s Hayek’s concept of the knowledge problem, not the KP blog that Lynne and I operate.  But since we appreciate the significance of Hayek’s insight on the mobilization of knowledge, it seems appropriate to draw attention to Glenn Reynold’s op-ed, “Progressives can’t get past the Knowledge Problem,” appearing today in the Washington Examiner.

In his “The Use of Knowledge In Society,” Hayek explained that information about supply and demand, scarcity and abundance, wants and needs exists in no single place in any economy. The economy is simply too large and complicated for such information to be gathered together.

Any economic planner who attempts to do so will wind up hopelessly uninformed and behind the times, reacting to economic changes in a clumsy, too-late fashion and then being forced to react again to fix the problems that the previous mistakes created, leading to new problems, and so on.

Market mechanisms, like pricing, do a better job than planners because they incorporate what everyone knows indirectly through signals like price, without central planning.
Thus, no matter how deceptively simple and appealing command economy programs are, they are sure to trip up their operators, because the operators can’t possibly be smart enough to make them work.

Hayek’s insight into economics and regulation is often called “The Knowledge Problem” ….

Reynolds is giving us a simplified version of Hayek’s view (which isn’t surprising since he’s writing an op-ed for a newspaper), but mostly avoids the cartoon version that sometimes shows up in editorializing about government and the economy.  Hayek is clear in the article that his point is a comparative one: decentralized economic activity does a better job than centrally managed economic activity because it does a better job of mobilizing the knowledge relevant to the decisions that need to be made.  Reynolds gets the point right, “Market mechanisms … do a better job.”  Not perfect, just better.

Hayek says that the issue isn’t one of planning vs. not planning, but rather a question of who does the planning: “whether planning is to be done centrally, by one authority for the whole economic system, or is to be divided among many individuals.”  Both central planners and de-central planners (i.e., individual business people) will make mistakes, but there is very good reason to believe that the aggregated errors of decentralized decision makers will be smaller than the errors of a centralized decision maker.

Hayek allows that there are different kinds of knowledge and likely different kinds of organizations or processes that would be appropriate to putting the different kinds of knowledge to best use.  Hayek’s main point in his article is that the most important knowledge with respect to economic activity is the knowledge of particular circumstances of time and place (that is to say, what resources can be found to address particular needs, what terms govern access to those resources, and which combination of resources is likely to bring about the most desired outcome).  This kind of knowledge exists in a widely dispersed and fragmented state and may not even be revealed except in cases in which people are placed in a position in which they must choose between alternatives.  As such, this kind of information is not readily rolled up into statistical summaries for easy and appropriate centralized decision making.

While Reynolds does a pretty good job of simplifying Hayek without being too simple-minded, I cringe a bit to see the knowledge problem wielded as a rhetorical club to beat upon policy proposals that the editorialist doesn’t like. Relative to, say, outright nationalization of health care services, I’d conclude that “Obamacare” preserves a lot of decentralized decision making.  Reynolds doesn’t seem to notice this not-so-subtle point.

Certainly relative to the status quo, the health care law centralizes decision making and is appropriately made subject to this Hayekian critique.  But I don’t think Hayek’s insights can quite do all of the anti-big government work that Reynolds wants. While political and intellectual elites have mostly given up the enthusiasm for outright socialism that was common when Hayek wrote his article in 1945, government is bigger and more intrusive now then it was then.  Nonetheless, big government has not collapsed under the burden of these knowledge problems; it grows, mostly with public support or at least apathy.

Opponents of Obamacare don’t need a theory that says centralized decision making always fails (and by the end of Reynold’s article we are to this cartoon version of Hayek).  Rather, such opponents need a theory that explains when government action works and when it doesn’t, and then make the case that Obamacare falls into this latter category.  Hayek’s insights into the fundamental value of decentralized economic decision making are obviously useful but not sufficient to make this case.

NOTE: Hayek’s “The Use of Knowledge in Society,” published in the American Economic Review in 1945, is available online at the Library of Economics and Liberty.

h1

What does Waxman-Markey have to say about kites?

June 8, 2009

Michael Giberson

The answer to the title question is “nothing,” according to Marc Gunther (though he admits of the difficulty of being certain when the bill is 932 pages and still in process). The kind of kites he has in mind are high-altitude solar power collectors, under development by the Makani Power company. That Waxman-Markey has nothing to say on kites is a good thing, says Gunther, since no one knows whether the idea is practical or likely to become economical.

Gunther then contrasts that silence to the many mentions of coal, cars, battery technology, financial services, wood stoves, and so on, and stumbles onto the conclusion that it all adds up to a heavy-handed attempt to manage the energy economy via central government industrial policy.

After quoting the bill’s elaborate conditions for coal plants to gain federal assistance for carbon capture and storage, and terms by which the plant’s could lose access to federal assistance, which may or may not be under an accelerated timetable, and for which the EPA might or might not grant extensions to particular non-complying generators, Gunther observes:

This is the kind of thing that worries me about Waxman-Markey in particular and about the willingness of the Obama administration more generally to manage so much of the economy—energy, autos, financial services. It’s hard, nay, impossible to know if CCS is a smart, workable and affordable technology or when, if ever, it should be deployed. Maybe kites are a better idea. Maybe not. But should the government decide?

It’s not just clean coal. As I wrote last week, Obama & Co. are ready to place bets on which companies and regions are likely to develop battery technology for electric cars.

[...more quotes on geothermal heat pumps and wood pellet stoves...]

Do you see the problem here? Just to be clear, I don’t have a position on biomass stoves and geothermal heat pumps. I just don’t think the government should have a position either.

To be sure, we need an energy revolution. … But how do we get from here to there? With clean coal? Electric-car batteries? Heat pumps? Wood stoves? Kites?

The unfortunate answer is no one knows. Not the president, not Congress, not Nobel laureate and energy secretary Chu, not venture capitalists or energy company CEOS or Fred Krupp or Jeff Immelt or Joe Romm. Our energy and climate problems are knowledge problems, too. And the best way to solve knowledge problems – which is to say the best way to spur technology change – is to create conditions that get lots of scientists and engineers to work on them, invite lots of investors to place their bets on which ones will work and then test those ideas in the marketplace. By aggregating thousands of decisions, markets will help us figure out which of today’s technologies or which ones yet to be invented get us closer to the clean energy economy we need.

Did you notice the felicitous phrase embedded twice in that last paragraph, “knowledge problems”?

[Via the Energy Collective.]

h1

Edmund Phelps explains “knowledge problem”

April 17, 2009

Michael Giberson

Occasionally we hear from readers curious about the blog name, “knowledge problem.” Edmund Phelps explains the knowledge problem in an excellent essay that appeared in the Financial Times. (Registration may be required for FT.com; the essay is also posted in full at the FT‘s Capitalism blog.)

Joseph Schumpeter’s early theory proposed that a capitalist economy is quicker to seize sudden opportunities and thus has higher productivity, thanks to capitalist culture: the zeal of capable entrepreneurs and diligence of expert bankers. But … most growth in knowledge is not science-driven. Schumpeterian ­economics – Adam Smith plus sociology – captures very little.

Friedrich Hayek offered another view in the 1930s. Any modern economy, capitalist or state-run, is a great soup of private “know-how” dispersed among the specialised participants. No one, he said, not even a state agency, could amass all the knowledge that each participant “on the spot” inevitably acquires. The state would have no idea where to invest. Only capitalism solves this “knowledge problem”.

There is much more in the essay than this brief clip reveals. In fact, the very next paragraph provides the one of the best brief explanations of Hayek’s central insight into capitalism. In addition to a little Schumpeter and a lot of Hayek, Phelps nods to David Hume and invokes some Frank Knight on uncertainty.

The whole thing is worth reading.

(HT to Greg Ransom at Taking Hayek Seriously.)

h1

Working at Wal-Mart

February 2, 2009

Lynne Kiesling

Charles Platt, a former senior writer for Wired magazine, is doing a guest blogging stint at Boing Boing right now. Yesterday he wrote an extremely interesting post about his experience as a Wal-Mart employee. His motivation started with Barbara Ehrenreich’s Nickeled and Dimed, and the extent to which it “didn’t ring true” to him. So he applied for a job to check it out.

His whole post is extremely intereresting, but one detail in particular caught my attention:

My standard equipment included a handheld bar-code scanner which revealed the in-store stock and nearest warehouse stock of every item on the shelves, and its profit margin. At the branch where I worked, all the lowest-level employees were allowed this information and were encouraged to make individual decisions about inventory. One of the secrets to Wal-Mart’s success is that it delegates many judgment calls to the sales-floor level, where employees know first-hand what sells, what doesn’t, and (most important) what customers are asking for.

In many ways, Wal-Mart internalizes the value of local, private knowledge directly into its business strategy, and does so better than any other company. They are a wonderful example of the value of using technology to harness “edge intelligence”, which is the trendy phrase for the diffuse private knowledge about which Hayek and others have written so extensively, and that inspires the name of this web site.

h1

Cass Sunstein, OIRA, and nudging

January 13, 2009

Lynne Kiesling

On Thursday President-elect Obama named law professor Cass Sunstein to head the Office of Information and Regulatory Affairs, an executive-branch office with the mission of analyzing and coordinating federal regulation. Most recently, Sunstein is known for his work with Richard Thaler on “choice architecture” and behavioral public policy, including their book Nudge.

Others have already opined about Sunstein’s appointment, including Saturday’s editorial from the Wall Street Journal and this Forbes column from Glenn Reynolds. Matt Welch helpfully aggregates several commentaries on Sunstein’s appointment in his Reason post. The consensus seems to be that Sunstein’s approach to regulation is not premised upon bureaucratic imposition and control, that his primary focus is to improve markets and competition, not to stifle them through government regulation, and that much of this is likely to be irrelevant because OIRA is a politically impotent office, so his efforts will have little effect.

Most observers will be watching to see what form Sunstein’s focus on “choice architecture” and behavioral public policy will take when and if it gets implemented. The premise of this behavioral approach to what I (and Elinor Ostrom) would call institutional design is Sunstein’s and Thaler’s so-called “libertarian paternalism”. Sunstein and Thaler draw their normative policy implications from behavioral economics research that suggests that real-world decision-making does not always result in individuals making the decisions that theoretical models suggest would be optimal, or rational. In his review of Nudge, Will Wilkinson cites some examples:

If a cafeteria puts its key lime pie a bit out of the way, fewer people will succumb to delicious temptation. If employees are not required to fill out confusing paperwork to enroll in a savings plan, more of them will enroll.

One area where this behavioral issue comes up in electricity instutional design is the design of fixed-price default service contracts, and whether the default contract should be opt-in or opt-out. Those who argue it should be opt-in base their argument on consumer inertia and status quo bias — if the heretofore-regulated incumbent is the default service provider and the default contract is opt-out, innate consumer inertia would predispose them to stay on the default contract, even if competing suppliers could now enter the market and offer services that would likely be appealing to those customers. Thus by reinforcing existing inertia and status quo bias, the opt-out default contract reduces competitive entry into new retail electricity markets, and stifles retail competition. This opt-in/opt-out design dimension is relevant for health care plans, retirement plans, any sort of contractual choice situation with a status quo option.

In some ways this approach is a refreshing change from regulation that is informed by traditional, neoclassical economic modeling of individual choice, which is devoid of any formal attention to the real cognitive constraints that individuals face daily when bombarded with information and making choices in the face of budget constraints, information constraints, and (to use the Austrian phrase for it) radical ignorance — some of what individuals need to know to make the decisions that they would in our formal theoretical models is simply unknowable. Note here that the ideas I am using originate in the work of Herb Simon on bounded rationality and the use of heuristics in individual decision-making, and of Michael Polanyi on tacit knowledge and the extent to which we know stuff that we don’t know we know, and our brains subconsciously make choices constantly, which enables us to function in our information-rich environment without being overwhelmed. This behavioral approach to institutional design takes into account two important aspects of reality:

1. Individuals do face cognitive limitations on their decision-making, and have evolved adaptation mechanisms to deal with them.

2. To live together in civil society, we do have to engage in collective action, which requires institutional design.

In other words, take people as they really are, and when we have to engage in collective action, design institutions that take into account those cognitive realities. Yes. But here’s where my skepticism about the Sunstein-Thaler libertarian paternalism kicks in. Much of what is important in public policy is shaping incentives and future behavior. If we take Simon and Polanyi, and Hayek, seriously, then it’s crucial to acknowledge that a lot of what shapes incentives and future behavior is not just unknown — it’s unknowable. We don’t know our own preferences completely — we discover them through market processes, and through the process of making decisions. We don’t know what future choice sets will look like, let alone what our preferences over those choices will be. These things are simply unknowable, and therefore designing institutions to affect those unknowable incentives and outcomes is almost always likely to end up in failure. This approach to choice architecture cannot overcome the knowledge problem. So I think the best that we can hope for from such a so-called “libertarian paternalism” approach to regulation is small, local changes in design parameters that are well-known and relatively non-controversial.

Another aspect of the challenge to this approach to institutional design is indeed the elitist-paternalist critique: how will the regulator know in every situation that Choice B actually would make the individual better off than Choice A? What makes you think that you know what is better for me than I do? Seriously? Again, the unknowability of preferences until presented with a real choice situation, even to the person possessing them, makes it highly unlikely that any third party can know that one outcome would be better for a person than another outcome. This is old territory that has been covered elsewhere, including this April post from Will Wilkinson, so I will stop here, except for one observation: just because we face cognitive limitations to decision-making, that does not necessarily mean that the ultimate outcomes of decision-making are inefficient, or irrational, or “anomalies”, to use one of Thaler’s favorite linguistic tropes. Again, taking Simon and Polanyi and Hayek seriously, we devise conscious and subconscious adaptations to these cognitive realities, we use technology to help us filter and to make decisions that better reflect our preferences as we perceive them. Who are you to say that the outcome of that process is irrational or inferior?

Another question to ask when thinking about the process of institutional design and behavioral economics: why would these “Nudge” rules be any less prone to political manipulation, lobbying, and rent-seeking than more traditional top-down institutional design? The electricity default contract is an example; the traditional “consumer advocate” community generally uses its political and advocacy power to argue that opt-in default would expose consumers to too much unwanted price volatility and too much effort to avoid that price volatility, so the default should be everyone going on a fixed-price contract unless they want something else. “Nudge”-based institutions are still institutions of collective action, and thus are still going to have the same incentives for manipulation, lobbying, and rent seeking.

Finally, while I do think that the process of institutional design, including regulatory institutional design, can benefit from thinking about these choice architecture questions, I do think the “libertarian paternalism” nomenclature does a disservice to the language. “Nudge” applied to regulatory policy is not strictly libertarian, because regulation is still ultimately grounded in coercion. The “choice architecture” language does a much better job of communicating what this idea is about — incorporating behavioral and cognitive reality into institutional design. At least the “Nudge” approach to regulatory policy is more respectful of individual autonomy and choice than the likely bureaucratic alternatives.

In devising OIRA policy I’d like to hear Sunstein invoke another of his former Chicago colleagues, Ronald Coase, and state that in promoting and facilitating open, transparent markets, the most important role of economic regulatory policy is to reduce the transaction costs that prevent private parties from engaging in mutually-beneficial exchange. That means using OIRA to evaluate the entry barriers and other transaction costs that federal regulation can create. OIRA does cover other areas of regulatory policy where the ideas of transaction costs are not as strictly relevant, but thinking in terms of reciprocal benefit, reducing transaction costs, and the alignment of, for example, environmental and economic incentives through the reduction of transaction costs and better-defined property rights transcends economic regulation.

Follow

Get every new post delivered to your Inbox.

Join 50 other followers