Michael Giberson
President Obama has created the “SAVE Award,” a process by which federal employees can submit ideas for “how their agency can save money and perform better.” A committee of OMB officials will review the submissions and submit a short list to the President, and the President will pick the winner. The federal employee with the winning suggestion will get to meet the President and have the idea included in the government’s next budget.
NewsFutures, a “wisdom of crowds” system vendor, suggests on the company blog that the SAVE Award approach just scratches the surface of the possible benefits of crowdsourcing innovation. Here is their diagnosis and recommendation:
What’s wrong with this process?
There is no aggregation of the collective wisdom. The crowd is called on only to submit ideas, not to help evaluate them, which is the critical step in the delicate wisdom of crowd recipe. Instead, a bunch of political appointees in the OMB will decide which ideas get on the short list that the President will see. Needless to say, we would not dare propose such a simplistic process to our enterprise clients looking to harvest and select innovative ideas from their employees.
Here’s how it should be done: After everyone has proposed their ideas, ask everyone to bet (not vote) on which idea the President will ultimately select – or alternatively, on which ideas will be short-listed by the OMB officials. The crowd’s betting will quickly and efficiently produce a ranking of the best-to-worst ideas, and the result will be less arbitrary than what you can expect form a close-knit group of bureaucrats. Now, it doesn’t mean that the OMB officials cannot have the ultimate say on what ends up on the short list that goes to the Oval Office, but their choice is at least informed by the crowd’s aggregate choice.
Let’s ask NewsFutures’s question again: “What’s wrong with this process?”
Since the crowd is being asked to bet on which item(s) will be chosen, not which one would actually produce the best productivity improvement, the betting will not “quickly and efficiently produce a ranking of the best-to-worst ideas.” Instead, it will produce a ranking of what the bettors believe is most likely to be selected by the OMB and President.
If the short list was actually determined by the betting rather than independently selected by the OMB, the process takes on elements of a Keynesian beauty contest. No longer would bettors be betting on which ideas would be best, nor which idea among all submissions is most likely to be chosen by the President. Instead, the winning bettors would be those that best predicted which idea would be chosen from among those ideas most heavily favored by other bettors.
Rather than aggregating collective intelligence to identify the best idea, the process instead becomes one in which, in Keynes’ words, “we devote our intelligences to anticipating what average opinion expects the average opinion to be.”
UPDATE: Jed Christiansen’s post, Starting from the wrong metaphor – Prediction Markets and Ideas, was written a few weeks ago in response to another ideas market operation. His comments seem relevant to the issues here.
(HT to Chris Masse at Midas Oracle)
Pingback: Can a Keynesian beauty contest improve Pres. Obama’s suggestion box for U.S government employees? | Midas Oracle .ORG
Read again, Michael, we do not propose that the short list be determined by the betting, so your “Keynesian beauty contest” objection is irrelevant in this case.
In our design, the bettors would be betting on what the OMB advisers would decide to put on the short list, presumably according to the particular intrinsic and perceived merits of each idea in regard to pre-specified requirements (listed on the SAVE Award home page). A bettor’s task in this case is to divine the ultimate selection of the advisers, not the preference of other bettors. The ranking produced by the betting – quickly and efficiently – just serves to inform the decision makers and make their selection task easier.
In the Private sector, this design works beautifully. There’s no reason why it would not work for the Administration as well.
The above was cross posted at Midas Oracle, and a discussion has popped up in the comments there. I’ve responded to Emile’s comment at Midas Oracle and invite the interested reader to follow that discussion.
In brief, I agree that as proposed the mechanism isn’t exactly a K. beauty contest. But to the extent that OMB’s short list choices are “informed by the crowd’s aggregate choice,” then the process takes on a K. beauty contest aspect.
A more complete response is, as noted, in the comments at Midas Oracle.