John List’s $10 million crazy idea field experiment in education

Michael Giberson

Bloomberg Markets Magazine has a feature on economist John List and his $10 million research project on education. Along the way we get an introduction to List’s work on field experiments in economics, a splash of lab-based economics back story, and the reaction of education specialists who think List’s project is wholly off target.

List, along with collaborators Steven Levitt and Roland Fryer, has obtained a $10 grant for a program which randomly assigned 3-5 year old students to one of three groups: (1) free all-day preschool, (2) “parenting academy” for the student’s parent or guardian, or (3) a control group with neither intervention. The program intends for follow the students into adulthood in order to assess the long-term effects of the intervention.

List says he doesn’t know much about education theory, so he enlisted specialists to consult on the preschool curriculum. One such consultant, Clancy Blair, a New York University professor of applied psychology, says he was astonished by the size of the project and by how it focuses on financial incentives without looking at such variables as how the parents interact with their children.

“That’s a crazy idea,” says Blair, who studies how young children learn. “It’s not based on any prior research. This isn’t the incremental process of science. It’s ‘I have a crazy idea and I convinced someone to give me $10 million.’”

List says too many decisions in fields from education to business to philanthropy are made without any scientific basis. Without experimenting, you can’t evaluate whether a program is effective, he says.

“We need hundreds of experiments going on at once all over the country,” he says. “Then we can understand what works and what doesn’t.” …

“What educators need to know are what are the best ways to educate kids, and this is trying to short-circuit that,” Blair says. “We have fundamental problems in education, and this is sort of a distraction.”

List says he understands the objections. “If I was in the field, I’d hate me, too,” List says in November while driving to his sons’ indoor baseball practice in one of Chicago’s south suburbs. “There should be skeptics.”

Ronald Coase interview

Michael Giberson

Interview with Ronald Coase, on the occasion of the establishment of the Coase China Society, an effort to stimulate study and application of Coase’s ideas in China. Interview conducted by Wang Ning, a student of Coase’s now teaching at Arizona State University and co-author with Coase of the book How China Became Capitalist.

HT to Paul Walker of Anti-Dismal.

A few selections illustrating Coase’s views on Coasean economics, experimentation and institutional reform, and where Hayek had a good point:

WN: You mentioned many times that you do not like the term, “Coasean economics”, and prefer to call it simply the “right economics” or “good economics”.  What separates the good from bad, the right from wrong?

RC: The bad or wrong economics is what I called the “blackboard economics”. It does not study the real world economy. Instead, its efforts are on an imaginary world that exists only in the mind of economists, for example, the zero-transaction cost world.

Ideas and imaginations are terribly important in economic research or any pursuit of science. But the subject of study has to be real.

[...]

WN: The second question many Chinese have in mind for you is, what you think other countries can learn from the Chinese experience of market transformation? Is there any general lesson to be learned from the China model?

RC: I don’t know. You don’t know what you can learn until you try to learn.

WN: I think this point is critically important. If I understood correctly, you are saying that learning from China or any other example is not like learning from a book or cooking recipe, but more like learning by doing. If the Chinese economic reform is an experiment, learning from China remains an experiment. Different countries will learn different things even if they learn from the same model.

RC: Exactly. What we do is all experiment.

[...]

RC: Nothing guarantees success. Given human fallibility, we are bound to make mistakes all the time.

WN: So the question is how we can learn from experiments at minimal cost. Or, how could we structure our economy and society in such a way that collective learning can be facilitated at a bearable price?

RC: That’s right. Hayek made a good point that knowledge was diffused in society and that made central planning impossible.

WN: The diffusion of knowledge creates another social problem: conflict between competing ideas. To my knowledge, only people fight for ideas (religious or ideological), only people are willing to die for their ideas. The animal world might be bloody and uncivilized. But animals, as far as we know, do not fight over ideas.

RC:  That’s probably right. That’s why we need a market for ideas. Ideas can compete; people with different ideas do not need to slaughter each other.

Jamie Oliver, children and food, and field experiments

Lynne Kiesling

Several years ago Jamie Oliver set out to improve school food for a group of British children. In part he was motivated by wanting to impart a love of good, healthy food in children by sharing his own joy in food, and in part he believed that healthier school meals would lead to less obesity and better academic performance. As Tim Harford noted in his Undercover Economist column in November:

Oliver’s mission to persuade schools to serve healthier lunches – and get children to eat them, and stubborn mothers not to stuff chips through the school railings – became a national phenomenon in 2005. Tony Blair and David Cameron fell over themselves to jump on the Naked Chef’s bandwagon, and soon everyone in the country had an opinion on the campaign.

Harford then discusses a working paper by Michèle Belot and Jonathan James that uses Jamie Oliver’s school lunch work as a field experiment; from Belot’s web site:

“Healthy School meals and Educational Achievements”, with Jonathan James

Children’s diet is a major source of preoccupation in many developed countries. The concerns have mainly been focused on the implications for obesity and health outcomes. However, the effects of children’s poor diet may extend beyond health; food is an obvious input in the “learning production function” and deficiencies in diet may result in important deficiencies in nutrients playing an essential role in cognitive development. This study exploits a unique experiment in the UK, the “Feed Me Better Campaign” where the meals served in the 81 schools of one area (Greenwich) were changed drastically by the British Chef Jamie Oliver. Because the campaign was literally designed as a large-scale experiment, it offers a unique opportunity to assess the causal effects of healthier food on educational outcomes. We find that educational outcomes did improve in Englsih [sic] and Science, although we cannot rule out small effects. We also find that the campaign reduced absenteeism by 15% .

A well-designed field experiment in economic policy is hard to achieve, particularly because to get a representative distribution among treatments you have to randomize who participates in what treatment, which is politically difficult (I can tell you stories about this in electricity, but it will require a cocktail). This experiment does not randomize, but it does provide a large-scale test with comparative demographics that minimize the selection bias problem. As Harford notes,

The chef had convinced Greenwich’s council and schools to change menus to fit his scheme; he mobilised resources, provided equipment and trained dinner ladies. Other London boroughs with similar demographics received none of these advantages – and indeed, because the programme wasn’t broadcast until after the project was well under way, probably knew little about it. The result was a credible pilot project. It wasn’t quite up to the gold standard of a randomised trial, but it wasn’t far off. …

… Surely what counts is that a new idea was tried out on a respectable scale, and now we have a chance to figure out whether it worked. What astonishes me is that it took a television company and a celebrity chef to carry out a proper policy experiment.

Oliver’s work has led to his receiving a TED Prize for 2010, drawing attention to his work to improve diet, food knowledge and understanding, and cooking for children, focusing on the UK and US. His TED address from a couple of weeks ago is well worth a listen; he does go a bit histrionic for my taste in some parts of it, but his passion for incorporating knowledge about food into education is obvious. And, he knows how to design a policy experiment.

Economic experiments on growth policy

Lynne Kiesling

Tim Kane at the Kauffman Foundation is going to do some experimental economics research (with Dan Houser at George Mason) on some questions of growth and entrepreneurship:

The experiments are for the study of work and entrepreneurial behavior under different policy regimes, including taxation, welfare, and social insurance. I’ll share a link to the preliminary literature review / experimental design working paper in a future post. Questions on my mind are:

1. Will higher federal taxes reduce entrepreneurship and growth, or are those fears overblown?
2. What’s the best way to design an experiment for work under different policy regimes? Once we nail that down, what is the best way to design the choice of riskless versus risky (entrepreneurial) work?
3. Is there research already in existence informing this topic?

Tim’s post summarizes some of that literature and provides some insights into why he thinks an experimental approach will expand our understanding of these questions. I look forward to seeing this research.

Experiments in business

Michael Giberson

“The level of experimentation is abysmal,” says Prof List. “These firms do not take full advantage of feedback opportunities they’re presented with. After seeing example after example, we sat down and said, ‘We have to try to do something to stop this.’ One change we could make is to teach 75 to 100 of the best MBA students in the world how to think about feedback opportunities and how to think about designing their own field experiments to learn something that can make their company better.”

From a story in the Financial Times about a class by Steven Levitt and John List, “Using experiments in firms,” taught in the business school at the University of Chicago.

“We’re on a proselytising mission of bringing a different way of thinking,” says Prof Levitt.“We’re trying to bring about a revolution in business….”

(HT to Steven Levitt at Freakonomics)