neurotaylor

The science police and the flaky dilettantes: food gets scientists going

I recently read the following tweet, from influential science blogger Ed Yong:

If you want to look at a field that is basically noise, with virtually no signal, check out nutritional epidemiology ncbi.nlm.nih.gov/pubmed/23193004

— Ed Yong(@edyong209) November 30, 2012

My first response was, ‘Have I ever even heard of nutritional epidemiology?’ So I clicked on the link, which took me to the abstract of a study just published in the American Journal of Clinical Nutrition. It’s called ‘Is everything we eat associated with cancer? A systematic cookbook review’.

Oh, right. That kind of twaddle. Now glamorised with a long scientific name. OK, Ed, I’m with you.

Is this cheesecake even worse for you than you thought?

I should declare an interest here, incidentally. I’ve done some writing for the Institute of Food, Brain and Behaviour, a small UK research charity struggling, against the mass of assorted twaddle, to promote better research standards in the study of how food affects brain function.

My second reaction was to smile. Not that I find cancer funny, but the abstract reminded me of Ben Goldacre’s mocking Bad Science column: “You will be familiar with the Daily Mail‘s ongoing project to divide all the inanimate objects in the world into ones that either cause or prevent cancer.”

My third reaction was relief. At last, a proper, citeable paper in a reputable journal, taking on the sprouting fields of nutri-nonsense.

Then, with the emotions out of the way, I settled my limbic regions (I know, I know, we don’t believe in the limbic system any more, and that phrase should be, ‘my prefrontal cortex stepped up its inhibition’, forgive the loose semantics).

And I thought.

Specifically, I looked in hope from a further tweet from Ed, exhorting scientists to get in there ASAP and help clean up this dubious field of research.

Alas, no such tweet was forthcoming, and that set me thinking further: about science and the varying attitudes of its practitioners — attitudes which, while not themselves strictly scientific, are hugely influential. Brace yourself, because what follows is sheer speculation.

Roundheads and Cavaliers

A traditional division in research is between intuitive visionaries (brilliant, if sometimes a little flaky) and meticulous data-crunchers (also brilliant, but solid and less charismatic). I’m tempted to borrow from 1066 And All That and call the visionaries Cavaliers (wrong but wromantic) and their thorough counterparts Roundheads (right but repulsive).

But that’s unkind, since science needs both tendencies. Besides, Cavaliers are more fun, at least for outsiders, and sometimes ahead of their time rather than wrong. Roundheads may also be wrong, and I’ve met some who weren’t in the least repulsive. I’ve also met Cavaliers so far out on the fringe of academic sanity that the only recourse was to dodge and run; they desperately needed a Roundhead or several to bring them down to earth. The best scientists combine the two, holding visions in check without getting mired in the details of their data.

Explorers and Defenders

Instead, there’s another categorisation, perhaps even more fundamental, and it has to do with how people react to threats. By threats, here, I mean primarily ideological threats. (As I argued in Cruelty, I suspect that the mechanisms for dealing with dangerous beliefs have their roots in older, physical defences, but that’s by the by.)

Faced with such a challenge, some people retrench, strengthening the barriers between safe and unsafe, self and nonself. They may deny the existence of the threat, downplay its power, or use distraction. They may attack the source, or at the least treat it like an enemy, emphasising the gulf of difference between ‘us’ and ‘them’ and reaffirming their own core beliefs and values. They react, in short, by policing boundaries. These are the defenders.

Halite (rock salt) crystal, via Rob Lavinsky, iRocks.com – CC-BY-SA-3.0

Science has plenty of defenders. They see it as an enterprise whose core of knowledge is solid truth, an enterprise which will one day bring everything under its remit. Yet this towering accomplishment is continually vulnerable to infiltration and corruption by nutters, marketers, and pseudoscientific propaganda. Hard though it may be to say what counts as good science, the attempt must be made if purity is to be preserved.

Defenders, perhaps, are more likely to be Roundheads, solidly building up rigorous, trustworthy knowledge. In visual terms, they seem to see scientific knowledge as a slowly-growing crystal, fractured by revolutions and breakthroughs, but beautifully clear nonetheless.

Slime mould, via Denis Barthel / Creative Commons licence

Other people react to threats with more-or-less cautious investigation. Instead of reinforcing boundaries, they try to dissolve them — not to weaken science, but to incorporate the threat source into their own worldview. They are the Borg of science, constantly aiming to assimilate. They’re the explorers, treading where more conservative colleagues fear to. For explorers, science is multifarious and its borders inevitably vague. Overlapping ideas about scientific methods, practices and qualifications scope out the scientific domain. Knowledge content doesn’t, because it is always provisional.

Explorers may incline towards the Cavalier, dashing off hypotheses, drawn to adventures. Their image of science might be more like a slime mould, still structured, but less solid, more flexible. And perhaps not quite so beautiful, but more alive.

In the English Civil Wars, Roundheads and Cavaliers were bitter foes. In science, the vitriol is verbal. Explorers are derided as flaky dilettantes who aren’t proper scientists. Defenders are scorned as scientistic, bigoted, embarrassing would-be dictators. It’s the narcissism of small differences, since both camps love and want to help science. They just have different ways of going about it.

When scientists feel threatened

My guess, and it’s no more, is that threat responses are at the base of these different reactions to attacks on science.

Perhaps the explorers’ more malleable view of their profession means they don’t see every challenge in the apocalyptic terms that defenders can seem to? (Note to new atheists: please stop worrying about astrology, it’s not going to hurt you.)

Perhaps explorers are more open to new experiences, or less conscientious and less concerned about rigour, than defenders? (Note to people using EEG to ‘promote spiritual development’: please, just stop.)

Psychologists know a lot about threat responses. But applying such knowledge to practising scientists is controversial in itself, especially among defenders, where the concept of scientist-as-supreme-rationalist still lingers. What’s needed may be more research on scientists, as well as done by them, but more consensus-building would help: after all, explorers and defenders are on the same side.

As to who’s right, of course, that’s another question, and in general terms it’s unanswerable, because each strategy has its uses. To be open and welcoming to a powerful and determined enemy bent on destroying science looks foolish, but to treat minor skirmishes from politically weak extremists as if they herald the end of enlightenment is to risk crying wolf.

In a world where scientists are being forced closer to politics, steering a careful course through competing extremes is not going to get any easier. But dismissing one or the other as ‘not proper science’ is not going to help. We need both defenders and explorers to build consensus, and explain to others that science is neither the rigid pursuit of eternal truth, nor a label that can be stuck on anything done with fancy-looking equipment, nor any bright idea someone happens to have.

Should we improve bad science or defend ourselves against it?

If you want to look at a field that is basically noise, with virtually no signal, check out nutritional epidemiology ncbi.nlm.nih.gov/pubmed/23193004

— Ed Yong(@edyong209) November 30, 2012

So when Ed Yong dismisses nutritional epidemiology (NE) as no-signal, high-noise, the explorer in me, which may just be the oppositional defiant phenotype trying to get out, says, “OK, so let’s focus on that field and do some good research!”

After all, the crapness of many studies isn’t just down to the food industry trying to buff its profits by pumping out press releases. Not all NE work is industry-funded, any more than it’s all about cancer. At least some of the problems arise because this kind of work is seriously expensive, time-consuming, and difficult to do properly. Consider:

1. Each food may contain many physiologically active compounds, interacting with other compounds in the other foods being eaten (and, in studies, being poorly recalled, or somewhat less poorly logged in a food diary, or, rarely, analysed by dieticians). No one study can model this lot, or even log it.
2. Each human body has its own unique microbiota (whose existence is rarely if ever taken into account in NE studies) and genetic and epigenetic variety (ditto), hiking the number of variables to horrendous levels.
3. Volunteers aren’t generally keen on living the kinds of micromanaged lives essential to top-quality data for the weeks or months required. Nor are they keen on frequent blood samples. Nor are funding agencies keen on the bills for big studies.

As so often in human studies, the number of potentially relevant variables makes adequate controls impractical. So researchers resort to randomised controlled trials. But to provide enough meat for the statistical techniques, they need big numbers. And when those aren’t available, as they often aren’t, the result is low-signal, high-noise. Add in the fact that food news sells, and you have a recipe for poor science.

I’m no historian, but wasn’t chemistry like that once, back in the days of alchemy and Paracelsus and the search for the philosopher’s stone? Then explorers colonised the flakedomain, gradually taming the badlands of pseudoscience. As order was imposed, imposing it became easier, and the discipline settled into, in Kuhn’s phrase, normal science. And wow, did the rest of us gain from that.

Perhaps, if we sent out enough explorers, and made a louder case for better studies, we could tame the noise of nutritional epidemiology too.