data for technorati (please ignore if you are a human)

6V6FNC2MPMF4

Truth or dare: on the pain of not being a relativist

Sometimes I wish I was a relativist.

If I were a relativist (or whatever fancy name they have now) then I don't think I would have tied myself into such epistemological knots as I did just a few hours ago. I was doing a bit of web research for my previous blog post on academic publishing and Web 2.0. Specifically I was trying to find out about arXiv (the document sharing platform used by physicists, mathematicians and the like), even more specifically I was researching the claims that some physicists (including Nobel Prize winners) were blocked simply because of who they were rather than the content of their articles. Terrible stuff.

One of the articles I referred to (by Frank Tipler) consisted on an excoriating attack on the weakness of the peer reviewing process, arguing that (1) nowadays 'genius' papers are likely to be reviewed by 'stupid' (his words) people, and (2) some topics will be dismissed out of hand because they go against current scientific orthodoxy. I found myself nodding in half agreement at these arguments while expressing a certain caution at his choice of words which tended to be rather bellicose.

Then I read on.

It turns out that one of the topics he believed was off limits was intelligent design as espoused by Michael Behe and William Dembski and argued that these folk should have a voice. "OK" I thought, maybe he has a point. Researching Tipler a little further it turns out that he has a pretty glittering career in mathematical physics (Nature and Science publications). Then I read this in Wikipedia:

"In his controversial 1994 book The Physics of Immortality,[4][5][6] Tipler claims to provide a mechanism for immortality and the resurrection of the dead, consistent with the known laws of physics, provided by a computer intelligence he terms the Omega Point and which he identifies with God. The line of argument is that the evolution of intelligent species will enable scientific progress to grow exponentially, eventually enabling control over the universe even on the largest possible scale."

Err....

Apparently, however, his views were supported (to some extent, at least) by David Deutsch the parallel universes guy who is pretty well respected. But then some of Deutsch's ideas can be a little left field as well. But then, isn't all theoretical physics left field nowadays?

Tipler's article is fascinating but problematical for four reasons. The first I have already dealt with above, should I believe the opinions of someone who believes what appears to me to be crackpot ideas? The second concerns the fact that the article is of uncertain provenance. Rather undermining the argument of my previous post I kept asking myself "was this peer reviewed". My suspiscions were further aroused by the fact that (third reason) there was no reference section and (fourth reason) it contained typos. Surely in the title

"Refereed Journals: Do They Insure Quality or Enforce Orthodoxy?"

"Insure" should be "Ensure", no? (OK, I guess he could have used either but "ensure" seems more traditional). Further, physicist Max Planck is referred to at one point as"Man Planck".

I do typos too of course (I'm sure you're aware of this, as doubtless there are some in here) but this is an opinion piece, dashed off, rather than a deeply considered piece of writing. The more serious a piece is the more typos matter.

Typos and the like aside, theoretical physics messes with people's heads because it relies on fiddling around with mathematics until it tells you something. The great thing about doing this is that it can lead you to some really surprising predictions (e.g. the quantum indeterminacies that underpin the Schrödinger's cat thought experiment) unfortunately psychology seldom avails itself of such mathematical reasoning which is possibly why most of its theories (if not its data) are almost indistinguishable from common sense. Maths does that, it's not that these guys necessarily believe their theories. This kind of jiggery pokery leaves belief far behind; the maths tells them that it must be so, even if what it is telling them is weirder than the worst acid trip. Physicists are in this way as much a slave to their equations as the "computer says no" benefits operative. Of course the other way of doing it is to simply start with a random belief, that God is made from cheese, say, and prove this as an ineluctable fact by similar mathematical jiggering and pokering -- which approach Tipler used is hard to judge, though I have my money on the latter.

It's not just theoretical physics, though, determining the truth is a tricky task. In many ways science makes things easier (no, really) because it provides (more or less) an agreed-upon framework for testing hypotheses. And in much of my own domain -- psychology -- I can usually make some kind of a judgement as to whether a particular hypothesis is supported or not by the data by examining the results sections of academic papers. But on some of the stuff, I haven't a clue. I've tried reading some of the stuff on game theory -- the really heavy mathematical stuff-- and I'm just not equipped to judge. Likewise theoretical physics, likewise pretty much anything outside my narrow domain of expertise.

So what do I do? I do what everyone does; I rely on (a) authority and (b) consensus.

For (a) I happen to have a few folk whose views I happen to hold in high esteem. I know Richard Dawkins isn't everyone's cup of tea, but I have a deep seated admiration for his singlemindedness, his powers of explanation and (sharp intake of breath) his humility (honestly). (I think I also like him because his voice reminds me of Oliver Postgate of Clangers and Bagpus fame, which is why I think Charlie Brooker -- whose views I also admire but not on issues such as these -- describes Dawkins as "looking and sounding exactly like Professor Yaffle” the aforementioned bookend, carved into the shape of a woodpecker was voiced by Postgate.) The philosopher Daniel Dennett is someone else whose opinions I will take seriously. I don't blindly follow them, of course, but in certain areas I will follow them somewhat myopically.

For (b) well everyone does this don't they, at least in some areas? And don't we keep hearing with reference to global warming about the 'scientific consensus'? Well if consensus was what mattered the scientific consensus 60 years ago was that plate tectonics (or continental drift as it was known then) was nonsense leaving its founder (Alfred Wegener) an object of ridicule among the scientific community. Not that I am a climate change denier*, of course, just to point out that one era's consensus is another era's pseudoscience (phlogiston anyone?).

So here I am in an epistemological knot. Not knowing what to believe. If I were a relativist I would be untroubled, if there's no such thing as the truth then there's no need to be concerned when I can't lay an easy hand on it. But is anyone a relativist, really? I had a colleague, a Sociologist, who used to refer to himself as a "nine-to-five relativist". Relativism was his day-to-day stock in trade, he wrote papers about it, used it as a interpretive framework for his academic research which was on the social construction of learning in the planarian flatworm [!] (he also smoked a pipe). But when he was driving home and saw a red light he would put his foot on the brake: traffic signals might be socially constructed but he clearly wasn't going to put his life on the line testing his own world view.

He might have been a nine-to-five relativist but I’m a 24/7 realist and as a result the truth always bothers me, whether it’s the true location of my door keys or more arcane philosophical truths. The truth hurts, that’s for sure, but its absence hurts even more.

*The word "denier" is a funny one. If you look it up it most commonly refers to a measurement of textiles. Female readers will be most familiar with it as a measure of the density of what used to be called 'hosiery' in the department stores of my childhood. With this interpretation I advance a new product with the following strapline "climate change denier: tights that keep your legs cool as the world heats up."

Academic publishing and Web 2.0

Which of the following is true and which is false?

(1) An academic article is only as good as the journal in which it is published

(2) A journal is only as good as the academic articles that it publishes

Of the two (1) seems to me to be obviously false. Of course researchers and their research can gain a kudos for being published in a high-impact factor journal (Science, Nature, and so on) but it is that "only as good as" that sticks in the craw: there are independent factors that contribute to the quality of a piece of research other than where it is published. Naturally there will be a high correlation between some independent assessment of "research quality" and the impact factor of the journal in which it is published but the correlation will not be perfect (there are doubtless very good papers published, for whatever reason, in lower impact factor journals, and doubtless also some dross published in the "good" ones).

So now let's examine statement (2), this seems to me, at least, to be entirely true, at least in the long term. If the editors of Nature, say, started to suddenly publish low grade research then pretty soon fewer people would read it, it would thus have less influence and its impact factor would tumble.

But Nature, Science, Cell and the like are unlikely to start publishing rubbish so what am I talking about, where is this thought experiment going?

It has seemed to me for quite a while that the whole nature of academic publishing is the wrong way round. Having written up their experiment(s) researchers will usually strive to get the paper in the highest impact factor journal they can given their discipline, topic area, methodology and the like. This 'aim high' strategy sometimes works, but often the paper will be rejected (either before or after review) and the researchers will then move down the "quality" ladder until a journal accepts the paper (or they give up!).

But this seems wrong because, as the answer to the above conundrum seems to suggest ultimately journals have more to gain from accepting good articles than researchers have from publishing in good journals. So it is the journals that should be soliciting high-quality articles from the researchers rather than the researchers going cap-in-hand to the journals. (Note that I am using "should" in an ideal world kind of way here, rather than referring in a real world kind of way -- more of which later.)

This seems to happen in some scientific disciplines. I was interested to read here a story concerning the first experiments conducted on the Large Hadron Collider (LHC). The data were collected on Monday 23rd of November 2009, the paper written up by the following Saturday, and three days after this it was accepted by the European Journal of Physics. Now, it has to be said that the impact factor of this journal is not very high, about 1.7 compared to around 30 for Nature and Science, read into that what you will, but the point I am making is that it took just over a week to go from laboratory to "in press" -- unbelievable if you compare it to psychology (my field) where the same process is likely to take a year or more. (This assuming that it is accepted by the first journal with only minor revisions required by the reviewers.)

How did this happen? Well physicists can upload papers to a server called arXiv (pronounced "archive" as the X is supposed to be the Greek letter Chi) where it is moderated by other physicists which can lead to the authors revising the manuscript (or sometimes the moderators get in on the act of revisions too). Whatever, the process is much more rapid than the glacial act of peer review. How did the European Journal of Physics get in on the act? Well the article doesn't say but the implication is that the editors visited arXiv and decided to publish the article. Why? Because for a relatively lowly journal picking up on the first data to come out of the LHC will gain it a great deal of publicity which may, in the long term, lead to greater influence subscriptions, money and the like.

This is exactly the process that we've seen in other industries such as popular music. In the old days (pre-internet, I mean) a band would scrape together some cash to record a demo tape which they would send to the A&R department of various record companies in the hope that one of them would give it a listen, like it and sign them. This may still happen, but many artists and record companies are forgoing this process. The band puts their music on Myspace or wherever and waits for the record companies to find them.

The world has changed but academic publishing is still in the era of cassette tapes and jiffy bags. It is actually worse than this. Pre-web 2.0 musicians could submit their cassettes to as many record companies as they liked to maximise their chances of getting heard and maybe hoping that they could stimulate a bidding war if more than one company was interested in signing them. When you submit an article to a journal you have to sign a form (electronic, thankfully) stating that your manuscript has not been and will not be submitted to another journal: the journal has exclusive rights to review your paper.

Do we want bidding wars between journals? Won't that harm science in the long run? Maybe, but I guess the future is a world without journals as we understand them today. Quite a few influential papers are 'published' in arXiv and never end up in a journal. But, you might argue, if these articles haven't been peer reviewed how can we guarantee academic quality? Well, you can't of course, but then you never could. I will only refer you again to Alan Sokal's paper that was accepted for publication in a high-profile discourse journal despite being peer reviewed and despite being deliberate nonsense and to this interesting if occasionally borderline unhinged article by physicist Frank J. Tippler and move on. It seems to me that the community will provide far better checks and balances on academic quality than three anonymous reviewers who only (usually) get one bite at the cherry.

ArXiv isn't perfect, and there have been some claims that the administrators have blacklisted some scientists from publishing on arXiv simply because they have expressed views that run counter to current scientific dogma; but such problems should be relatively easy to solve by for example, expanding the number and diversity of administrators, or by having papers be submitted anonymously in the first instance ensuring acceptance is based upon quality of research rather than on judgements made ad hominem. (My feeling is that this also happens in traditional journals, btw, as well as its opposite: low quality articles gaining acceptance simply as a result of their being authored by someone with a lot of intellectual clout.)

If we as social scientists want our research to be truly current, not two or more years out of date then we need something like arXiv, academia needs to catch up with the Web 2.0 revolution.