Better Fact Checking than in Science


Magazines and newspapers often have better fact checking than scientific journals. Not just a little bit better, but way better.

Check out this John McPhee article from the 9 February 2009 issue of The New Yorker:

The article goes into amazing detail about fact checking at The New Yorker magazine. In particular, it shows a great sensitivity to implications of punctuation and grammatical decisions.

I wasn’t the only one struck by this article. Check out Peter Norvig’s response, which covers more of the details:

at the Language Log:

and some related posts by Andrew:

It also invoked comparisons to software:

Of course, so many more people read The New Yorker or the The New York Times than a scientific paper, that there are lots of blogs about where their fact-checking went wrong.

2 Responses to “Better Fact Checking than in Science”

  1. curious Says:

    thoughts from someone who has logged a few years fact-checking for a sciency magazine:

    It’s true that journals/peer review are not a great system for checking facts. But with a scientific paper, the writer is [supposedly] submitting text based on his/her own data and observations — i.e., s/he *is* the “primary source.” There’s no one else who can check that it’s correct without repeating the experiment. So in lab research, the “fact-checking” happens after publication, when other labs try to replicate and extend the work.

    In checking science magazine articles intended for a general audience, we sometimes rely on research papers as sources — the fact-checker aims to confirm that we’re accurately reporting the content of a paper. It’s a different context and a different type of process, but I don’t know that I would describe it as better. In both cases, it’s only as good as the source.

    Re: the NYer piece, I find it funny/suspicious that they’re running an article about fact-checking from a *writer’s* perspective. (To a fact-checker, writers are those lazy slobs we have to clean up after!) It would be much more revealing to hear from the checkers. :o)

  2. lingpipe Says:

    That’s an excellent point about experimental data.

    The problem I see with scientific reviewing is that there’s not even the primary fact checking with citations or that the math is consistent. You very often see other theories completely misrepresented (not necessarily on purpose). And certainly not the same attention to grammar and other linguistic issues, which actually matter just as much in science. I know this is supposed to be up to reviewers, but they’re volunteers often doing a cursory job of reviewing. A contributing factor to the problem is that you need specialists to do this kind of reviewing, and we’re all too busy being lazy slob writers to do a good job on the reviewing side.

    I’d love to hear from the fact checkers. I found the copy-editor for my first book (Cambridge Uni Press) did an amazing job of making sure my notation and references were consistent. She found inconsistencies in claims I made across dozens of pages. My second book (MIT Press) and journal submissions were only lightly copy-edited.

Leave a Reply to curious Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: