Monday, September 27, 2010

Writing Reviews

A couple of weeks back an interesting discussion was had (well, it appears to be over) at DrugMonkey's Scientopia page, on the issue of citation practices. I chimed in with my own practice ... I don't give a rats patooie what journal the manuscript is in, as long as it was good science. And that would have been it, had not someone uttered something I found completely unfathomable.
I do not have adequate time to do all of the fact-checking myself for every reference in my 200-reference review article.
Excuse me? I responded here ...
Say what? You include references into an article without fact-checking them yourself? I can understand going through the abstract, skimming the R&D, and reading the conclusions … but if you can’t be bothered to read the damn article, why are you citing it?!
... and here ...
Yah … I’m following up my own reply. If by fact-checking you mean subsequent articles which support and/or refute said article … IMNSHO, if it’s important enough to cite, then it’s important enough to know the history behind the article and how it was received IN THE FIELD YOU’RE WRITING A REVIEW ARTICLE IN. If you don’t know the history behind the literature in said field, let someone else write the review article.
I also said ...
What I said is that if you’re in a field you should at least know the relevant literature, especially if you’re writing a review on said literature. Cold-citing an article? Piss poor.
I continue to stand by those comments. Why would anyone cite an article without taking a look at and understanding said article? How would you even know where to begin, to properly cite the article, when you have no idea what was contained within the manuscript and/or how it was perceived post-publication? Taking someone else's word for it? Sure, but how do you know they got it right?

I was damn sure then, and I remain damn sure now, that if you can't bother to pick up the manuscript yourself, read at the very least the Abstract, Results & Discussion, and Conclusions (if your journal has them) yourself ... skim them at the very least ... you have no business reporting on that article in your own article/review. I don't care if you have one week to write the article, or one year. You should have at least a passing familiarity with the work in order to cite it.

But hey, I was just being argumentative, right?

But wait, it gets better. I then find a blog entry on the discussion. Stumbled across it really. It does its very best to completely misconstrue the entire point of my comments over at DM's blog ...
There have been implications on the interwebz that I totally cannot do a good job of this if I am a non-expert and if I do not spend infinity bajillion hours researching Mango Skin Allergies. That I will do the world of science a serious injustice if I do not provide the very best reference for every point in my article. That it takes a real expert to know the field, the history of the field, to put all of the confusing shit in the context of the field, appreciate the field, take the field out for some lobster, and try to put the moves on the field after dinner.

Bulls**t indeed Candid Enginner. Bulls**t that for someone with a graduate degree your reading comprehension is abysmal. No one, not me, nor anyone else at DM's blog stated that you must spend countless hours on the review (though a week really is crap IMNSHO) or that you must provide the very best reference, or that you must be a real expert to know the field, or know every minute detail of the history of field. I did say that ... you should understand the damn papers you are going to cite! And yes, you should also know how those papers relate to the field. Now, since she didn't clarify what she meant by "fact check" I suppose we can't truly understand what she was trying to say, but honestly I think a fairly reasonable conclusion is that "fact checking" meant (at least in part) how the manuscript fits in the "big picture" of the entire field.

To which I say ... if you personally don't know how the manuscript truly fits into the big picture of the field (because you didn't check for yourself, because you're busy writing super-important-review) ... how the heck are you writing a review on it?


Now, I'm willing to be educated on this issue. It's possible that I may have overlooked an argument which really makes sense on this issue, but so far I haven't. Not really.

Oh, and by the way ... when writing a review, it helps to keep the people who are going to take credit/blame on said review in the loop. Otherwise you wind up wasting tons of time. Telling a field they don't know how to do their work in the field is going to make so many enemies for your PostDoc adviser that it's irresponsible to even consider writing such a treatise. It's one thing to play fast and loose with your own career. It should be a crime to play fast and loose with someone elses.


microbiologist xx said...

On the subject of references:
I have some serious pet peeves about referencing. I always check papers before I reference b/c I do not want to perpetually reference bad papers or bad references. For example, one of my grad school lab competitors regularly would publish a paper where they would properly reference someone whose work they were mentioning, then they would reference themselves (referencing that work) for all subsequent publications. So of course, their papers got more references, even though it wasn't their original work or even building on the original work.

I am not saying everyone is this bad, but this lab did this ALL the fucking time. It pissed me off to the point that I check all references, if for no other reason than to make sure that I don't perpetuate that kind of BS.

Tom said...

Yes, I've seen more than a couple instances of that. It's damn annoying, especially when you need something from the original and you need to backtrack several layers to get to it.

I've also seen one citation issue which should make anyone hesitate about cold-citing an article. I was reading a paper and came across an observation that was PERTINENT to my own work. They cited the article (which ran somewhat counter to this papers observations ... but which matched mine ... at least in the way it read). I ran to the literature and pulled up the PDF. The manuscript cited had nothing to do with the issue it was attributed to in the article. I eventually wrote the authors asking them if there had been an error in the citations, and they told me that indeed that had happened. They pointed me in the direction of the proper article (which proved valuable to me). If I had cold-cited the article though (and not fact checked), and just gone on their word ... I would have propagated the error ... and looked like a moron (at least more of one) when it was all sussed out.

Jess said...

Here here! I always make the effort too. In fact, it is exactly what I am about to do today for a Review Article I am writing.

So many times people cite things that don't exist... frustrating.

Loving your blog too. ;)

Tom said...

Jess, exactly. And thanks. I've moved, so check out my new digs!