Scientists Who Fake Reviews of Their Papers

The pressure to publish is being blamed for the latest uptick in cases of scientific fraud and the appearance of new and devious means of getting one's work in print. An ongoing case involves a South Korean scientist who submitted glowing reviews of his own submitted papers (The Chronicle of Higher Education).

Hyung-In Moon submitted a manuscript to the Journal of Enzyme Inhibition and Medicinal Chemistry along with the names and email addresses of potential reviewers. So far, so good....many journals allow, or even require, such a list of reviewers by authors. The associate editor who handled the paper sent it out for review but became suspicious when they came back within 48 hours (those of us who have served as associate editors will recognize that the odds of two reviewers responding so quickly to be about the same as winning the recent Mega-Powerball lottery). The editor then sent the paper to two reviewers of his choosing; however, they recommended publication with some revisions, so the paper was published in 2010.

Apparently flush with success, Moon continued to submit manuscripts to JEIMC over the next year. According to an interview by the Chronicle, the associate editor who handled Moon's initial paper was still suspicious and took a closer look at the recommendations for reviewers. Although the referees appeared to be real scientists, the email addresses provided by Moon were commercial (g-mail, yahoo) rather than institutional. The editor finally wised up and contacted the reviewers directly. Not surprisingly, the email addresses provided were not theirs, and they had never heard of Moon.

Turns out, Moon had set up fake email addresses so that he could receive the journal's request for a review, and then it was a simple matter to send in a glowing review of his own work. When questioned, he apparently admitted this chicanery to the editor as well as to fabricating some of his data (!). The ensuing investigation has resulted in the retraction of his papers from various journals. According to Retraction Watch, 35 papers authored by Moon have been retracted.

Here is part of the retraction notice for the papers that were published in JEIMC:

"The peer-review process for all of the above articles was found to have been compromised and inappropriately influenced by the corresponding author, Professor HI Moon. As a result the findings and conclusions of these articles cannot be relied upon.

The corresponding author and the publisher wish to retract these papers to preserve the integrity of material published in the journal. The publisher acknowledges that the integrity of the peer review process should have been subject to more rigorous verification to ensure the reviews provided were genuine and impartial. The publisher apologizes for any inconvenience rendered to the readers of the journal and wishes to assure the reader that measures have been taken to ensure that the peer review process is comprehensively checked to avoid a similar error occurring."

Another journal editor-in-chief (Immunopharmacology and Immunotoxicology) has stepped down after 20 papers he handled by Moon were retracted.

To add insult to injury, Moon has suggested ways to avoid such problems in the future:

"There is nothing wrong with soliciting reviewers from authors, as long as there are some checks. Of course, authors will ask for their friends, but Editors are supposed to check they are not from the same institution or coauthors on previous papers. I know so many journals ask for *potential* reviewers, which they then add to a database of reviewers for the field the submission was made in. They then send the paper for review to other people on that database."  

He points out that it's the responsibility of the editor and the journal to check the feasibility of potential reviewers. It seems that he saw a loophole and took advantage of it....so we shouldn't be so hard on him. 

When I was an associate editor, I never used the reviewers suggested by the corresponding author unless I was pretty certain that the suggested reviewer had no conflict with the author (and in those cases always solicited another review from someone of my choosing). I could not imagine being comfortable accepting two reviews by people suggested by any author. It may take a bit more effort, but it's relatively easy to identify appropriate reviewers from the paper's list of citations, the journal's database, and an internet search of the paper's topic.

And accepting without question the names and addresses of suggested reviewers from an author is just plain foolish.

How I Learned to Stop Worrying about Being an Impostor

 This post is a continuation of the previous discussion of the "impostor phenomenon" and the "Dunning-Kruger effect". To review, people suffering from the former condition don't believe themselves to be qualified, despite ample evidence as to their skills and qualifications, and worry about being uncovered as an impostor.  The latter condition is characterized by over-confidence, completely out of proportion to actual skills (the greater the incompetence, the higher the confidence).  For more details, see the previous two posts. 

You can also test yourself as to your "impostor" tendency here.

In this post, I'd like to explore how someone with a tendency toward "impostor phenomenon" (IP) might learn to avoid such negative feelings and associated behavior.  The basic problem, according to psychologists, is that such people fail to internalize their accomplishments and skills, and so view themselves as less capable than others.  I can imagine that this condition might arise through various personality traits and environmental factors during childhood.  One particular interaction that occurs to me is how much a person depends on the approval and affirmation by others--too much reliance on accolades from others might predispose someone to the condition, for example. 

Even if you are thinking that you don't have this problem, consider this: our self-image changes over time and in response to our surroundings and how others treat us.  One thing is certain in life, and that is change.  I think most people can remember one time or another when they felt inadequate and were worried about being able to accomplish a task, do well on an exam, or live up to expectations (of parents, advisers, employers). I pointed out in the previous post that students are susceptible to IP upon being thrust from school into a job where they are expected to be fully competent (after years of being told they are not yet competent).  I experienced this feeling upon completing my Ph.D., but it faded with time.  I can imagine this could strike anyone at any point in their career--going from one job to a new job, for example.

There may be a couple of ways to deal with IP--both of which helped me.

I mentioned above the reliance on others for affirmation of one's skills.  Therein lies one possible source of help: surround yourself with people who are supportive, who reaffirm a positive image of yourself, and who encourage you to view yourself realistically (by pointing out your obvious skills and accomplishments when you express doubts, for example).

Obviously, it would be better if you were not reliant on others to avoid feeling like a fraud, but it's not likely to be easy for you to change this aspect of your personality without extensive therapy.  However, you can avoid people who make you feel worse and seek out people who are supportive.  However, that will likely get you only part-way to squashing any impostor feelings.

Another method that I found helpful was to remind myself often about my own accomplishments. I've read about accomplished scientists (with IP) who would frequently reread one of their best papers to remind themselves just how good it was (therefore, how good they are).  People with IP tend to be highly accomplished, so having such reminders is usually not a problem.  I've also done this on occasion when I was feeling wobbly about starting a new and challenging writing project. By reading something I had previously written, which was published in a good journal, helped me internalize the fact that if I was capable of doing it once, I could do it again--no sweat. Anyone can use this approach --with a paper they've written, an exam they aced, or a great blog post they wrote that got a lot of positive comments!

For more on stories of female scientists suffering from IP and strategies for overcoming IP, see this article at Naturejobs.

Image credit: modified photo from davcorp.com

Beyond the Imposter Syndrome

I came across a blog post at "On Becoming a Domestic and Laboratory Goddess" in which the blogger, Isis, announces that the "imposter syndrome" is a scam.  In case you've not heard of it, this condition is one in which the person secretly feels like an imposter in their field, despite being highly competent and successful.  They are afraid that eventually everyone will find out that they are not a real scientist. Graduate students (in general) and women in STEM fields seem to suffer disproportionately from this problem.

Anyway, Isis expresses frustration over how many women are moaning about suffering from this condition. I think the concern here is that women will attribute any failure (to land a grant, to publish a paper) to this syndrome and this behavior will, consequently, enhance the belief that they are imposters.  I also suspect that those who have read or written a lot about the imposter syndrome have become weary of it and think that there is nothing new to be learned.

However, I think there is more to be understood about imposter syndrome and its opposite, the Dunning-Kruger effect, which I think of as the "overinflated ego syndrome".  The Dunning-Kruger effect is one in which the person overestimates their abilities...and the more incompetent they are, the higher their level of confidence in their abilities. At first glance, it seems incomprehensible that people who constantly make mistakes or fail to meet performance expectations would paradoxically believe that they are competent or skilled. However, the mental skills required to be competent at any task (e.g., ability to assess what steps to take to accomplish the task, to gauge standards of performance) are the same skills needed to accurately evaluate one's own competence.

Kruger and Dunning summarize the key features of such people:

1. They tend to overestimate their own level of skill;
2. They fail to recognize genuine skill in others;
3. They fail to recognize the extremity of their inadequacy

Thus, people who are generally incompetent are unable to gauge their own performance correctly and tend to over-inflate their performance (in their minds) relative to the performance of others.  Ehrlinger et al. provide the following example:

"..to produce a grammatically correct sentence, one must know something about the rules of grammar. But one must also have an adequate knowledge of the rules of grammar in order to recognize when a sentence is grammatically correct, whether written by one's self or by another person. Thus, those who lack grammatical expertise are not in a position to accurately judge the quality of their attempts or the attempts of other people. In addition, because people tend to choose the responses they think are most reasonable, people with deficits are likely to believe they are doing quite well even when they are, in reality, doing quite poorly."

Sufferers of the imposter syndrome are just the opposite.  In psychological terms, imposter syndrome is a condition in which the sufferer is not able to internalize his or her accomplishments and skills.  They are highly critical of themselves and tend to over-inflate the performance of those around them.  If we use graduate students as an example, we can see the dynamics involved in creating the imposter mindset.  Students are in a subordinate position from virtually every standpoint. They interact with people who are obviously competent and superior (professors) and who constantly tell the student (in various ways) that they are not yet competent at the skills required to succeed in their field of study.  When the student is thrust into a position of responsibility (they graduate and get a job), they may find it difficult to transition from the "incompetent" to the "totally competent" mindset.  They may go on to develop "imposter syndrome".

These two syndromes are neatly summarized in one of my favorite sayings: "Ignorance more frequently begets confidence than does knowledge." Charles Darwin.

Of course, these are two extremes along a wide spectrum of mental states. Most of us fall somewhere in the middle, perhaps with more women toward the imposter end and more men toward the inflated ego end (I have no data, just more than 40 years of observations). We may shift between these mental states over time, depending on our history and new situations.  I recall feeling like an imposter right after attaining my Ph.D., but this faded with time and experience. I can envision someone taking on a job in a new field and initially feeling like an imposter...at least until some experience in the new position is gained.  You can probably think of many other examples.

So what does all this have to do with women in science and does this information lead to any insights as to how we may solve some of the "challenges" we face?  More in the next post.

Constructive Criticism III

This post continues my musings about criticism. We are now at the final stage of maturation in the development from novice to mature scientist in terms of accepting and providing criticism:

Late Career (full professor, senior scientist and beyond)

As I’ve tried to convey in the previous two posts, these transitions in one’s perception of and reaction to criticism accompany the change from a position in which one only receives criticism (e.g., a student) to someone who additionally serves as a critic (e.g., advisor, reviewer, editor). As a scientist’s experiences as a critic expand, her viewpoint regarding critiques of her own work begin to shift. At least, that’s the hope.

In addition to a better understanding of the critic’s viewpoint, the mature scientist is more informed about scientific standards in one’s field, expectations of reviewers and editors, and the reality of scientific “politics”. The mature scientist may develop a “thicker skin” to buffer against the vagaries of reviewer comments, particularly the harsher ones that border on (or are) personal attacks. This dermal protection may be enhanced by the recognition that vigorous criticism of one’s work sometimes indicates professional jealousy or a deep-seated inferiority complex on the part of the reviewer. As this week’s quote (side panel) suggests, it’s far better to be the subject of criticism than to be ignored. But that sentiment is small consolation when your work has been shredded by another.

It’s also useful to keep in mind that it’s the reviewer’s job to try to poke holes in your work. You actually want them to do this, because they will often identify weak points that you will then be able to address before publication. Some critics are a bit more enthusiastic in this regard than others. A few clearly have an ax to grind (either with you or what your work represents). Others are too lenient or lazy.

It goes without saying that it’s far better to receive constructive, rather than destructive, criticism. Even the most inexperienced person can recognize the difference—and the motives—behind these two types of criticism. Most people, upon receiving a critique, can readily see whether the comments are meant to help or to harm. Some reviews are a mixture of the two.

The goal for the author is to deal with each type of comment effectively. The mature author also knows that she cannot depend upon the editor to make the distinction between helpful and harmful comments (or act upon this knowledge). So she must learn to respond with finesse rather than with emotion. I realize this is far easier said, than done.

However, keep the goal in mind: getting your paper past the goal-keepers.

The mature author is not interested in “getting back at” the reviewers by ridiculing their remarks. Instead, she calmly explains where the reviewers may have gone wrong in their assessments (perhaps acknowledging that this may have been partly the author’s fault due to awkward wording) and then offers a well-worded explanation. She also honestly thanks the reviewers for being so thorough, so forthright, or whatever the case may be.

If the reviewer has written a mostly personal attack with no documentation to back up outrageous statements, then the mature author simply restates what her findings have shown and that in the absence of any contradictory evidence (which the reviewer has failed to provide), she has no alternative except to ignore the reviewer’s criticism (or provide only minimal response). By keeping her cool and refraining from outraged reactions to criticism, the mature scientist conveys a demeanor of professionalism and confidence. She doesn’t have to say that the reviewer is a jerk—it will be obvious.

In the end, the mature author derives much greater pleasure from out-maneuvering the mean-spirited critic than by reacting emotionally to a personal attack.

The mature scientist also spends considerable time critiquing others’ work--as advisor, reviewer, and/or editor. Giving constructive criticism is difficult to do well, however.  If you read articles about constructive criticism, they invariably focus on the technique of sandwiching the criticism between two compliments in an effort to disarm the recipient and avoid a defensive reaction. This method rarely works because it’s so transparent. Although a few skilled critics might pull it off, most people clumsily apply this technique, which then fails or backfires. This approach, which is akin to giving someone a bitter pill hidden in a piece of cake, is not how I define constructive criticism.

For criticism to be constructive (i.e., helping to improve), it must do more than simply compliment the author on some aspect of the work. The criticism should involve suggestions that will substantively improve the work. An example might be a suggestion to better display the results in a figure or table—an explicit description of how to do it, not just the statement that the change is needed. Or the suggestion might be a reorganization of information to better develop a logical argument—again, with explicit instructions as to how this might be accomplished. This all takes effort on the part of the critic, which is one reason it’s not often done well. Most critics will say that something needs improvement and then leave the author to figure out how to do it.

How much constructive criticism one gives is determined by a number of factors: time, motivation, etc. As I’ve discussed previously, it’s not always advisable to spend a lot of time on recommendations for revising a manuscript when the science is fatally flawed. If that’s the case, then it’s best to tell the authors that the study needs to be repeated or substantially augmented. The mature scientist resists the impulse to spend more time than necessary on such reviews. In other words, why waste time correcting flawed English, typos, and other editing problems when the manuscript is obviously not publishable? Copy editors will catch most, if not all, typos and other errors in an accepted manuscript. So the scientific reviewer is of most help in assessing the science and its significance. If you can provide one or two good suggestions for improvement, then you’ve likely helped the authors more than most reviewers.

Of course, not all authors (especially experienced ones) are grateful for explicit suggestions for improvement, so your efforts may be for naught. However, this should not deter you (in your role as constructive critic) from trying. I’ve found that in figuring out a way to improve someone else’s paper, I’ve added to my own toolbox.

Well, those are a few thoughts on constructive criticism in science, not by any means an exhaustive examination of the topic—and admittedly, a very personal view.  I’m sure others will have different viewpoints and experiences.