Double-Blind Peer Review

Nature Geoscience recently announced they are offering the option for authors to remain anonymous to reviewers (here is a link to an editorial). Actually, both Nature Geoscience and Nature Climate Change are testing this option. The editorial that describes this new option explains that the journal's editors are convinced that this double-blind process will remove unconscious bias. In particular, the editorial states that "One of our motivations for setting up a double-blind trial is the possibility that female authors are subjected to tougher peer review than their male colleagues–a distinct possibility in view of evidence that subtle gender biases affect assessments of competence, appropriate salaries and other aspects of academic life ((Proc. Natl Acad. Sci. USA109, 16474–16479; 2012)." 

As the editorial mentions, some reviewers will attempt to guess the identity of the authors. There will be clues in the paper, no matter how careful the author is. A few reviewers will be certain they have guessed correctly (even though they likely will be flat wrong). To what extent this guessing will influence the review, is not clear. Hopefully, most reviewers will simply shrug and focus on the paper's content, which is what they should be doing anyway.

Coincidentally, I just completed a double-blind review (my first) a few days ago for a different journal. Interestingly, the journal did not explain that the paper's authors would not be revealed to me. Instead, I was provided the usual links to the paper, which contained no mention of the authors or their affiliation. Not finding that information anywhere, I finally realized that this was a double-blind review.

Did not knowing the author's identity affect how I conducted the review? Not really. In fact, I liked not knowing who the author was quite a bit more than I expected. I wonder if other reviewers will feel the same way?

Myths About Giving Presentations: Imagine Your Audience Is Naked

I've been discussing myths about giving presentations. A lot of myths deal with fear of public speaking. The recommendation to imagine your audience naked or in their underwear or sitting on the commode is meant to allay the speaker's fear of being on stage. Supposedly, a naked audience is less intimidating. Rumor has it that this idea came from Winston Churchill, who used this technique (among others) to overcome his fears of speaking in public (he also apparently recommended that people practice their speeches in front of a mirror). I could not find confirmation that Churchill was the source of this advice, but it has certainly been repeated extensively.

This is one recommendation, however, that I've never even considered following, even though it has been suggested to me on a number of occasions throughout my career. I've also heard well-meaning teachers and advisors give this advice to students. First off, it sounds incredibly silly. I have enough trouble trying to remember what I should be saying without the extra mental effort that would be required to envision the audience without their clothes on. I could see myself walking onto the stage, turning to face the audience, trying to picture a roomful of naked people, and then blanking out completely on my speech as my head filled with unsavory images of my colleagues in their birthday suits. How would I face them later and carry on a conversation with a straight face?

No, no, no. This is bad advice not only because it's disrespectful of your audience but because it puts a barrier between you and them. Just the opposite of what you should be doing: Connecting with your audience. If you are afraid or contemptuous of your audience, then you are not connecting with them. If you are not connecting, then it's likely they are not paying much attention to what you are saying. Putting up barriers between you and your audience (huddling behind a podium, reading your speech and never looking up, looking at the screen with your back to your audience) makes for a bad presentation. Audience members will understand at a gut level that you are not confident about yourself or your message. People are incredibly skilled at reading body language, even if they are not quite aware of it at an intellectual level.

Along these same lines, I've heard about another "trick" that some speakers use to avoid looking into the audience members' eyes (but appear to be doing so). The trick, apparently, is to look at people's foreheads instead of their eyes. They supposedly cannot tell that you are faking eye contact. I'm not so sure that's true. If you are faking eye contact, you likely are giving off other subtle hints that you are afraid or insincere, which the audience will spot. Avoiding eye contact tells another person that you are not trustworthy and have something to hide. Faking eye contact is also bad advice because it means that you are missing important feedback from your audience. If you are not looking, you don't know if your audience can hear you, can understand you, or is interested in what you are saying. If you see negative expressions, maybe you can't do anything during that speech but it will be valuable information to help you improve your next talk.

Whenever I see someone in the audience frown or shake their head–it is disturbing–but I try not to dwell on it. I continue to look around the room until I see someone nodding or smiling. Another thing to keep in mind is that the audience members could be mirroring your expressions and body language. If you are exhibiting nervous tics or even fear, your audience is going to be uncomfortable and fidgety. If you are relaxed, smiling, and looking straight into your audience's eyes, they will likely feel comfortable and good about themselves and you.

Being nervous is no excuse for failing to make a connection with your audience. In fact, on the few occasions when I've heard a really nervous speaker admit to the audience that they were terrified of speaking, the audience immediately became sympathetic and encouraging. Most people want you to succeed and can empathize with your nervousness. If you have to imagine your audience in any condition, imagine that they admire you and are there to learn something from you.

Coming Soon to a Journal Near You: Video Abstracts

Some of you may be familiar with a new trend at some science journals: video abstracts in which the authors explain their findings on camera, sometimes enhanced with animations or other visually-rich media. A few journals routinely accept and publish video abstracts prepared by authors. Two of these are the New Journal of Physics and Cell, and you can see examples of their video abstracts by going to their websites.

These video abstracts are typically short (3-5 minutes) and often published on a video-sharing site such as YouTube, rather than on the journal website. By providing video summaries on such accessible and popular video-sharing sites, authors make their work widely available. Anyone can access these media without having a subscription or paying a fee.

In addition to the above journals, several other journals are currently "experimenting" with video abstracts. This movement reflects the overall trend in multimedia communication of information on the internet, in combination with the availability of digital devices and software for creating and sharing video.

What are the advantages for an author? By using video, authors can explain their work in a way that they are not able to do in print, such as showing footage of their laboratory setups or methods, field sites, and/or study organisms. The authors are able to provide a more personal explanation of their findings and put them into a broader perspective. By posting a video on the internet, the author can raise the visibility of themselves and their research because search engines rank video high in comparison to text-only descriptions (especially if it's the only video out there on the topic). People searching for information on a topic will be more likely to find their video abstract, and the video will lead viewers to the technical paper. Also, if the video is published on YouTube, the authors are free to embed their video abstract on their own websites, something they often cannot do with their journal publication because of copyright restrictions.

What are the advantages for the reader? Video can provide a richer, more interactive experience for a reader. For non-specialist readers, a video in which the authors explain their work in everyday language  would provide greater insight, spark their curiosity about the topic, and possibly encourage them to learn more about it. For example, as a scientist, I'm interested in keeping up with major discoveries in other fields. Although I'm not likely to read a technical paper about the Higgs boson, I would watch a video that explains what's been discovered and what it means.

Are video abstracts just a fad or will it become a common practice at science journals? Hard to say.

Some video abstracts are well-done:


Others are pretty awful:



Some science disciplines (physics) seem to be getting on the multimedia bandwagon faster than others. Whatever the future of video abstracts, we are clearly in a learning phase. Many of my colleagues have never even heard of video abstracts and expressed no interest in doing one, even if offered the opportunity. Students seemed to be more receptive to the idea, and I suspect this is because they are more technically-savy and accustomed to watching YouTube videos than most of their professors.

If video abstracts become standard practice, authors will need to either develop some skills at creating such videos or will need access to multimedia specialists who can help them. My guess is that most authors will end up paying someone, either at their institution or a free-lancer, to produce a video abstract. Possibly some journals will offer the service at a price. It will be interesting to see how this practice evolves.

For more on the video revolution in science communication, see this video:

Image Credit: modified photograph from USAID

Myths About Giving Presentations: Never Apologize for a Bad Slide

We've all heard or read advice about how to give presentations, or more specifically, how not to screw up in front of your peers. It's sometimes worthwhile, however, to revisit sage advice to see if it is really germane or if it needs some clarification or modification.

One recommendation I often hear is, "Never apologize for a bad slide." Yet many of my colleagues seem not to have ever heard this one. They continue to put up slides with long tables filled with data in font so small that it's impossible to read even from the front row. These presenters apologize for the busy slide, then say that they just want to point out one or two data points and that the audience should just ignore the rest.

Well then why not just create a slide with those one or two important data points in large font? It's easy enough in PowerPoint. The audience will be able to read the values and can concentrate on them. Better yet, add a graphic or photo that emphasizes the relevance of those data to the point you are trying to make. See the following example.

Much more effective than a dense table. So why do people persist in presenting bad slides? Part of the reason is sheer laziness. They cut and paste the table, graph, or diagram from one of their papers (or someone else's paper). They think that to do anything else requires more time and effort (which is usually not true). The other reason is that they collected all those other data and so must show it....right? Wrong. The audience will be aware of the effort you went to to get to that key data point. Showing extraneous data has the opposite effect of annoying, rather than enlightening your audience.

Showing a slide that is impossible to read or understand is also insulting to the audience. It sends a clear message that you don't care about them. And if you apologize, it means that you are well aware of the poor quality of the slide but didn't care enough about the audience to fix it.

The advice about never apologizing for a bad slide suggests that it's OK to use bad slides as long as you don't acknowledge it. Actually, most of us realize that this advice means never use a bad slide. No slide is better than a bad slide. However, it's easy for novices to misinterpret this advice....which is why I bring it up. I should mention here that using tables and complicated graphs or diagrams in posters is OK (because the viewer has the time to digest them and likely wants to see the data), but you should still design them well so that your point is clearly made.

I would amend this recommendation to be, "You should never HAVE to apologize for a bad slide. If, while practicing your presentation, you find yourself saying, "Now, I realize you can't see the data on this slide, but I just want you to focus on this number...", delete that slide and create a new one. Your audience will thank you.

The Scientist Who Happens To Be a Woman

This month is Women's History Month, and last Friday was International Women's Day. Such celebrations are set to recognize and pay tribute to the contributions made by women throughout history. Many of those contributions were by women in science.

A growing number of women, however, are becoming increasingly uncomfortable with efforts to highlight women in science because such emphasis may lead to the conclusion that there is something unusual about female scientists or that we need "special help" to succeed (special awards, fellowships, grants).

The problem seems to be exacerbated by news stories in which the female gender of the story subject is emphasized over whatever it is she has accomplished. Articles about women scientists invariably talk about the fact that they are women (".....she's one of a handful of women to work on this topic.") and about family matters ("....she had to leave her three-year-old for a month to do field research in the Antarctic"). Some focus on special challenges they face as women in a male-dominated field ("....she had to accomplish twice that of her male colleagues to be acknowledged as competent").

I came across this post by Christie Aschwanden, who suggests the Finkbeiner Test for articles profiling women in science. It's similar to the Bechdel Test, which assesses gender bias in film, which I've described here previously. To pass the Finkbeiner Test, the article cannot mention any of the following:

• The fact that she’s a woman
• Her husband’s job
• Her child care arrangements
• How she nurtures her underlings
• How she was taken aback by the competitiveness in her field
• How she’s such a role model for other women
• How she’s the “first woman to…”

I agree that over emphasis on gender can backfire and send the wrong message. But I also understand the journalists' viewpoint in trying to make a technical topic more attractive to readers (especially female readers) by emphasizing gender and/or the unusual. To get people interested in reading about a topic, the journalist has to find a "hook", and anything unusual works well to attract readers. Look at the recent media buzz about Danica Patrick, the first female NASCAR driver to win the pole position in the Daytona 500 competition. Even though media attention is focused mostly on her gender, she's edged out male competitors to become the most popular NASCAR driver, especially among female fans of the sport. Her crew chief said that he's given out "more lugnuts to young girls this week than he's given to anyone in his career". 

I do wish that we could just write and read about the accomplishments of a scientist or race car driver who just happens to be a woman. But that's not going to happen as long as the subject's gender is seen as something unusual. Journalists can help things along by de-emphasizing gender in articles about female scientists (as Aschwanden and Finkbeiner suggest). As scientists, we can decline to provide answers to gender-based questions in interviews and refocus attention on the science. 

Can You Write Me a Reference Letter, Preferably a Good One?

You've carefully put together your application package for that internship, fellowship, or dream job, including a compelling statement about your research or teaching goals and a list of all of your publications, awards, grants, and other accomplishments. You think you are well qualified, with achievements that are above average for someone at your career stage. You submit your application and then wait anxiously for that email telling you that you've been invited for an interview or selected for that fellowship or internship.

When it finally comes, the news is not good: "Dear Candidate, we regret to inform you that your application was not successful. There were many excellent candidates this year but....". At the end is a personal note that says, "You might want to rethink your list of references."

Huh? What could that mean? You are not sure, but it sounds as if one of your reference letters was not exactly glowing or perhaps even negative. You think about the three people you asked to write a letter of support for you. They all were people who you thought would write an excellent recommendation. They all agreed to prepare a letter for you. What could have gone wrong and was this what sunk your application?

Reference letters have been on my mind recently since I just participated as a panelist for a student award. I was struck by the huge discrepancy among reference letters, even those written for the same student. Here are some excerpts (modified slightly to hide any identifying information) followed by my assessment:

"I strongly recommend Ms. X for this award. She is hard-working, eager to learn, and quite intuitive. Her academic record is outstanding with a 4.0 grade-point average. She routinely asks penetrating questions in class and shows a more in-depth grasp of the topic than most students. She has developed a unique angle for her thesis research project, which puts it at the cutting edge of the field. Among the 100 or so graduate students I have interacted with as professor, I would rank Ms. X in the top 5%." The letter goes on for several paragraphs to extol the many virtues of this student, providing detailed examples and heart-felt opinions. My opinion: there is no doubt in my mind that this student is held in high esteem by her professor and deserves serious consideration.

"I wish to recommend Ms. X for this award. She is a student in our department and has taken a couple of classes with me. I understand that her advisor is also writing a letter for her. She is conducting thesis research in an area related to mine, which is ...." This letter then goes on to describe the author's area of research, rather than the student's. This professor states no opinion about her abilities or skills or qualifications for this particular award. The letter ends with, "If you need any more information, please don't hesitate to contact me." My opinion: this person has no idea how to write a letter of reference and/or did not want to (or have time to) write a positive, substantive letter for this student (but agreed to write one anyway). The student made a serious mistake asking this professor for a letter.

"I am writing this reference at the request of Mr. Y who is applying for this award. I have been acquainted with this student for a short time. He is about to complete his second semester in our department and I believe he has just initiated his field research. Based on his undergraduate record and the accomplishments he lists in his CV, I can recommend him to receive this award. Feel free to contact me for further information." My opinion: this person knows next to nothing about this student and should not have been asked to write a letter.

"I am writing to offer my enthusiastic support for Mr. Z's application for this award. He has made excellent progress on his thesis research and is excelling in all his coursework. He has all the personal qualities of the most successful graduate student, and I expect him to make important contributions to our understanding of [research topic]. Mr. Z is a bright, motivated, and energetic young scientist that I am pleased to have in my lab. He is a worthy candidate for this award, and I strongly encourage you to consider him. I would be happy to provide additional details about Mr Z's qualities should you need more information. I unreservedly recommend Mr. Z to you for this award." The letter goes into great detail about the student's research and his role in its design and novel ideas for experiments. My opinion: this is a student whose professor has no reservations about recommending him for the award and deserves serious consideration.

The above examples should give you some idea of how varied real reference letters can be. In no case was there a letter that stated anything negative or described any perceived weaknesses. Yet it was easy to categorize letters as outstanding, good, fair, and poor. At the one extreme were the letters that contained glowing, but detailed, comments and that stated flat-out their unreserved recommendation of the student. At the other extreme were the shortest letters that contained no positive comments or even a single opinion about the student.

Some people were clearly outstanding writers of reference letters and understood the importance of providing detailed information and personal opinions about a candidate. Perhaps ten percent of the letters fell into a category I would call "glowing". In several cases, we got letters like number 3 above and even a couple that were from the student's major advisor, who had had them in their lab for five or six years. Yikes. In those instances, it was sometimes clear that the advisor did not hold the student in high regard but still agreed to write a letter. In one case, the letter from the (very busy, very famous) advisor was three sentences long and basically said "I recommend this person for this award". In other cases, it seemed that the letter-writer had no idea that their letter, which lacked any observations or opinions, positive or otherwise, would be perceived as a bad reference.

In the cases where the student got a glowing letter from one person and a poor letter from someone else (like Ms. X above), it was possible that the second letter-writer had some beef with either the student or the advisor. Even so, we could not discount it because at a minimum it showed that the student had not done their homework in selecting people to write a letter for them. Also, their overall rating would suffer in comparison with another student whose letters were all outstanding.

So what can you do, if anything, to ensure that your reference letters are all good ones? I'll explore that in upcoming posts.

Image credit: modified photo from morguefile.com

Can Movies Provide a Model for Assigning Scientific Credit?

Imagine if the credit for a scientific publication was ascribed the same way movie credits are.

What got me thinking about this was a reader comment on a paper published in PLoS Biology about author sequence and how to calculate credit for author contributions in multiauthor publications.

Determining author contributions to a scientific paper is an increasingly important topic as the number of multiauthor papers has increased over time. The reason this issue is of concern is because evaluation committees may use author position on a person's publications (along with other measures) as a criterion to gauge their scientific worth and leadership abilities. The commenter suggested that the solution (to the dilemma of calculating credit) might be resolved by using the movie credit model. A post on the PLoS blog also makes the argument for a movie credit-style approach:

"Such a research credit system would have huge benefits for one’s career prospects; and it might encourage more effective collaborations. Moreover, these credits could easily be tracked by scientist or project in a database akin to the Internet Movie Database (IMDB). It could provide an alternative to the ever-so-important citation factors as a means of assessing one’s scientific impact. And maybe one day there will even be an Academy Awards of Science."

Those of you who read this blog know I like movies and often use them to discuss, among other things, how women in science are perceived by society. So, of course, I found this suggestion intriguing and decided to delve a bit further. It's not as crazy as it sounds, so bear with me while I explore this idea a bit.

In a movie, there are producers, actors, set designers, cinematographers, screenwriters, and so forth. The credits that roll during the closing sequence list everyone who contributed to the making of the movie. Some contribute more than others and are listed first in the credits, but everyone is listed, from the director down to the lowliest person who held the lead actor's cigarette during takes. Relevant to our discussion is the fact that a movie is the end result of a technological effort involving a team of people making different contributions to the end product, very similar to a modern scientific paper. The movie, however, is never solely credited to a single person or category of filmmaker professional. The success of the movie, which depends on all of these people performing their jobs well, clearly cannot be ascribed to a single person or group of people such as the screenwriters (although some, such as the director, make a more substantial contribution than others). The Oscars and other awards, which are given to honor not only screen actors but also cinematographers, costume designers, and others, reflect the fact that filmmaking is a joint effort of a variety of film professionals.

Unlike other publications that are the result of the sole efforts of a single author (e.g., novels), modern scientific publications often reflect the work of a team of scientists, technicians, statisticians, graphic artists, and others in much the same way a movie is made. So why do all the people working on a movie get credit (which they claim on their resumes), while not everyone who contributes to a scientific paper can claim credit for it?

Interestingly, early movies (before 1960) listed only the director and actors (and maybe the costume designer if it was a period movie), and these came at the beginning of the film. Someone who worked as an illustrator would never get credit; only the art director might be given onscreen credit. The change to listing everyone who had anything to do with the making of a film was due to trade unions. The unions argued that their members should get credit because it was essential for getting jobs on future films and because non-actor artists should be able to list the movie on their resume. Nowadays, virtually everyone who is in filmmaking belongs to a trade union, which looks out for its members....hence, the 15 minute long credit sequences we see at the end of movies.

In scientific publications, the authors are really the only individuals who receive credit for the work and who can list the paper on their resume, even though a number of non-authors may have contributed to the final product, and in some cases, may have made a critical contribution (for example, a statistician who helped with the experimental design). Such people usually are mentioned only in the acknowledgements, where their contributions are lumped all together in what is essentially a footnote. Because of this, non-authors cannot list the publication on their resume....and can describe only their general role ("In 2012, I assisted in field sampling during a study of coral reefs in Indonesia"). In contrast, a cinematographer, an animator, a sound technician, or even a gaffer can list on their resume the movies they've worked on.

The assignment of credit for a scientific work in part reflects the nature and history of scientific research in which the early discoverer of new knowledge, who was the sole author of the paper, got the credit. This pattern held until the early 1900s when more scientists began collaborating and tackling complex projects requiring teams of people. The numbers of authors on a paper increased while each author's contribution became less clear and quantifiable. Scientific studies increasingly involved technicians and others whose contributions were considered insufficient to count as authors. The tradition of all credit going to part of the scientific team via authorship has persisted despite the increase in non-author contributors to scientific research. Some perhaps suggested the idea for the study; others provided the laboratory space or other essential logistics; many collected data or provided technological expertise; a few gave statistical guidance....none enough to warrant authorship or credit for a scientific discovery but who played a role in the ultimate outcome.

Many scientific publications today are produced by multi-disciplinary teams, each with many people contributing in much the same way as teams of animators or set designers on a film. Papers in some disciplines such as genomics can list 100 authors, a situation that is likely to increase in the future. There is even one recent paper describing the Large Hadron Collider that lists 2,926 authors. In such cases, how do you know who did what and who, other than the first author, made a major contribution? Although some journals (e.g., Nature journals) require an author contributions statement, these are often awkward and potentially lengthy. Would it be more feasible to use a movie credit model in such cases or would that lead to even more confusion?  

Could the movie credit model even work for scientific publications? Well, I can see some problems with it, not the least of which would be resistance from scientists accustomed to getting all the credit. At issue would be who should get credit for the scientific discovery reported in the paper and how would this be indicated in the credits? Of concern, also, might be that someone listed in the credits, but with a minor contribution, would claim a greater role on their CV. I don't think the latter would be any more of an issue than it is in the movie business (a lighting technician is not going to claim she made the movie). In fact, it might reduce this possibility. With the current model of listing multiple authors (using an often unspecified method of assigning position*), minor contributors can get undue credit.

Another stumbling block might be how to sequence the credits to indicate relative contribution. The credits could be listed in a sequence of decreasing contribution, much like they are today in author lists, except that their role would be clear (Principal Investigator, co-Principal Investigator, Laboratory Director, Field/LaboratoryTechnician, Statistician, etc.), and people who often are inappropriately listed as "authors" today would be more accurately credited for their actual role in the study. The Principal Investigator(s) (who conceived and designed the study, conducted the research, and wrote the paper) would be appropriately credited as the discoverer(s) of new knowledge. A movie credit-style listing might add to the length of a paper, but concern about length is likely to be less of an issue as we move to primarily electronic publication of scientific papers and away from printed journals.

The biggest obstacle, perhaps, would be coming up with a standardized system that everyone agrees upon and then how to get everyone to adopt it. It would likely take some of the major journals making the move first and showing how it might work. Several have already implemented the requirement for an author contributions statement, which is an attempt to go in this direction.

I'm sure there would be other objections to the movie credit model, and one could argue that it would not resolve the issue of how to quantify credit for a scientific paper. However, spelling out the roles that each person played might make it easier to quantify credit than using some formula based on the author's position in the author list (as proposed by the paper in PLoS Biology). The other thing such a change would do, which seems to be an improvement over the current model, is to allow those contributors who are now mentioned only in the acknowledgements to claim credit for their contribution to the work.

I'm not necessarily advocating a change from the traditional model to the movie credit model. Possibly, such a change would introduce unforeseen problems and perhaps add to the bickering that already occurs in some research groups as to author sequence and who should get credit for a discovery. However, it's an interesting exercise to contemplate how it might work and who might benefit.

The problem of how to quantify scientific credit is not going to go away and will only worsen as the average number of authors per paper increases. We eventually may be forced to consider a radically different credit system that better reflects the complexity of modern scientific research.

*Someone unfamiliar with how authorship position on scientific papers is determined might assume that the lower the position in the list, the lower the contribution and, thus, the less credit deserved. However, that logic does not always match how author position was determined, which can vary from discipline to discipline and even from lab to lab. Although the first author position is typically occupied by the person who deserves most of the credit (but not always), the ranking of the succeeding authors may have been determined alphabetically, for example, when there are collaborating groups. In some cultures, the last author position frequently goes to the person who deserves substantial credit; this is typically the PI who got the funding and who designed and oversaw the project. In others, the last position may go to the person who contributed the least. Consequently, some may benefit unduly from their position while others may be underrated, depending on how the author sequence is assessed relative to how it was actually determined. 

Image Credit: modified image from morguefile.com