Dr. Arpita Bose teaches undergraduate microbiology students to be critical readers by examining journal articles that are riddled with errors.
Assistant Professor of Biology, Washington University in St. Louis
PhD, MS, and BSc in Microbiology, MTech in Biotechnology
When Arpita Bose entered graduate school, she assumed that research in journals was rarely flawed. “I thought that if something is published there’s a sanctity to it. It cannot be inaccurate because it’s gone through the peer-review process, and an editorial team has looked at the results,” she says. Today, as a professor and published microbiology researcher, Bose wants to help her students realize that this logic is flawed—as are too many published papers.
That is why Bose and her team at Washington University in St. Louis go beyond teaching students how to read primary literature. “We don’t just ask students to explain Figure 1, Figure 2, and so forth,” she says. “We ask them to critically understand the data and question the conclusion.”
Bose explains that she has two objectives: First, she wants to teach students not to trust scientific information on face value. Second, she wants students to see what other scientists do in order to evaluate existing research and expose erroneous conclusions. “The general thinking of young students is that published data are never wrong and they can never be inaccurate, but this course helps them really realize they need to be critical about scientific information,” she says.
Below, Bose shares how she sets up the lesson, along with tips for other educators who want to help students put research studies under a microscope.
See resources shared by Arpita Bose, PhDSee materials
“Undergraduate students don’t know how to look at an article to determine whether it passes the test of judgment. By the time they leave the course, they know that when paper X makes a certain conclusion, they should always go back to paper X and read it to see whether the data set supports the conclusion.”— Arpita Bose, PhD
Course: BIOL 349 Microbiology
Course description: This four-credit lecture course focuses on the molecular biology of bacteria, archaea, and viruses. Topics include: the bacterial cell cycle, gene regulation, stress response, cell-cell communication, viral and bacterial pathogenesis, microbial ecology, and metabolic diversity. Friday tutorials stress analysis of the primary literature with an emphasis on current research related to material covered in lecture.
Bose’s two-paper approach to exposing flawed research
Bose explains that scientific studies may be retracted for many reasons—usually under the umbrella of scientific misconduct. This can include plagiarism, duplicate publishing, data errors, and conflicts of interest. Sometimes another group of researchers tries to repeat the results of a study and is unable to do so, leading them to challenge the original paper.
In many of the articles that Bose introduces in her course, the errors are egregious and there are clear signs of scientific misconduct, fraud, or contradictions. But she makes sure students know that errors are not always quite so obvious. “It often takes a little bit of effort by the scientific community to recognize that a paper might be flawed,” she says.
Below are the steps she takes to help students begin to see how erroneous research is identified and exposed:
Use a two-paper approach to provide a well-rounded view
Bose does not want students to read only flawed or retracted articles. “That in itself does not really portray the procedure and the way that the fallacies and inaccuracies are identified,” she says. So she pairs each flawed paper with another one. Often the second paper is one that challenges the first. Other times, she simply shares two papers that are on the same topic, even if there was no direct challenge between the researchers.
Bose recommends the website retractionwatch.com, where educators can search for retracted papers on the specific topics their students are studying. Having a second paper that questions the first is not essential, she adds. (Sometimes one does not exist.) In that case, instructors must know enough about the topic to identify the errors on their own.
Begin with a shocking example to get students’ attention
The first paper Bose has students read, one whose researchers were found to have engaged in severe misconduct, is about an organism called Bacillus subtilis. The study, published in 2012, showed that Bacillus subtilis produces a molecule that can dissolve structures called biofilms. When a second group of researchers was unable to reproduce the results, they challenged the original authors to replicate their findings. They could not do so and the paper was retracted. (Bose says she selected this combination as her first set of examples because both were published in the same scientific journal, Cell.)
Create a process for reading and discussion prep
Bose’s Microbiology course is lecture-based, with Friday classes reserved for discussions about primary literature. (That equates to 10 discussions per semester.) At the beginning of each week, she gives students one or two papers to read. They must also complete a questionnaire, which they turn in for a grade. On Friday, one of Bose’s teaching assistants moderates a discussion about the papers, beginning with a review of the homework questions.
Ask more advanced questions in class to spark spirited debate
Class discussion is essential to getting students to understand the problems that plagued the papers they are reading, says Bose. For this reason, she has the TAs follow the questionnaire review by prompting a deeper exploration of the topics. “We ask them very pointed questions,” says Bose. Some examples:
- What do you think were the reasons to study this problem in the first place?
- What do you think was the most problematic data set in this paper?
- Do you think the conclusions of this publication were valid? Why or why not?
- What do you think are the future directions of this study?
“The students find the questions to be intellectually motivating,” says Bose. “They’re very interested in answering questions, and the answers can be very different from student to student. It’s never the same answer from all of them.”
Often the students will all notice the same fault in a paper, says Bose. “That’s great, because then the instructors know that the students know how to identify problematic data.”
Bose says her students enjoy the exercise and find it thought-provoking. “The students absolutely love reading and evaluating the retracted papers,” she says. Many note that it is the first time anyone has ever introduced them to the existence of research that falls short of the mark. “They have read papers, but they don’t know the scientific process,” she adds. “This exercise completely realigns their thinking on how science is done and what science means.”