Revisiting “Peer Review” in the Post-Pandemic Era

In WAC, we consider writing as a process that promotes critical thinking. One of its implications is to emphasize revision. The revision process is built into the scaffoldings of big projects during the semester. In revising, we make new connections and discover new ways to understand and express thoughts. Perhaps one support for this argument is to observe the differences between the second edition (2011) of John C. Bean’s Engaging Ideas, one of the key texts for WAC, and its third edition (2021) (in which Bean co-authored with Dan Melzer). Of course, much can be discussed regarding the differences between the two, as we have new developments of writing pedagogy in the past ten years (Preface ix-xi), face new challenges (e.g., covid-19), and adapt ourselves to new ways of learning and teaching. In the preface of the third edition, Bean and Melzer list major changes in the new edition (xi-xii). In this essay, I want to focus on the “peer review,” one of the key strategies for revision. I’ll trace and discuss the changes in Bean’s book and reflect on my own experience in teaching. I’ll also compare the revision process with the one from the design industry and show what I think can be improved in our teaching.

In the second edition of Bean’s book, guidance for peer review is in Chapter 15 “Coaching the Writing Process and Handling the Paper Load,” a chapter focusing on how to minimize teachers’ workload while at the same time encouraging students to write. Peer review is considered one of the key strategies to save time (295-302). Although it is still included as a strategy for saving time in the same chapter in Bean’s third edition, “peer review” is singled out and reframed into a new chapter “Helping Students Use Self-Assessment and Peer Review to Promote Revision and Reflection.” In this newly added chapter, not only does it include all the useful advice in the second edition, but also lays out recent research findings for the benefits of peer review (trying to advocate its value; e.g., research shows that “peer review from at least three peer readers results in feedback that is strikingly similar to instructor feedback and leads to as much or more global revision” (252)), admits the problems with peer review (244-245), and adds a new digital platform (Eli review), in addition to the original two (namely, Calibrated Peer Review and Peerceptiv). I like the shifting from a teacher-centered view to a student-centered one; in other words, the third edition emphasizes more how peer review benefits students.

Including the three digital platforms raises an interesting question: how should we use media to facilitate peer review in a post-pandemic era? Before the pandemic, I used to ask students to print out hard copies and bring them to class (this causes some problems: some students tend to work on their drafts at the last minute and cannot find a place to print their work in an early morning class). In class, they exchange drafts, write comments, and give them back to each other. Transitioning to online teaching, I used Zoom breakout rooms and Google Doc to do the work, with guided instructions. Coming back to the classroom again this semester, I find that students are more likely to bring their laptops to class and feel more comfortable with digital tools, so I continue using online platforms such as Google Doc and Google Form. To me, a digital copy can be accessed more easily than a hard copy, considering sometimes the print copy gets lost. Sometimes handwriting is not easy to recognize compared to the regularized typography. Of course, there are disadvantages. Different platforms can be overwhelming as students need to learn how to manage them. Students also lose the freedom of writing on a piece of paper, and of arranging and rearranging ideas in a non-linear way. Additionally, they are limited by a small screen, instead of a wider view of multiple pages. Handwriting is also more intimate than the standard font. We should also consider students’ needs and accessibility: some students prefer physical copies, while others like the digital version better.

We may continue discussing the pros and cons of learning through different media, but I want to return to the topic of how to do peer review. I believe that we should guide students what we instructors practice when we comment on their papers and lay out some guided instructions and templates for their reference. Without guidance, I find students tend to focus on editing sentences such as syntax, grammar, etc. In the prompt for peer review, I usually ask students to write marginal comments of only higher-order concerns, and an end-comment or a one-paragraph comment (312-313), which consists of comments on the strong and weak points of the draft, as well as the suggestions for revision. In this way, reviewers can focus more on the higher-order elements. I also consider it important to ask students to write specific and descriptive comments and support their comments with evidence and analysis, in line with Bean’s proposal for “descriptive questions” instead of “judgment questions” (247).

Although I previously asked students to talk/communicate with each other about the comments they make, to clarify any confusions and make a to-do list for revision, I found it hard for students to translate the feedback into revisions. I also find Bean and Melzer spend most of their time addressing how to do peer review and give good feedback, but not so much how to integrate comments into revision. Indeed, there is a short section on “Revision Plans” (239), which includes a sheet to ask students to reflect on if the reviewer can correctly identify the important elements (such as thesis, analysis, etc.) and think about how to revise. The set of questions are helpful to guide students to reflect on some important writing elements, but sometimes this top-down approach may ignore certain aspects that are not listed in the guidance but exist in the feedback. Hence, I would propose a bottom-up approach.

Peer review is not only limited to academic practices. It is also a critical process in designing an app or a website. During the summer, I took a series of courses on Google UX Design Professional Certificate, in which they discuss peer review in detail. Some suggestions are also valuable for my teaching.

I think perhaps one missing piece in Bean’s book is how to read feedback. The first tip would be to “stay open minded.” Reading feedback sometimes can trigger mixed and even defensive feelings. It is important to see the feedback as something to expand one’s own limited view, and to “think about where that feedback is coming from,” how it is different from one’s own expectations. Secondly, keep in mind one’s goals. For a writing assignment, what are the goals and rubrics? Could applying the feedback improve the writing to better meet the goals? If not, think about why the feedback is not effective and document it for later consideration. Thirdly, keep one’s own agency. There will be moments when one disagrees with the feedback, and that’s okay. Everyone has their own opinions about the work. When we disagree, we should not completely ignore the feedback. Instead, we need to think about where the disagreement comes from, and whether and how the different thought is supported by evidence and reasonable analysis. We then need to judge if the feedback would fit the writing goals.

In addition to how to read feedback, I suggest arranging the feedback in an “affinity diagram.” The diagram asks one to pull the comments from different sources to one single place (with sticky notes; online platforms such as Miro, Jamboard, Notely, Mural, Padlet can do something similar.) Sometimes marginal comments are spread out through the entire paper, and can be confusing if one receives feedback from more than one reviewer. Writers may not notice some common patterns without stepping back a little bit. The affinity diagram helps with synthesizing the feedback into groups. One may rearrange/reorganize the comments into big categories such as thesis, transitions, structure, support, evidence, citation, etc. The categories/related ideas/themes can also be customized to fit one’s needs. Think: if there are any connections or relations between feedback from different reviewers? Do any common patterns/themes stand out particularly? For example, if one notices that a few comments from different reviewers are about the transitions in different paragraphs of the essay, one may need to revisit some common strategies for making transitions and go back to examine the transitions again. Gathering all the comments together in one place and organizing them according to themes offer a global view on the feedback from bottom up and allows one to identify patterns that may not be easily recognized when the feedback is from different sources.

After organizing and thinking about feedback in the affinity diagram, the next step is to come up with “insights for each theme”/pattern. The “insight” is to write out a specific “next step” for how to improve the writing. The specific insights should be based on the feedback, evidence and analysis, and the goals and criteria of the writing assignment. They also should lead to direct actions to revise. When writing insights, one may follow a template like this: It was observed that xx out of xx reviewers think that ______________ (description of the feedback). This means that ______________ (analysis of the feedback). An insight based on the theme/pattern is: ______________ (what to do next). One should also prioritize key elements in revising. Revising thesis should be prior to revising transitions, as transitions may need to change according to the revised thesis. Grammar and sentence editing should be the last step because high-order concerns will lead to sentence-level changes. Some insights “should be considered a Priority Zero,” meaning “they must be fixed” for your essay, such as having an argument, not a descriptive statement for writing an argumentative essay.

In this article, I focus on peer review, one of the teaching strategies for revision by tracking the changes in two editions of Bean’s book, and by reflecting on my own teaching experience. I’m hoping the above tips from another industry that practices peer review—reading feedback, creating affinity diagrams, drawing insights from feedback—can offer some practical steps for students to translate the feedback into effective revision and complement Bean’s wonderful suggestions for peer review.

Works Cited

Bean, John C. Engaging Ideas. 2nd Edition, Jossey Bass, 2011.

Bean, John C., and Dan Melzer. Engaging Ideas. 3rd Edition, Jossey Bass, 2021.

Google UX Design Professional Certificate. Coursera.