In the context of the academic and research ecosystem, the process of abstract review becomes the foundation of discourse in events. Abstract blind reviews come in many different types. Â These different forms of reviews serve different goals, and in certain situations, the level of transparency comes into play.
The abstract review models help reduce bias in the review process. Each model offers different levels of anonymity. This also influences the overall quality, fairness, and transparency of reviews, resulting in redefining the academic integrity of an event.
In this article, let’s gain a clear understanding of the most widely recognized abstract review models: triple-blind, double-blind, and single-blind.
Triple-Blind Review
In the triple-blind abstract review model, neither authors, reviewers, editors, nor event organizers will get to know who is reviewing the abstract. Its main focus is on eliminating bias that could influence the decision-making process. This is considered one of the best ways to get good results, but it is quite challenging to manage as well.
First, the authors submit anonymized abstracts. Then these abstracts go through a system or to an independent coordinator. They assign these abstracts to the reviewers without providing any details to them about the authors.
Now, after receiving the abstracts, reviewers evaluate the submissions anonymously. If there is a need to interact, reviewers and editors only communicate through the system until the process concludes (and sometimes, permanently).
Example: Paper #123, assigned to reviewers A and D (no names for either reviewer nor author), is shown.
Pros
-
- This model reduces administrative and editorial bias.
- It eases decision-making and reduces the influence of personal biases.
- Ensures a fairer review process because the decision-makers lack details of the authors and reviewers.
- Triple-blind is useful and valuable in comparison to all the abstract review models.
- It ensures impartial evaluation and maintains the integrity of the review process.
- It promotes ethical ways of review. There is less chance of getting influenced by knowledge, conflicts, or favoritism.
- Authors develop trust in the system and feel reassured. There is no “back-channel” influence and editorial interference that affects their results.
- This review model also provides a dedicated commitment towards fairness and equality. This improves the overall reputation of the academic event.
- The novice researchers get an equal opportunity for the recognition of their work.
- Reviewers can provide very honest feedback anyway; no one knows their identity.
Cons
-
- It is one of the most complex abstract review models to work with. It requires robust digital systems to manage all the information well without compromising on security issues.
- The event organizers should follow a strict process and ensure that participants adhere to all the guidelines, and identities remain hidden forever.
- Resolving queries becomes quite difficult because of hidden identities.
- It’s still an experimental abstract review model and most conferences still don’t follow it.
- Considerable drawbacks related to transparency and networking.
- Administrative staff or reviewers usually require re-identification of who the authors are for presentation or programming. This leads to significant delays.
Double-Blind Review
The authors’ and the reviewers’ identities remain hidden in this abstract review model. This helps in eliminating the bias. If you’re gathering submissions for double-blind abstract review, then it is important that you collect in a way that hides the authors’ identities completely. There should be no repeated citations related to their own work or colleagues throughout the submission process.
According to a study conducted by the Publishing Research Consortium (PRC), 71% of respondents said the double-blind abstract review process is highly effective.
Example: Reviewer gets to see: Paper #123; author gets to see: reviewer 6 (anonymous comments)
Pros
-
- In a double-blind abstract review, both authors and reviewers are know to each other. This helps protect the authors from reviewer bias.
- This model ensures submissions from renowned individuals based on their individual merit, rather than their reputation.
- This approach minimizes reviewer bias from factors such as the author’s gender, race, country of origin, academic status, and previous publication history.
- Reviewers judge the work fairly and accurately, without considering the author’s identity, status, and previous achievements.
Cons
-
- Double-blind review isn’t always truly “double-blind” in practice.
- Reviewers might still guess the author’s identity due to a narrow field of study, familiar writing style, references to specific work or projects
- To maintain anonymity, authors may need to exclude identifying details, which can impact research quality, context and relevance of the work.
- This can lead to challenges in presenting comprehensive research while maintaining anonymity.
Single-Blind Review
The single-blind abstract review model is the most traditional type, and it is still the most common and widely used in many conferences. In this method, the reviewers know the identity of the author, but the word “blind” is for authors only.  Authors don’t get any information related to the identity of the reviewers.
According to a survey from the Publishing Research Consortium, 85% of participants had gone through a single-blind review, only 52% stated it was effective (and it was the preferred option for only 25%).
Example: Reviewer gets to see: “John Smith, Columbia University”; author sees: “Reviewer 8” (anonymous comments).
Pros
-
- In the single-blind abstract review, the authors have zero influence on the reviewers because they do not know the identities of the reviewers. This means less pressure on the reviewers, as they can evaluate the paper with more focus and clarity. There is no scope for fear or anticipation regarding facing backlash from the authors.
- Freedom to reviewers allows them to give honest feedback and not be affected by the response to the feedback.
- This abstract review model also protects the integrity of the review process because the author cannot bribe the reviewer to go easy on the review process.
- This model is widely implemented across several fields of academic conferences.
Cons
-
- Many times, leads to bias among the reviewers, which can be of many types, like gender, country of origin, academic status, previous publication history, and more.
- Knowing the author’s details often leads to favoritism, especially towards the institutions and for the well-known researchers.
- It isn’t always considered a positive option in the academic community for the review process.
- It allows reviewers’ personal interests and preferences to interfere with the matters of examining and judging the author’s work.
- Reviewers might misuse the benefit of being anonymous as an opportunity to provide harsh feedback to the authors.
- Some reviewers might even take steps to reject the lesser-known or new author’s work.
- This review model might also lead to stagnation and damage the reputation of the conference.
Triple-Blind vs Double-Blind vs Single-Blind Review
| Category of Difference | Triple-Blind | Double-Blind | Single-Blind |
|
Level of Anonymity |
Reviewer, author, and editor are all anonymous | Reviewer and author are anonymous | Only the reviewer is anonymous. The author is known. |
|
Risk of Bias |
Less |
Moderate to less |
More because of revealing the author’s identity |
| Transparency | Very high | High | Minimal |
| Real-time application | Rarely used | Used moderately | Vastly used |
| Review Timeframe | Slow | Moderate but acceptable | Quick |
| Technical Requirements | Advanced | Standard | Basic |
| Administrative Role | High | Medium | Low |
| Reviewer’s Context | Less | Limited | Very Strong (has author’s details) |
| Fairness for New Authors | Very fair | Fair enough | Sometimes biased |
|
Suitable for |
Ethical, high-standard academic events | Large and diverse academic conferences | Traditional and small-scale conferences |
 Examples of Blind Review ImplementationÂ
The Web Search and Data Mining conference (WSDM) 2017 evaluated the double-blind vs single-blind abstract review models, mainly focused on understanding the reviewer bias. WSDM had mainly used only a single-blind model for their review process, with an acceptance rate of 15.6% only.
Min Zhang from Tsinghua University and Andrew Tomkins of Google, who were the 2017 conference’s co-chairs, were asked to consider choosing the double-blind review. They found that no experiment was conducted related to single and double blind in computer science to understand the differences and gather results. This drew their interest towards the review conditions that can reduce reviewer bias.
So, they decided to experiment with the assistance of William D. Heavlin, who is also from Google. The purpose of the experiment was to determine whether implicit reviewer bias relates to the author’s gender, nationality, status, and affiliation or not. Thus, they divided the reviewers of their conference into two groups:
Group 1: Reviewers with access to the authors’ data
Group 2: Reviewers who cannot access the authors’ data
After that, both groups place bids on papers to express interest in reviewing them. The organizing committee appointed two reviewers from each cohort to each application after the bidding was over. After analyzing the review data, the organizers found 3 important differences in the behavior of the conference’s single and double-blind reviewing groups.
-
- Reviewers in the single-blind group bid on an average of 22% fewer papers as opposed to those in the double-blind group.
- In contrast to the bids of double-blind reviewers, single-blind reviewers’ bids highlighted bias, especially towards submissions from reputed colleges and businesses.
- In comparison to their double-blind reviewers, single-blind reviewers had a higher chance of giving positive feedback for the submissions whose authors are either well-esteemed or from reputable organizations.
The findings of this experiment highlight how the single blind abstract review model impacts reviewers’ decisions with information related to authors and their respective affiliations. Single-blind also offers good perks for submissions from authors who belong to a prestigious organization. This is due to the fact that more bids boost the chances that they will be assigned to the best reviewers.
A submission from an eminent author from a reputed organization may get a more favorable evaluation under the single-blind model. But the same work from an upcoming researcher from an unidentified organization may go unnoticed or sometimes rejected.
Although the single-blind abstract review model remains common in various disciplines where reviewer context is vital, such as medicine, engineering, and law. Some of the academic conferences are increasingly shifting towards a double or triple-blind abstract review model because of its standards and outcomes.
Many reputed events in computer science have actively adopted a double-blind abstract review model to eliminate bias in academic conferences. Some of these institutions are the Association for Computing Machinery (ACM) and the Institute of Electrical and Electronics Engineers (IEEE).
Leading organizations related to social sciences and humanities also preferred the double-blind review because of the sensitivity of interpretative research. The triple-blind review model is also emerging slowly in niche domains such as bioethics, data science ethics, and interdisciplinary research, where neutrality is given first and foremost importance.
Which is the Most Effective Method?
The decision depends on the level of transparency one requires in their conference. Remember that the amount of transparency changes among these systems. Additionally, each of them has varying degrees of application difficulty. Some of the issues include costs and time.
Triple-Blind – It’s worth choosing triple-blind for event organizers working in domains that are prone to institutional conflicts. It is also important to consider the technical complexity this review model possesses and the administrative costs it demands.
Double-Blind – It’s the gold standard for the majority of academic conferences that are looking for meritocracy and equity, mainly in interdisciplinary or competitive environments.
Single-Blind – Ideal for resource-constrained conferences, where communities are already large and diverse. The ease of administrative simplicity outweighs the possibility of potential bias. It’s also appropriate when expectations are set by discipline tradition.
You should carefully consider what industry expectations are and what you are capable of handling. If you require any assistance in developing a more effective pipeline for your conference’s abstract review process, please contact us. Dryfta offers the best all-in-one event management platform that supports the success of your conference.
Wrapping Up
Every abstract review model has its own pros and cons. Although single-blind reviews are the most straightforward, they are prone to biases. Double-blind evaluations are useful in many academic conferences because of their efficiency, transparency, and robust protection against the majority of biases. Triple-blind abstract reviews are still rare, but they offer the highest level of impartiality at all levels. However, the expense of higher logistical and technical difficulties makes them complex to implement.
Organizers must assess their review processes, considering field-specific concerns, logistical resources, size of the reviewer and author pool. They must focus on their mission for fairness to achieve excellence. By understanding the nuances of these three models, event organizers can select the model that best supports both academic rigor and equity for all participants.
Dryfta’s comprehensive abstract management software platform streamlines the entire process of managing every element required to successfully run an academic conference. It contributes to raising your academic conferences to a global standard.