University Libraries

As a librarian, I see a lot of literature and I can assure you, just because something is "peer-reviewed" or a "systematic review" does not ensure high quality. In fact, I see a lot of systematic reviews that are published, but poorly done.

Here are my red flags:

1. No librarian or informationist or methodologist is a co-author or attributed in some way in the study.
2. The full search strategy is not listed in the methodology or in an appendix. Just listing the keywords searched is not sufficient as it's not transparent or reproducible.
3. Authors do not mention or cite PRISMA, PRESS, Cochrane, Joanna Briggs, or any other protocol/best practice documentation.
4. Less than three authors performed the "systematic review".
5. The journal does not have an experienced person peer-reviewing this type of methodology. See "Guidance for systematic reviews in journal author instructions: Findings and recommendations for editorial teams https://doi-org/10.1002/cesm.12050

Resources:

• Librarian Peer-Review Database - This database connects librarians who have evidence synthesis expertise with journal editors who need peer reviewers with this expertise.
• ROBIS: A new tool to assess risk of bias in systematic reviews was developed (2016)
• Analyzing Reviewer Responses to Systematic Review Search Methodology through Open Peer Review
  • Townsend, MacEachern, Song "Simplified Peer Review Rubric for Searches"
•  International health library associations urge the ICMJE to seek information specialists as peer reviewers for knowledge synthesis publications
• Guidance for systematic reviews in journal author instructions: Findings and recommendations for editorial teams