STM Integrity Hub Supports ReviewerZero Integration After a Successful Pilot: Lessons Learned
After a pilot with several publishers, STM Solutions will support ReviewerZero's integration in the STM Integrity Hub. We share the trends screening found, what publishers told us, and what we changed.
We are thrilled to report that STM Solutions has confirmed that it will support ReviewerZero's integration in the STM Integrity Hub.
Over the past months, editors at several publishers have sent manuscripts to ReviewerZero through On-Demand Screening in the STM Integrity Hub. These screenings are typically for manuscripts that are checked just before acceptance or that are suspected of having a problem. Let's see what the screenings found, what publishers told us about it, and how we changed to adapt to their feedback.
Before diving into the details, Dr. Lisa Clancy at Compuscript best captured how publishers felt about ReviewerZero:
ReviewerZero helped us catch problems we would otherwise have missed, in papers that ultimately went on to be rejected. Our editorial offices particularly value the reference checks, which surface hallucinated references and citations that do not support the statements they are attached to. It helps keep our publishers' portfolios clean, and that is exactly what they want.
What screening found
Keeping in mind that the rates below are biased because they describe manuscripts that were perhaps already off in some way, we found:
- Almost half of the manuscripts raised at least one high-severity issue in author verification, references, statistics, or tables.
- A little over 1 in 4 manuscripts with recomputable statistics reported a result whose p-value contradicts its own test statistic enough to change the conclusion.
- About 1 in 14 had a reference that cited retracted work.
- Around 1 in 20 carried a reference that could not be corroborated against any index or live source, and about a quarter cited a link that no longer resolved (e.g., 404 error).
- A little under 1 in 20 manuscripts with tables repeated a block of numbers that the study design did not explain.
- Nearly two thirds (64%) contained at least one passage flagged as likely AI-generated.
What publishers told us
Hanrik Dong at IMR Press told us that author verification, statistical validation, and reference analysis make screening more efficient for publishers and journal editors:
ReviewerZero's integrity analysis tools provide publishers with effective solutions that enhance efficiency in quality control, specifically through features such as author verification, statistical validation, and reference and citation analysis. These tools significantly improve the efficiency with which publishers and journal editors screen manuscripts, proving particularly effective in identifying potential academic misconduct.
Publishers were just as direct about what got in the way. They asked for fewer and stronger flags, for findings they can paste straight into a message to an editorial office, and for integrity findings rather than peer-review comments.
What we changed
- We now prioritize which flags are sent to the Hub. Integrity analysts are overwhelmed and need actionable feedback rather than a long list of minutiae.
- We stopped sending editorial recommendations based on AI peer review. Publishers felt that judgments about novelty or impact are better left to editors and human peer reviewers.
- We revamped the ability of AI review to notice multiple-page inconsistencies and complex integrity issues. See more details about our improved AI review capabilities.
- We increased our ability to detect statistical inconsistencies in the text, tables, and figures. See more details about our improved statistics capabilities.
- We expanded our hallucinated-reference check several times to cover work that is not covered by the large international indexes, including those that register their DOIs with regional agencies: China National Knowledge Infrastructure (CNKI) and the Institute of Science and Technology Information of China (ISTIC), Airiti, the Korea Institute of Science and Technology Information (KISTI), and the Japan Link Center (JaLC), as well as the multilingual European DOI Registration Agency (mEDRA).
- We read difficult citation formats more reliably, including the Chinese national citation standard (GB/T 7714) and its document-type codes, bilingual references that pair the original title with an English translation, Vancouver-style author names, and references split across two PDF pages.
Acknowledgments
We thank Joris van Rossum, Program Director at STM Solutions, for his support and guidance during the pilot, and Ivo Verbeek, Strategic technology services partner in STM Publishing, and his team for their support during the integration.
What comes next
Publishers in the STM Integrity Hub that work with us can use ReviewerZero inside On-Demand Screening. See what our platform offers.
To bring ReviewerZero to your team, book a demo or write to hi@reviewerzero.ai.
