Paper mills often exploit the academic currency of citations and engage in unethical practices like authorship for hire, citation manipulation, and image fraud which ultimately compromises the integrity of scholarly work creating a distorted academic landscape where quality is sacrificed for quantity.
Citation manipulation is exemplified by "citation cartels" in institutions across China and Saudi Arabia. These groups artificially boost colleagues' citation counts, aiming to improve university rankings. Notably, Clarivate excluded math from influential lists due to such practices when mathematicians Domingo Docampo's analysis revealed patterns suggesting manipulation, with citations often coming from the same institution. Critics argue that relying on citations as a measure of scientific quality is flawed, prompting the need for refined evaluation systems to uphold research integrity.
Authorship for hire can best be seen in a case from September 2023, where a publisher took action by retracting nearly 40 papers associated with the unethical practice of selling authorship. This revelation prompted the implementation of stringent new policies to prevent further abuses. Previously, authorship changes were relatively unrestricted, but the revised policy will only allow changes under exceptional circumstances.
New Threat: Image Manipulation with AI Failures or “Deepfakes”
As technology advances, the manipulation of scientific images has become more concerning, such as instances where AI has been utilized to generate seemingly authentic images that can pass peer review, contributing to a 2023 record volume of fraudulent and/or retracted publications.
The same publisher mentioned above, retracted an article less than 6 months later that used fraudulent generative AI-created images with no scientific or deducible language,and even though the authors told the publisher they used AI images - it passed their peer review.
This is different from traditional image fraud because a "deepfake" (a term originating in 2009 related to video manipulation, but used more generally today) can be defined here, as a form of synthetic media created using artificial intelligence to create, superimpose, or manipulate existing images, videos, or audio, often leading to realistic but fabricated or misrepresented content.
If accepted, the new threat, “deepfakes” clearly extends to manipulated scientific images and raises questions about the credibility of scientific research at large. Elisabeth Bik, who has dedicated ten years to investigating image manipulation in academic publications, found in 2016 that approximately 4% of papers in a 20,000-sample had manipulated images. Then, in a 2022 New York Times opinion piece, she expressed concerns about the potential misuse of artificial intelligence (AI) in the academic review process, highlighting the contrast between its promised acceleration of reviews and the possibility of nefarious activities. (Cosmos Magazine)
“Is any type of 'image review' part of your peer review program policy?”
“Is your publisher allocating a budget for image manipulation forensic services as a systematic part of their peer review process?”
“Do you think with AI and deepfakes infiltrating science, image manipulation will rise above 4%?”
This poses a serious threat to society, as misinformation within scientific research can have far-reaching consequences if not controlled or suppressed. Even government agencies like DARPA, which has been at the forefront of combating deep fakes since 2018, recognize these threats, underscoring the potential harm that fraudulent publications can inflict on society. The Retraction Watch Top 10 Most Highly Cited Retracted Papers is a good example of bad science having a large voice when not controlled and suppressed properly.
Companies in the broader field of scholarly publishing and research integrity which offer related services:
1. KnowledgeWorks Global Image Forensics: The process uses exacting image analysis techniques to detect image manipulation in supplied grayscale or color images
2. Crossref: Provides infrastructure for scholarly content identification, including DOI (Digital Object Identifiers), which contribute to content traceability.
3. COPE (Committee on Publication Ethics): An organization providing resources and guidance on publication ethics, including detecting and addressing image manipulation.
4. PlagScan: Specializes in plagiarism detection tools that can be applied to scientific content.
5. Clarivate Analytics (Publons): Offers tools for researchers and publishers to track and verify peer review and authorship contributions.
6. Penelope.ai: Focuses on AI-driven tools for manuscript screening, including identifying potential image manipulation.
7. iThenticate (Turnitin): Offers plagiarism detection services that can help identify manipulated or fraudulent content.
8. Cabells: Provides services related to scholarly journal information, including journal whitelists and blacklists.
9. SWB Consultants: Enhancing research impact for researchers, academic institutions, corporations, and enterprises.
*Please note that the landscape of companies offering these services may evolve, and it's advisable to check for the latest information and developments in the field.
The future of research integrity hinges on addressing the challenges posed by deep fakes in scholarly publications. It is imperative to engage professionals with expertise in deep fake identification and suppression technologies and integrate them into your peer review system. In the multifaceted realm of scholarly publishing, addressing challenges such as image manipulation, AI failures, and the emergence of deep fakes necessitates a united commitment to vigilance and proactive measures.
Recognizing the threats posed by deep fakes in scientific research and adopting advanced identification technologies are critical steps. Concurrently, fostering collaboration among industry experts and upholding rigorous peer review standards through the expertise of editorial teams and boards are vital pillars for preserving research integrity amid the evolving landscape of scholarly communication.