Submit an article to Indago - a peer reviewed journal
Submit an article to Indago - a peer reviewed journal
Submit an article to Indago - a peer reviewed journal

As we prepare to celebrate Peer Review Week later this month, news broke about an artificial intelligence (AI) system that can conduct the full cycle of scientific experimentation and produce research papers. The AI Scientist proposition is that its research process emulates the scientific process, and that its AI can run on an open-ended loop whereby it uses its previous ideas and feedback received, to improve and create better ideas. This AI is marketed by its developers, Sakana.AI, as “the first comprehensive system for fully automatic scientific discovery, enabling Foundation Models such as Large Language Models (LLMs) to perform research independently.” The developers claim it has “an automated peer review process to evaluate generated papers, write feedback, and further improve results. It is capable of evaluating generated papers with near-human accuracy.” 

AI and the Scientific method

Thomas Jefferson said, “A patient pursuit of facts, and cautious combination and comparison of them, is the drudgery to which man is subjected by his Maker, if he wishes to attain sure knowledge.” When Jefferson uttered these profound words, AI did not exist. It was accepted by scientists that the research field inevitably involved years of data collection and data analysis drudge work.

The scientific method is known to be pioneered by Aristotle, and many philosophers over the centuries have developed it to its modern-day shape. This is a structured process by which research is undertaken and, although it is not necessary to always follow the steps in sequence, the essence of the process involves making observations, forming a hypothesis, conducting experiments and analysing the outcomes to test the hypothesis. Peer review is held in high regard when applying the scientific method as outcomes must be capable of replications and be verified independently by the scientific community.

Nowadays, there are many computer programmes available to assist researchers to automate the process of data gathering and analysis. AI is now used to collect and analyse data enabling scientists to avoid the drudge work, speed up the research process, and reach a hypothesis much quicker than they would have if they used traditional research methods. ​

The scientific method is flexible to allow for computational methods and AI to conduct structured inquiry with pre-conceived hypotheses on much larger and more complex datasets than humans can process with minimal bias. Scientists can use AI as a research tool to do all the drudge work and then replicate and validate findings. These findings can then be used to by scientists to make new enquiries and established knowledge is built upon much quicker.

Advantages of using AI and other technology as a research and peer-review tool

There are many advantages of integrating AI and other technologies in research processes. The cost of the research process may be very high and using AI may help cut it down. Whilst initial investments may be required for development or purchase of AI software, its computational power would allow for much faster and more precise outcomes, and for more opportunities for real-time collaboration with other researchers across the globe and across scientific disciplines. Bianchini with co-authors (2022) have emphasised the importance of creating a conducive environment in institutions and facilitating dialogue and cross-fertilization among research communities.

AI can enhance efficient and cost-effective training of new scientists and new peer reviewers from marginalised communities. The automation of the administrative tasks in the peer review process can free up reviewers to tackle the more high-level tasks. There will be faster and more accurate outcomes due to computational capabilities of computer aided research tools and AI. Human bias and error in research can be reduced.

Hypotheses that were previously impractical or impossible to investigate can now be done by computer modelling. The AI Scientist claims that their AI is able to create its own hypotheses using available data and investigate these. The fact that it is open-source technology means there is already a measure of transparency that could bode well for future uptake.

Inherent risks posed by technology and AI to the research process

AI and other technologies often represent an unknown and fear of the unknown can inhibit the use of these technologies and innovations by researchers. This is fuelled by the lack of transparency in AI processes. The improper application and use can compromise the process of scientific inquiry, and if this becomes widespread, there is a risk of collapse of the research process. There is also the inherent corruption risk in the domain of artificial intelligence, which is defined by Transparency International as the “Abuse of AI systems by (entrusted) power holders for their private gain.”

Research integrity at the National Museum

The National Museum undertakes research in various disciplines according to the best ethical standards to produce high quality results and to communicate the relevance and meaning of these results, thus contributing to current important debates and trends, to as broad an audience as possible.

Museum research programmes aim to promote a greater awareness of the importance and role of previously marginalized communities in contributing to South Africa as a nation. National Museum scientists conducted 51 peer reviews for journals, books etc., and published 33 articles in peer reviewed journals from 1 April 2023 to 31 March 2024.

Why do we celebrate Peer Review Week?

Academia has recently been grappling with the issue of research integrity and the integration of AI tools in research and the peer review process. The theme for Peer Review Week 2024 is “Innovation and Technology in Peer Review” and this subject seems even more apt, following the recent AI Scientist innovation. If successfully implemented, the AI Scientist will likely impact the research and publishing landscape in a number of ways.

Peer Review Week is celebrated to emphasise the central role peer review plays in scholarly communication; to showcase the work of editors and reviewers; to share research and advance best practices; and to highlight the latest peer review innovations and applications. The AI technology is advancing quickly, and each new development adds a more nuanced layer to the industry’s research integrity crisis debate.

Image: W A Snell

References

Bianchini, S., Müller, M. & Pelletier, P. 2022. Artificial intelligence in science: An emerging general method of invention. Research Policy, 51(10), Art. 104604. https://dx.doi.org/10.1016/j.respol.2022.104604

Elali, F. & Rachid, L.N. 2023. AI-generated research paper fabrication and plagiarism in the scientific community. Patterns, 4(3), Art. 100706. https://doi.org/10.1016/j.patter.2023.100706

Jefferson, Th. 1838. Notes on the State of Virginia. Boston: Lilly and Wait. https://www.biodiversitylibrary.org/item/78187 (Accessed 12 September 2024)

Köbis, N.C., Starke, Ch. & Edwad-Gill, J. 2022. The corruption risks of artificial intelligence. Transparency International.  www.transparency.org; https://doi.org/10.13140/RG.2.2.21566.77129

Peer Review Week 2024. www.peerreviewweek.wordpress.com/ (Accessed 5 September 2024)

The AI Scientist: Towards fully automated open-ended scientific discovery. Sakana AI, 13 August 2024. https://sakana.ai/ai-scientist/ (Accessed 5 September 2024)

Avatar photo
Author

Comments are closed.