AI in Research
See also the Education resources under Cornell, ETH Zürich, and University of Texas at Austin, which also cover AI use in research.
Help Wanted: Add your resources to this catalog! See contributing for details.
Research Papers
Some select research papers that have studied various aspects of AI use in research contexts. The are listed alphabetically by title.
Can REF output quality scores be assigned by AI? Experimental evidence
PDF. Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, UK.
Authors and Affiliation: Mike Thelwall, Kayvan Kousha, Paul Wilson, Mahshid Abdoli, Meiko Makita, Emma Stuart, and Jonathan Levitt, Statistical Cybermetrics and Research Evaluation Group, University of Wolverhampton, UK.
Supplementary materials are discussed here.
Abstract: This document describes strategies for using Artificial Intelligence (AI) to predict some journal article scores in future research assessment exercises. Five strategies have been assessed. These are summarised here for completeness, but we recommend that AI predictions are not used to help make scoring decisions yet but are further explored through pilot testing in the next REF or REF replacement. The pilot testing should assess whether using AI predictions and prediction probabilities alongside, or instead of, bibliometric data would be helpful for any UoAs. For example, depending on UoA, AI predictions may be used to help mop up difficult scoring decisions near the end of the assessment period, to gain interdisciplinary input, as a tiebreaker in the way that bibliometrics are currently sometimes used, or to cross check the final scores.
LLMs as Research Tools: A Large Scale Survey of Researchers’ Usage and Perceptions
Citation: Zhehui Liao, Maria Antoniak, Inyoung Cheong, Evie Yu-Yen Cheng, Ai-Heng Lee, Kyle Lo, Joseph Chee Chang, and Amy X. Zhang, LLMs as Research Tools: A Large Scale Survey of Researchers’ Usage and Perceptions, ArXiv, October 30, 2024 https://arxiv.org/abs/2411.05025.
Abstract: The rise of large language models (LLMs) has led many researchers to consider their usage for scientific work. Some have found benefits using LLMs to augment or automate aspects of their research pipeline, while others have urged caution due to risks and ethical concerns. Yet little work has sought to quantify and characterize how researchers use LLMs and why. We present the first large-scale survey of 816 verified research article authors to understand how the research community leverages and perceives LLMs as research tools. We examine participants’ self-reported LLM usage, finding that 81% of researchers have already incorporated LLMs into different aspects of their research workflow. We also find that traditionally disadvantaged groups in academia (non-White, junior, and non-native English speaking researchers) report higher LLM usage and perceived benefits, suggesting potential for improved research equity. However, women, non-binary, and senior researchers have greater ethical concerns, potentially hindering adoption.
Smart software spots statistical errors in psychology papers
Citation: Baker, M. Smart software spots statistical errors in psychology papers. Nature (2015). https://doi.org/10.1038/nature.2015.18657
The role of artificial intelligence in generating original scientific research
ScienceDirect page, research from School of Biological and Behavioural Sciences, Queen Mary University of London, UCL School of Pharmacy, University College London, and FabRx Ltd, London.
Citation: Moe Elbadawi, Hanxiang Li, Abdul W. Basit, and Simon Gaisford, The role of artificial intelligence in generating original scientific research, International Journal of Pharmaceutics, Volume 652, 2024, 123741, ISSN 0378-5173, https://doi.org/10.1016/j.ijpharm.2023.123741.
Abstract: Artificial intelligence (AI) is a revolutionary technology that is finding wide application across numerous sectors. Large language models (LLMs) are an emerging subset technology of AI and have been developed to communicate using human languages. At their core, LLMs are trained with vast amounts of information extracted from the internet, including text and images. Their ability to create human-like, expert text in almost any subject means they are increasingly being used as an aid to presentation, particularly in scientific writing. However, we wondered whether LLMs could go further, generating original scientific research and preparing the results for publication. We tasked GPT-4, an LLM, to write an original pharmaceutics manuscript, on a topic that is itself novel. It was able to conceive a research hypothesis, define an experimental protocol, produce photo-realistic images of 3D printed tablets, generate believable analytical data from a range of instruments and write a convincing publication-ready manuscript with evidence of critical interpretation. The model achieved all this is less than 1 h. Moreover, the generated data were multi-modal in nature, including thermal analyses, vibrational spectroscopy and dissolution testing, demonstrating multi-disciplinary expertise in the LLM. One area in which the model failed, however, was in referencing to the literature. Since the generated experimental results appeared believable though, we suggest that LLMs could certainly play a role in scientific research but with human input, interpretation and data validation. We discuss the potential benefits and current bottlenecks for realising this ambition here.
Use of artificial intelligence and the future of peer review
Citation: Howard Bauchner, Frederick P Rivara, Use of artificial intelligence and the future of peer review, Health Affairs Scholar, Volume 2, Issue 5, May 2024, qxae058, https://doi.org/10.1093/haschl/qxae058
Abstract: Conducting high-quality peer review of scientific manuscripts has become increasingly challenging. The substantial increase in the number of manuscripts, lack of a sufficient number of peer-reviewers, and questions related to effectiveness, fairness, and efficiency, require a different approach. Large-language models, 1 form of artificial intelligence (AI), have emerged as a new approach to help resolve many of the issues facing contemporary medicine and science. We believe AI should be used to assist in the triaging of manuscripts submitted for peer-review publication.
AI-focused Research Institutions
There are many research institutions focused on AI. Many include not only research programs, but provide public education and advocacy. Here is a partial list.
Allen Institute for AI (“AI2”)
AI2 builds open models, along with extensive information about how they were built, Asta, an agentic ecosystem that advances scientific discovery, and other research projects.
RPI - Future of Computing Institute
Quote: The Future of Computing Institute (FOCI) is Rensselaer’s hub for interdisciplinary research in advanced computing. By integrating emerging technologies across all fields, FOCI drives innovation, collaboration, and impact at scale.
Table of contents
- AI in Research
- Research Papers
- Can REF output quality scores be assigned by AI? Experimental evidence
- LLMs as Research Tools: A Large Scale Survey of Researchers’ Usage and Perceptions
- Smart software spots statistical errors in psychology papers
- The role of artificial intelligence in generating original scientific research
- Use of artificial intelligence and the future of peer review
- AI-focused Research Institutions
- Research Papers