Academic Research Challlenges? There’s AI tools for that
For students and professional scholars alike, starting a new research project typically means digging through academic literature to understand what others have already written.
That can take a considerable amount of time, with researchers tracking down and combing through journal articles to begin their research and contextualize their own findings. But a growing collection of AI-powered tools aims to make that process easier. These new tools can help researchers more quickly find relevant papers, pull out relevant information from them, or both.
“It can be a really helpful way to get started with research, especially for students who aren’t familiar with the research process,” says Breanne Kirsch, director of the library at Illinois College. “As long as they’re taught how to use it in an ethical way, and that they can then expand beyond what it does.”
A tool called Elicit can help researchers conduct what are called systematic reviews, which involve going through copious amounts of published research to find an answer to a question, like how a particular drug affects a medical condition. “It’s all very, very manual,” says James Brady, head of engineering at Elicit. “It takes teams of people many months, and you know, costs hundreds of thousands or millions of dollars to do these things.”
Elicit can make that process much faster, and also help researchers by quickly finding and summarizing published papers related to a particular question. It can also generate tables describing a whole set of relevant papers, with columns for data points like algorithms and statistical techniques used, variables examined, and the number of participants in experiments.
The company recommends researchers still look at the original papers, and Brady emphasizes that the tool doesn’t replace the human judgment and analysis necessary to scientific research. “It’s not like you take the final step of Elicit and hit the publish button and then it ends up in Nature or something,” he says, but it can still greatly speed the process of sifting through and understanding prior work.
Understanding how AI can help academic research is part of a larger industry question of how and when the technology can replace or supplement traditional web search tools. And since the 1990s, computer scientists have realized that the academic publishing landscape—where scholars cite each other’s papers and publish in journals with a particular reputation in a particular field—isn’t that different from the internet ecosystem. That means techniques for finding relevant materials, minimizing AI errors and hallucinations, and presenting useful and verifiable results to the user may transfer from academia to the broader web.
Indeed, not everyone searching for scientific answers is a professional scientist. And the organizations behind these tools say they can be especially helpful for people looking to understand new fields of interest, whether they’re students, professionals doing interdisciplinary work, or interested members of the public.
Eric Olson, cofounder and CEO at AI research search engine Consensus, says about 50% of the tool’s research is at academic institutions, where it’s often used by graduate students. “We typically do quite well with folks who need that easy, quick access to research but maybe aren’t a full-blown expert yet,” he says.
Consensus lets users type in natural language queries to get answers summarized from across published work. It surfaces summaries of particular papers, metadata like publication year and citation count, and an indication of how much scientific consensus there is about a particular question. Another popular audience for the tool is healthcare workers, including doctors, who use the tool to get insights more quickly than traditional scholarly search engines or Google can provide. Everyday users also use Consensus to research health topics, parenting practices, and policy issues in the news, Olson says.
Like other companies in the field, Consensus doesn’t simply rely on a single GPT-style large language model to generate answers to user questions. The company deploys a custom search engine to find papers addressing a query, and a variety of expert-trained language models to extract relevant information and—equally important—verify the paper is actually on topic, cutting the chance that an overzealous AI model will try to point out facts that aren’t actually there.
“I’m only gonna let this go to the model if we think that it actually has a relevant insight in it,” Olson says. “It’s a really great trick to reduce the risk of misinterpreting the paper.”
Academic publishing giant Elsevier has similarly developed a tool called Scopus AI to search through research collected in its Scopus database, which includes article abstracts and metadata from tens of thousands of journals (including those published by rival publishers). Scopus AI can generate summary responses based on particular queries, suggest additional questions to help users expand their knowledge of the field, and highlight “foundational papers” and “topic expert” authors who have especial influence in an area of expertise.
“We’ve actually found this is quite a shared need across a number of different people who are at this precipice of trying to understand another domain,” says Maxim Khan, SVP of analytics products and data platform at Elsevier.
Khan says users have confirmed it helps them understand new fields faster and come across papers they might not otherwise have discovered. Thanks in part to licensing terms, the tool doesn’t include full text, meaning users can’t directly query about material in articles beyond the abstracts and citations.
Other software can help users dive deep into specific research. An AI tool from JStor, still in limited beta, lets users see article summaries customized to their particular queries and can answer questions based on document contents, pointing to particular passages that contain the answer. That can help users figure out which papers are relevant enough for a close read, and the tool can also point to other topics or particular papers for a user to investigate based on particular passages.
“The user actually is now having a conversation with the article, and so they’re engaging with the article in a completely different way,” says Kevin Guthrie, president of JStor’s nonprofit parent, Ithaka. “Obviously, there’s a very big difference from just downloading an article or downloading the PDF and reading it.”
The organization, with its focus on helping students with research, deliberately doesn’t generate aggregate answers to particular questions from multiple articles. Beth LaPensee, senior product manager at Ithaka, says the software can help students learning research skills and specialized vocabulary understand material they might otherwise struggle with. In a June blog post, Guthrie and LaPensee compared the process to learning the basic plot of a Shakespeare play before diving into the antiquated text, and say it can be especially helpful with humanities and social science papers that customarily don’t include abstracts.
The software has also proven helpful to professors. “One faculty member we were talking to said that they could do in one day what used to take them four or five days,” LaPensee says.
And the organization has found participants in the AI beta, which is slated to expand in the fall, spend “significantly more time on JStor” than other users.
Measuring results—and even knowing what to measure—is naturally an important part of testing new AI resources. Since 2015, a project called Semantic Scholar has focused on using AI to analyze scientific papers. It’s part of Ai2, the AI research institute founded by late Microsoft cofounder Paul Allen, and today it includes features to help users understand papers, like surfacing definitions of technical terms from within a paper or other research it cites, answering general questions about specific papers, and generating “tl; dr” summaries of papers based on the types of descriptions authors post on social media.
How to test whether those summaries were helpful wasn’t immediately obvious, recalls Dan Weld, chief scientist and general manager of Semantic Scholar. If users were benefiting from them, they might either click more articles from search results—if the summaries indicated they were interesting—or fewer, if the summaries helped them weed out extraneous results. But when the summaries were later added to email alerts, the results seemed positive—users clicked fewer emailed articles overall, but were more likely to save articles they clicked, suggesting the summaries steered them to interesting work.
Evaluating a feature Semantic Scholar is currently testing to answer questions from across multiple papers is even more challenging, according to Weld, who says, “It’s really quite difficult to compare different systems. There are some other systems out there that do question answering—we think ours is better than theirs, but we can’t prove it yet.”
And since different AI research tools have access to different sets of papers as well as different features, researchers may still find they need to use multiple AI platforms—often along with traditional database tools—to find everything they need. It’s important to note, Illinois College’s Kirsch says, that reading AI summaries can’t substitute for working through actual papers and verifying that they say what the tools claim, tempting though it can be.
“While the generative AI tools may help as a starting point, just like Wikipedia would, you still want to go to some of those actual sources,” she says. “You can’t just rely solely on the GenAI tools. You also need to look at the sources themselves and make sure it really does make sense for what you’re trying to do and what you’re trying to research.”