Seven underused ways to use LLMs in science

Ask a language model for a verdict and it will give you prose. Ask it for a verdict plus a probability and you get something you can threshold, sort, calibrate, and audit.

The obvious objection is that the number is made up. Partly true, and that is exactly why the number alone is useless. But there is real signal in it. Tian et al. (EMNLP 2023) found that for language models fine-tuned with human feedback, confidences verbalised as output tokens were typically better calibrated than the model’s own conditional probabilities, often reducing expected calibration error by a relative 50 percent on TriviaQA, SciQ and TruthfulQA.

The number becomes a tool when you bind it to three other things: explicit reasoning, so a human can reject it on biological grounds; sources, so you know what evidence the score rests on; and a decision framework you already use, the same go/no-go thresholds you apply to any other predictor. Then you validate it like any other classifier: hold out ground truth, compute AUROC, and find out whether the scores separate anything.

Case study: an open-weights model as a synthetic lethality predictor

We used this directly. In Prosz, Sztupinszki, Diossy, Kilim, Zimon, Csabai and Szallasi (bioRxiv preprint, 2026), we asked open-weights models to estimate the likelihood of a synthetic lethal interaction for a gene pair: no fine-tuning, no training set, zero-shot. Most tested models reconstructed the results of three genome-wide CRISPR knockout screens better than chance and better than non-LLM baselines. Performance scaled with parameter count. Adding pathway and genetic context to the prompt helped surprisingly little, since the models were already carrying that knowledge from pretraining. Qwen2.5-32B-Instruct gave the best performance-to-cost balance at 0.715 AUROC, which we then used for an in silico screen of 398,277 gene pairs across 893 clinically relevant genes.

An AUROC of 0.715 is not a diagnostic. It is a prioritisation signal, which is precisely the point of the next section. We put that signal to work in our combinatorial therapy case study, where the same problem shows up at scale: two genes give roughly 200 million candidate pairs and three give about 1.3 trillion, so the only tractable move is to rank before you test.

For the gene pair {A, B}, estimate the probability that loss of both
genes is synthetically lethal in human cancer cells.

Return strictly this JSON:
{
  "probability": <float 0-1>,
  "reasoning": "<3-5 sentences, mechanism-level>",
  "evidence_type": ["pathway" | "screen" | "literature" | "inference"],
  "key_uncertainty": "<the single thing that would flip this>"
}

Do not hedge. Do not default to 0.5. If evidence is absent, say so
in key_uncertainty and score accordingly.

Run it at low temperature, parse the JSON, and treat the output as a column in a table, never as an answer.

This is the capability with the clearest commercial return and the most neglect. You have 200 candidate targets. Every one has a paper behind it. Every one looks defensible. Your wet lab can test twelve. Ranking is the bottleneck, not ideation.

Human working memory cannot hold indirect connections across DepMap, TCGA, ChEMBL, GWAS catalogues and forty years of literature simultaneously. A model can surface a two-hop link (target A is in a complex with B, B is synthetic lethal with C, C is amplified in your indication) and cite where each hop came from. You can then formalise those hops as a graph with sources, or feed them straight into the scored judgment from section 1 and sort.

There is published precedent. Adaszewski and Schindler (medRxiv preprint, 2025) ranked 522 Alzheimer’s-disease-associated targets across six criteria (biological confidence, technical feasibility, clinical developability, patient impact, competitive landscape, and safety assessment) using a QuickSort-based pairwise procedure with the model as a comparative oracle. They report that pairwise comparative reasoning exceeded pointwise scoring on five of the six criteria.

Two practical notes. Pairwise beats pointwise: “is A a better target than B, and why?” is a question models answer more consistently than “score A from 0 to 100,” on most criteria though not all. And rank stability is your validation proxy: there is rarely ground truth for “best target,” so re-run the ranking with shuffled input order and different seeds. Targets that stay in the top decile across runs are the ones worth a plate.

Most people notice that models agree with them a lot. That is not an accident, it is a documented property. Sharma et al. showed that five state-of-the-art assistants whose fine-tuning used human feedback consistently exhibit sycophancy across free-form generation tasks, and that preference data itself rewards responses matching the user’s stated view: both humans and preference models prefer convincingly written sycophantic responses over correct ones a non-negligible fraction of the time.

Liang et al. (NEJM AI, 2024) compared GPT-4-generated feedback with human peer review across 3,096 papers from 15 Nature-family journals and 1,709 ICLR submissions. Overlap between the model’s points and human reviewers’ points averaged 30.85 percent for the Nature journals and 39.23 percent for ICLR, comparable to the overlap between two human reviewers at 28.58 and 35.25 percent respectively. Overlap was highest for the weakest papers, at 43.80 percent for rejected ICLR submissions. In a study of 308 researchers, 57.4 percent found the feedback helpful or very helpful, and 82.4 percent found it more beneficial than feedback from at least some human reviewers.

Our own experience runs higher than the published figure: on our manuscripts, an LLM-guided review surfaced roughly 60 percent of the comments human reviewers later raised, enough to fix most flaws before the paper reached real review. That is an internal, small-n observation and not a benchmark. Plan around the published 30 to 40 percent; treat 60 as what you might get on a paper with obvious weaknesses and a well-tuned prompt.

You are Reviewer 2 for {journal}. You are a methodologist, you are
short on time, and you are looking for a reason to recommend rejection.

Read the attached manuscript and produce:
1. The three strongest reasons to REJECT, in order of severity.
2. For each: the exact claim, the exact sentence or figure that fails
   to support it, and what evidence would be required to support it.
3. Every statistical choice a reviewer could attack: multiple
   testing, sample size, batch effects, choice of baseline, selective
   reporting of the comparison that worked.
4. Every place the discussion states a conclusion the results do not
   license.
5. One paragraph: what a competing lab would say about this at a
   conference.

Do not summarise the paper. Do not list strengths. If you cannot find
three serious problems, say so explicitly, but look harder first.

Then run a second pass with the role inverted (“you are the author, rebut Reviewer 2”). The claims that survive both passes are your real results.

Modern assistants can run scheduled tasks, which turns the model from something you consult into something that reports to you on a field that will not stop moving. It is worth monitoring for hot preprints in a narrow area, competitor readouts, regulatory shifts affecting your indication or modality, and retractions or failed replications touching work you have cited. We covered the mechanics of scheduled tasks and connectors in our guide to using Claude for scientific work.

The failure mode is a digest nobody reads. The fix is to make the output decision-shaped.

Weekly, every Monday 07:00.

Search for work published in the last 7 days on: {target}, {pathway},
{indication}, {modality}.

Report ONLY items that would change one of these decisions:
- whether {target} stays in our top 5
- whether our {assay} is still the right readout
- whether a competitor has de-risked or killed this approach

Format, max 5 items:
[TITLE] / [SOURCE + LINK] / [WHAT CHANGED] / [WHICH DECISION IT
AFFECTS] / [CONFIDENCE 0-1 that this is material]

If nothing material was published, reply "Nothing material this week"
and stop. Do not pad.

That last instruction matters more than the rest of the prompt.

Systematic review screening is the clearest case where models are already good enough to deploy, with one caveat that is easy to get wrong. Across 23 Cochrane Library systematic reviews, models performing zero-shot abstract screening reached sensitivity up to 1.000 on the balanced development set against a human maximum of 0.775, and 66 ensembles combining models with humans or with other models achieved perfect sensitivity with workload reductions between 37.55 and 99.11 percent (Sanghera et al., JAMIA 2025). The caveat is in the same paper: on the full imbalanced corpus of 119,695 records, precision collapsed to between 0.004 and 0.096. Independent work with compact models found the same shape, sensitivity up to 100 percent with precision below 10 percent (Sciurti et al., Research Synthesis Methods).

The design rule follows directly: use the model to exclude, never to include. It is a sensitivity instrument. Everything it flags still goes to a human.

Run the same scored judgment ten times at non-zero temperature. If the model returns 0.8, 0.82, 0.79, 0.81, that is a different epistemic state from 0.8, 0.3, 0.95, 0.45, even though the means are similar.

The spread costs you ten API calls and gives you something a single verbalised confidence cannot: a measure of how stable the judgment is under resampling. Combine the two, mean score for ranking and variance for triage, and route the high-variance cases to a human. This is the same logic behind the ensemble screening results above.

Most labs cannot assemble a structural biologist, a computational chemist, and a methodological sceptic on demand. The Virtual Lab put a principal-investigator agent in charge of a team of specialist agents, including a dedicated critic agent, with a human giving high-level direction. It built its own pipeline from ESM, AlphaFold-Multimer and Rosetta, designed 92 nanobodies against recent SARS-CoV-2 variants, and on experimental validation more than 90 percent expressed as soluble protein (reported in the paper’s results), with two candidates showing improved binding to the recent JN.1 or KP.3 variants while retaining binding to the ancestral spike (Swanson et al., Nature 2025).

The transferable insight is not the nanobodies. It is that an explicitly adversarial agent in the loop improved the output, which is section 3 again, structurally.

A falsification-first position, stated plainly:

• Never put the model inside the scoring core. It belongs on the periphery of an inference pipeline: generating hypotheses, ranking, triaging, criticising. The thing that decides is a method you can audit.
• A score without ground truth is decoration. Benchmark it or drop it.
• Anything a model asserts about a specific paper needs the paper checked. Citation fabrication has not been solved.
• Prioritisation is not validation. The output of every technique above is a shorter list to test, not a result.

PharosBio builds Hydra, an autonomous agentic researcher for target discovery, on exactly this position. The synthetic-lethality work in section 1 is that principle benchmarked rather than asserted: a model on the periphery, producing a ranked list, validated against genome-wide screens before anyone acted on it. If you want the failure modes this discipline exists to prevent, see how to validate agentic AI in bioinformatics, or look at the case studies.