MnemoverseDocs

Appearance

Edward Izgorodin·Published Jul 12, 2026

Can You Trust an LLM Judge? A Field Manual ​

TL;DR

  • LLM-as-judge is an evaluation method in which a language model grades, scores, or compares outputs produced by another system.
  • Position bias is an LLM judge’s tendency to change its verdict when the same answers appear in a different order.
  • Human agreement can exceed 80%, but order, verbosity, and grading prompts can still move verdicts materially — and model-family overlap is a plausible additional risk.
  • Before trusting a judged table, identify the judge and rubric, check whether answer order was swapped, and inspect retrieval recall beneath the score.

A benchmark can hold the answers fixed and still reverse its conclusion.

Han et al. observed this in a RAG-vs-GraphRAG(Local) evaluation on the QMSum and ODSum summarization tasks. Reversing which system summary appeared first “significantly affects the predictions.” The judges made “completely opposite decisions depending on the order,” according to Section 5.3 and Figure 4 of the published evaluation.

That result does not mean every LLM verdict flips when answer order changes. It shows something narrower and more useful: a real retrieval evaluation produced opposite conclusions under an irrelevant presentation change.

The score was not just a property of the answers. It was a product of the answers and the measuring instrument. Reading its output without reading its calibration is like trusting a scale that was never zeroed.

LLM-as-judge reliability: the central tension ​

The strongest case for LLM judges and one of the clearest bias catalogs come from the same paper.

On MT-Bench, GPT-4 reached 85% agreement with human preferences, higher than the 81% agreement between humans in the reported non-tie setup (Zheng et al., abstract and Table 5).

That is why LLM judging remains useful. It can approximate human preference at a scale that manual review does not provide.

But agreement is an average over an evaluation design. It does not guarantee that any particular score is stable. It also does not establish objective correctness. A judge can agree with human preferences while responding to answer order, length, style, or familiarity. The agreement figure functions as both license and warning label.

The practical question is therefore not, “Are LLM judges trustworthy?”

Ask instead: “Under which protocol, for which task, and against which human or objective standard was this judge calibrated?”

Position bias, verbosity, and self-preference ​

Four measured failure modes matter when reading benchmark tables. Three concern preference judging and appear below; the fourth — collapse on objective correctness — is severe enough to get its own section.

Position bias changes pairwise verdicts ​

With the default pairwise prompt, GPT-4 produced a consistent verdict after the two answers were swapped only 65.0% of the time. Swapping in different placeholder names for the two answers in the judge prompt (Zheng et al.’s renamed-assistants variant, testing whether the “Assistant A/B” labels themselves carry bias) raised consistency only slightly, to 66.2%. Roughly one-third of verdicts therefore changed under order reversal in that experiment (Zheng et al., Table 2).

The same table reported stronger positional tendencies for weaker judges. The measure is specific: each answer pair is judged twice, once in each order, and a verdict counts toward it only when the judge picks whichever answer sits first both times. Claude-v1 did that in about 75% of answer pairs, and GPT-3.5 in about 50% — in those pairs the verdict was decided by placement rather than content. (A judge with no position preference scores near 0% on this measure; GPT-4’s figure was 30%.)

A single-order comparison hides this instability. A displayed win may reflect placement rather than answer quality.

Verbosity can earn unmerited credit ​

Zheng et al. tested a “repetitive list” attack that made an answer longer by rephrasing existing information rather than adding substance. The attack fooled Claude-v1 and GPT-3.5 in 91.3% of cases. It fooled GPT-4 in 8.7% (Table 3).

Judge robustness therefore varies sharply by model. A table that identifies only the evaluated systems, but not the judge, omits part of the measurement.

Length is not automatically a problem. The warning sign is a scoring method that cannot distinguish useful detail from repetition.

Model-family preference is plausible, not settled ​

Against human judgments, GPT-4 favored its own generations by about 10 percentage points of win rate. Claude-v1 showed a gap of about 25 points, while GPT-3.5 did not favor itself. Zheng et al. explicitly said they could not determine whether these differences established self-enhancement bias, so the figures remain suggestive rather than conclusive (Figure 2).

Later work offers possible mechanisms.

Panickssery, Bowman, and Feng found that evaluators can recognize their own outputs. Self-recognition correlated linearly with self-preference strength, and fine-tuning that increased recognition also increased preference (arXiv:2404.13076).

Wataoka et al. reported that models favor text with lower perplexity under their own distribution. Their interpretation is that familiar-looking text receives excess credit, while a model’s own outputs tend to look familiar. GPT-4’s measured self-preference was 0.520 on their −1 to +1 metric (arXiv:2410.21819).

These results justify checking for judge and evaluated-system family overlap. They do not justify assuming bias in every such pairing.

JudgeBench and the objective-correctness problem ​

Preference and correctness are different targets.

JudgeBench evaluates response pairs labeled by objective correctness across knowledge, reasoning, math, and coding. With vanilla AlpacaFarm-style prompts, many strong models performed only slightly above random guessing. GPT-4o scored about 50.9%, compared with 50% random (Tan et al., JudgeBench).

That result changes how a high judge score should be interpreted.

A preference judge may provide evidence that an answer appears complete, clear, or persuasive under its rubric. It does not automatically establish that the answer is correct. When correctness matters, the evaluation needs references, task-specific criteria, or another verifiable target.

A related caution applies to the scoring format itself: absolute single-answer scoring. Bare pointwise scoring is widely reported to be unreliable, with compressed score bands and run-to-run drift, and pairwise or reference-guided formats are widely reported to track human preference better, consistent with the formats Zheng et al. themselves favor. Our benchmark pipeline follows the same operating rule: do not treat an isolated absolute score as decisive.

Mitigations that hold up in practice ​

No mitigation makes the instrument neutral. The aim is to expose instability before publishing a conclusion.

  1. Evaluate both orders or record a tie. Swap the answers and count a win only when the verdict survives both presentations. Otherwise, classify the comparison as a tie. Zheng et al. propose this remedy for the position bias documented in Table 2.

  2. Anchor grading to a reference and explicit rubric. Zheng et al.’s reference-guided single-answer format improved agreement with human raters. A rubric also makes the target inspectable, although rubric choice can itself become a major source of score movement.

  3. Show multiple judges side by side. Zheng et al. reduced per-model bias by averaging multiple judges; a side-by-side panel goes one step further, exposing the model-specific spread that the average conceals. The Mnemoverse judge analysis reports judges side by side for this reason.

  4. Run human spot-audits. The MT-Bench agreement figures are meaningful because human raters provided a comparison standard. A sampled audit can reveal rubric ambiguity, answer-key errors, or systematic judge leniency before those issues define the final table. The stakes are concrete: an independent Penfield Labs audit (April 2026, one team, not peer-reviewed) reported that 99 of 1,540 answer-key entries in LoCoMo, a long-conversation memory benchmark — about 6.4% — were wrong, and that the benchmark’s default gpt-4o-mini judge accepted 62.81% of intentionally wrong, topically adjacent answers. Human audits catch what automated judges miss.

  5. Flag model-family overlap. The self-preference literature does not prove that every same-family evaluation is compromised. It does establish enough risk to require disclosure.

The field manual: three questions for any judged table ​

Read a benchmark table as a measurement record, not a scoreboard.

1. Who graded this, with what rubric—and would the number survive a strict one? ​

Demand columns or accompanying metadata for the judge model, grading prompt, retrieval depth, and dataset slice.

The grading prompt is not administrative detail. In a published prompt-swap control from our controlled memory-benchmark setup, the evaluated answers stayed fixed: LoCoMo conversation 26, 152 questions, one gpt-5 reader, and top-k 200. Changing only the rubric used by the same gpt-5 judge moved the score from 0.9013 under the lenient mem0 rubric to 0.4803 under a strict rubric — a 42.1-point drop that flipped 64 of 152 verdicts. Swapping the judge model itself, gpt-5 to gpt-4o with the lenient prompt held constant, moved the score by only 1.3 points. In that control, the prompt was the lever; the judge model barely moved the score.

A table that omits the rubric does not tell you what its score means. If you are interrogating a vendor rather than reading a table, use the linked analysis’s separate five questions for any memory leaderboard.

2. Was order swapped—and would the verdict survive the swap? ​

For pairwise evaluation, look for two directional results:

PassPresentation
ASystem A first, System B second
BSystem B first, System A second

A robust win survives both passes. A disagreement is evidence of an unstable comparison, not a reason to select the preferred ordering.

This test addresses both the consistency result in MT-Bench and the order reversal observed in the QMSum and ODSum retrieval evaluation. If the table reports only one ordering, treat the pairwise verdict as incomplete.

3. What was retrieval recall underneath the judged score? ​

A generated answer cannot use evidence that retrieval never supplied. The judge may still reward a plausible answer, especially under a lenient rubric.

The recall-versus-judge analysis, from the same controlled memory-benchmark setup, illustrates the gap. At k=10, recall was 0.669, meaning one-third of the gold evidence was not retrieved, while the judge returned 0.770. As k rose from 10 to 200, recall climbed 25.8 points while the judge score climbed only 13.1 — the judge was half as sensitive to retrieval depth as recall itself.

Before believing a delta between rows, apply a two-cell test:

  • Does the full evaluation recipe match?
  • Does retrieval recall move in a direction consistent with the judged result?

The comparability key includes judge, grading prompt, retrieval depth, dataset slice, and evaluation paradigm. Numbers are comparable only when that key matches. Default and tuned runs, for example, should not share an unqualified ranking.

Common questions ​

Can you trust an LLM as a judge? ​

Sometimes. Disclosure of the judge, prompt, order protocol, and validation is the minimum condition for interpreting the result. Zheng et al. report human agreement alongside position and verbosity biases: arXiv:2306.05685.

What is position bias in LLM evaluation? ​

Position bias occurs when an LLM judge changes its preference because the answers change order rather than quality. Zheng et al. document the effect and recommend evaluating both orders: arXiv:2306.05685.

Are LLM judges reliable for objective correctness? ​

Not by default. With generic prompts, JudgeBench found that strong judges performed only slightly above random guessing on response pairs labeled by objective correctness; correctness judging needs references or another verifiable target: arXiv:2410.12784.

Do LLM judges favor answers from their own model family? ​

The evidence is suggestive but not conclusive. Self-recognition and preference for familiar, low-perplexity text offer plausible mechanisms: Panickssery et al. and Wataoka et al..

How should I read an LLM-judged benchmark table? ​

Check the judge and rubric, verify that pairwise order was swapped, and inspect retrieval recall beneath the judged score. Then confirm that the full comparability key matches across rows: benchmark methodology.

— Edward Izgorodin · Last updated 2026-07-12