benchmarking llm

Benchmarking a large language model specifically has to deal with a problem static multiple-choice benchmarks were never built for: most of what makes one LLM response better than another — tone, reasoning clarity, following instructions precisely — has no single correct answer a fixed scoring rule can check against. Two methods have become the standard response to that gap. Chatbot Arena, run by LMSYS since 2023, has real users compare anonymous responses from two different models side by side and vote for the better one, then converts millions of these pairwise votes into an Elo-style ranking — the same rating system used in chess — letting open-ended quality get measured through aggregated human preference rather than a single graded test. LLM-as-judge, formalized in Zheng and colleagues' 2023 MT-Bench paper, instead has a strong model like GPT-4 score another model's open-ended responses according to a written rubric, trading the cost and slowness of collecting real human votes for a faster, cheaper, though imperfect, automated proxy for human judgment.

LLM-as-judge is fast and cheap precisely because it replaces a human with a model, and that substitution carries a specific, documented bias worth naming. A judge model systematically favors longer responses and responses formatted the way the judge model itself tends to write, independent of actual quality — a bias the MT-Bench paper itself measured and reported rather than papering over — which means an LLM-as-judge score reflects not just how good a response is but how well it happens to match the judge's own stylistic preferences, a confound that real human preference data, despite its own inconsistency problems, doesn't share in the same way.

Chatbot Arena's Elo-style ranking produces a single number per model, but that number answers a narrower question than it might first appear to. An Elo rating reflects how often a model wins head-to-head votes against whichever other models users happened to pair it against on the platform, over whatever mix of prompts users happened to submit — a genuinely useful signal for general chat quality as experienced by that specific user population, but not a direct substitute for a task-specific benchmark like HumanEval or MMLU if the actual decision at hand is which model writes correct code or answers factual questions accurately, since Arena's aggregate preference score was never built to isolate those specific capabilities.

LLM benchmarking split into these human-preference and judge-model methods for the same reason evaluation in general splits into benchmarks, human eval, and A/B testing: open-ended quality has no fixed answer key, so the field built proxies — a crowd's votes, or another model's judgment — to stand in for the ground truth a multiple-choice benchmark simply doesn't have for this kind of question.