glue benchmark

GLUE (General Language Understanding Evaluation) is a collection of 9 diverse NLU tasks (QA, NLI, Sentiment, Paraphrasing) combined into a single benchmark metric — introduced in 2018, it standardized model evaluation and drove the "pre-train then fine-tune" revolution (BERT era).

Tasks

• MNLI/RTE: Inference.
• QQP/MRPC: Paraphrase/Similarity.
• SST-2: Sentiment.
• CoLA: Linguistic Acceptability (Grammar).
• STS-B: Semantic Similarity.
• QNLI: QA-NLI.
• WNLI: Winograd (often excluded due to issues).

Why It Matters

• Standardization: Before GLUE, everyone purely tested on ImageNet or custom splits. GLUE created a shared leaderboard.
• Solved: BERT and RoBERTa quickly saturated GLUE (surpassed human baseline), necessitating SuperGLUE.
• Generalization: Forced models to be "generalists" (one model, many tasks).

GLUE Benchmark is the SAT for AI — the first standardized test suite that measured general language understanding capabilities across multiple domains.