hotpotqa

HotpotQA is a multi-hop question answering dataset requiring reasoning across multiple documents to find the answer — questions are explicitly designed to be unanswerable from a single paragraph, forcing the model to "hop" from one fact to another.

Structure

• Bridge: Q: "What award did the instigator of the 1906 San Francisco earthquake win?"
• Hop 1: Find "1906 earthquake instigator" $\to$ "The earthquake was not 'instigated' but..." (Bad example).
• Real Example: "Who played the wife of the actor who played Bond in GoldenEye?" (Hop 1: Bond in GoldenEye $\to$ Pierce Brosnan. Hop 2: Wife of Pierce Brosnan).
• Explainability: Models must output the "supporting facts" sentences used to reach the conclusion.

Why It Matters

• Reasoning: Breaks simple "keyword matching" QA.
• Retrieval: Requires iterative or multi-step retrieval (Retrieve Doc A $\to$ Extract Entity $\to$ Retrieve Doc B).

HotpotQA is connect-the-facts — enforcing multi-step reasoning chains where finding the answer requires synthesizing information from disparate sources.