🔍 Claim ID
✕ Close
🐥 Gold Context (from BIRD) BIRD ID: 5
Domain (DB) California Schools
Question How many schools with an average score in Math greater than 400 in the SAT test are exclusively virtual?
Answer 4.000
Schema: Omitted for brevity (available at bird-bench.github.io)
External Info: Exclusively virtual refers to Virtual = 'F'
Answer SQL: SELECT COUNT(DISTINCT T2.School) FROM satscores AS T1 INNER JOIN schools AS T2 ON T1.cds = T2.CDSCode WHERE T2.Virtual = 'F' AND T1.AvgScrMath > 400
📋 Claim Claim ID: 32
Every California school that is exclusively virtual and has an average SAT Math score greater than 400 provides each enrolled student with a school-issued laptop.
Label: NOT ENOUGH INFO(OUT-OF-SCHEMA)
⚖️ Judge Panel Approval Justifications ✅ Reviewed & approved by a panel of 3 LLM judges
grok-3-mini
The NOT ENOUGH INFO label is correct because the database schema does not contain any information about school-issued laptops, making it impossible to verify the claim from the gold context. The OUT-OF-SCHEMA category is appropriate as the claim depends on data not stored in the database, such as laptop provisions.
mistral-small
The claim cannot be verified with the given gold information because it discusses the provision of laptops, which is not mentioned in the database schema or the provided context. The claim is free of meta-references and does not expose any database schema details.
Phi-4
The claim is correctly labeled as NOT ENOUGH INFO because the database schema does not contain information about whether schools provide laptops to students. The category OUT-OF-SCHEMA is correct because the claim depends on information not stored in the database. The claim is free of meta-references and does not leak schema details.