ClaimDB | Claim ID: 32

🐥 Gold Context BIRD ID: 5

Domain (DB) California Schools

Question How many schools with an average score in Math greater than 400 in the SAT test are exclusively virtual?

Answer 4.000

Schema: Omitted for brevity (available at bird-bench.github.io)

External Info: Exclusively virtual refers to Virtual = 'F'

📋 Claim Claim ID: 32

Every California school that is exclusively virtual and has an average SAT Math score greater than 400 provides each enrolled student with a school-issued laptop.

Label: NOT ENOUGH INFO(OUT-OF-SCHEMA)

⚖️ Judge Panel Approval Justifications ✅ Reviewed & approved by a panel of 3 LLM judges

grok-3-mini

The NOT ENOUGH INFO label is correct because the database schema does not contain any information about school-issued laptops, making it impossible to verify the claim from the gold context. The OUT-OF-SCHEMA category is appropriate as the claim depends on data not stored in the database, such as laptop provisions.

mistral-small

The claim cannot be verified with the given gold information because it discusses the provision of laptops, which is not mentioned in the database schema or the provided context. The claim is free of meta-references and does not expose any database schema details.

Phi-4

The claim is correctly labeled as NOT ENOUGH INFO because the database schema does not contain information about whether schools provide laptops to students. The category OUT-OF-SCHEMA is correct because the claim depends on information not stored in the database. The claim is free of meta-references and does not leak schema details.