LLM Evaluation Interview Questions -Intermediate : The Numbers That Get You Hired

Definitions get you a callback; numbers get you the offer. This intermediate tier attaches a worked calculation to all ten LLM-evaluation questions. What you'll learn includes: validating a judge with Cohen's kappa; quantifyingcontamination with the public-vs-fresh accuracy gap; scoring detectors with precision/recall/F1; computing RAGAS metrics and agent pass^k; A/B sample sizing; annotation agreement (Krippendorff's alpha); accuracy-vs-compute curves and cons@k; CI gate design; safety operating curves with StrongREJECT; and paired, clustered, and bootstrap error bars. Part 2 of a 3-tier series. *Chapters:* From "what" to "how much" Q1 — Proving a judge with Cohen's kappa Q2 — Quantifying contamination Q3 — Detector precision/recall Q4 — Computing RAG & agent metrics Q5 — The statistics of shipping Q6 — Annotation agreement & set sizing Q7 — Accuracy vs compute, pass@k vs cons@k Q8 — Designing a non-flaky CI gate Q9 — The safety operating curve Q10 — Paired, clustered & bootstrap stats Synthesis: numeric instincts