Civic Evals · Model report card
openai/gpt-4o-2024-08-06
Historical external benchmark results for openai/gpt-4o-2024-08-06. The current full civic suite uses the newer models listed on the home page.
2 tests100 scores recorded0 flaggedaverage score 0.62
Score on each test
This model's average recorded score on each test. Stance scores describe a position; higher does not mean better. Click a civic test to open it.
| test | scores | avg score |
|---|---|---|
| inspect_evals/simpleqa | 50 | 0.42 |
| inspect_evals/truthfulqa | 50 | 0.82 |
Results by grading method
The same answers, averaged by what each grading method measures. Read quality and policy-position metrics separately.
| grading method | scores | avg score |
|---|---|---|
| choice | 50 | 0.82 |
| schema_tool_graded_scorer | 50 | 0.42 |
Worth a closer look
Answers whose scores meet the suite's review thresholds. Read the grading method and response together: a low stance score describes opposition, while a low accuracy score signals a possible error.
Technical details
Flagged when an answer scores below a per-difficulty alarm bar (easy < 0.90, medium < 0.70). For stance metrics, a lower score means opposition, not incorrectness. Hedge annotations, where available, record whether the model noted possible staleness or pointed to an authoritative source.
No completions on this eval scored below the alarm threshold (easy < 0.90, medium < 0.70).