in the loop

GPT-4o mini
#178 of 274 scored.

OpenAIJul 18, 2024Closed

Capabilities Index

Range 119.1–128.7

126.6

Context

Takes text, image, file

128K

Input

Per 1M tokens

$0.15

Output

Per 1M tokens

$0.60

Benchmarks

How it scores.

  • GPQA Diamond17.0%

    162nd of 186

  • FrontierMath0.7%

    78th of 81

  • ARC-AGI-20.0%

    72nd of 79

  • SimpleQA Verified8.3%

    76th of 77

  • Mock AIME6.9%

    143rd of 176

Everything else Epoch has for it

  • GSM8K91.3%
  • PIQA77.4%
  • MMLU75.7%
  • Lech Mazur Writing67.2%
  • GeoBench64.0%
  • MATH level 552.6%
  • DTBench24.0%
  • Balrog17.4%
  • LMCA12.2%
  • WeirdML11.8%
  • Aider polyglot3.6%
  • Mystery Game Puzzles3.1%
  • VPCT1.0%
  • Chess Puzzles0.0%
  • SimpleBench0.0%

On the index

Around it.

  1. #175Mistral Small 3Mistral · $0.05 in127.1
  2. #176Gemini 1.5 Pro (May 2024)Google126.9
  3. #177Claude 3 OpusAnthropic126.9
  4. #178GPT-4o miniOpenAI · $0.15 in126.6
  5. #179GPT-4 Turbo (Nov 2023)OpenAI126.5
  6. #180Llama 3.1-70BMeta125.9
  7. #181GPT-4 (Mar 2023)OpenAI125.9

Sources