in the loop

Claude 3 Sonnet
#195 of 274 scored.

AnthropicFeb 29, 2024Closed

Capabilities Index

Range 113.2–123.9

120.7

Context

Tokens

—

Input

Per 1M tokens

—

Output

Per 1M tokens

—

Benchmarks

How it scores.

  • GPQA Diamond20.8%

    156th of 186

  • Mock AIME2.4%

    159th of 176

Everything else Epoch has for it

  • MMLU67.9%
  • Winogrande50.2%
  • DTBench22.7%
  • MATH level 518.2%
  • WeirdML10.2%

On the index

Around it.

  1. #192Mistral LargeMistral · $2.00 in122.0
  2. #193phi-3-small 7.4BMicrosoft121.8
  3. #194phi-3-medium 14BMicrosoft121.2
  4. #195Claude 3 SonnetAnthropic120.7
  5. #196Claude InstantAnthropic120.3
  6. #197Claude 2Anthropic120.1
  7. #198Gemma 2 9BGoogle119.8

Sources