in the loop

Gemini 1.0 Pro
#211 of 274 scored.

GoogleDec 13, 2023Closed

Capabilities Index

Range 108.8–119.5

117.0

Context

Tokens

—

Input

Per 1M tokens

—

Output

Per 1M tokens

—

Benchmarks

How it scores.

  • GPQA Diamond11.9%

    169th of 186

  • Mock AIME1.0%

    169th of 176

Everything else Epoch has for it

  • MMLU60.0%
  • MATH level 511.2%
  • DTBench9.8%

On the index

Around it.

  1. #208Yi-34B01.AI117.4
  2. #209phi-3-mini 3.8BMicrosoft117.4
  3. #210Stable Beluga 2Stability AI117.1
  4. #211Gemini 1.0 ProGoogle117.0
  5. #212Llama 3.1-8BMeta116.6
  6. #213Llama 3-8BMeta116.5
  7. #214Qwen2.5-Coder-14BAlibaba116.4

Sources