in the loop

Gemini 1.5 Flash (May 2024)
#189 of 274 scored.

GoogleMay 23, 2024Closed

Capabilities Index

Range 115.7–124.6

122.6

Context

Tokens

—

Input

Per 1M tokens

—

Output

Per 1M tokens

—

Benchmarks

How it scores.

  • GPQA Diamond20.5%

    158th of 186

  • Mock AIME3.8%

    156th of 176

Everything else Epoch has for it

  • GSM8K82.4%
  • MMLU70.5%
  • MATH level 525.1%
  • DTBench19.0%

On the index

Around it.

  1. #186Gemma 3 12BGoogle · $0.05 in123.5
  2. #187GPT-4 (Jun 2023)OpenAI123.1
  3. #188Llama 3-70BMeta122.9
  4. #189Gemini 1.5 Flash (May 2024)Google122.6
  5. #190Gemma 2 27BGoogle · $0.65 in122.1
  6. #191Mixtral 8x22BMistral122.0
  7. #192Mistral LargeMistral · $2.00 in122.0

Sources