Multi-task Language Understanding on MMLU ; 1. Claude 3.5 Sonnet (5-shot). 88.7 ; 2. GPT-4o. 88.7 ; 3. Llama 3.1 405B (CoT). 88.6 ; 4. Tencent Hunyuan Large. 88.4. MMLU (Massive Multitask... · Gemini · The Llama 3 Herd of Models

23 июн. 2023 г. · The Open LLM Leaderboard is actually just a wrapper running the open-source benchmarking library Eleuther AI LM Evaluation Harness.

1 мая 2024 г. · The HELM MMLU leaderboard provides comprehensive MMLU evaluations of models using simple and standardized prompts, and provides full ...

The MMLU Benchmark (Massive Multi-task Language Understanding) is a challenging test designed to measure a text model's multitask accuracy.

Leaderboard. edit. Organisation, LLM, MMLU. OpenAI, o1-preview, 90.8. Anthropic, Claude 3.5 Sonnet, 88.7. Meta, Llama-3.1 405B, 88.6. xAI, Grok-2, 87.5.

Multi-task Language Understanding on MMLU ; 1. Claude 3 Opus (5-shot, CoT). 88.2 ; 2. GPT-4 (few-shot). 86.4 ; 3. Claude 2 (5-shot). 78.5 ; 4. Claude 1.3 (5-shot).

Accuracy breakdown for each of the 57 subjects; Full transparency of all raw prompts and predictions. Blog PostFull Leaderboard. Model.

LLM Leaderboard ; Best in Multitask Reasoning (MMLU). Data from the MMLU benchmark - Geneal capabilities & reasoning. ; Best in Coding (Human Eval). Data from the ...

Track, rank and evaluate open LLMs and chatbots.

iAsk Pro's MMLU Pro Benchmark Results 85.85% Exceeds Human Experts and Reaches AGI Level. The MMLU-Pro dataset is the most comprehensive and demanding multi- ...