MMLU (Massive Multitask Language Understanding) is a new benchmark designed to measure knowledge acquired during pretraining by evaluating models exclusively ...

The current state-of-the-art on MMLU (Clinical Knowledge) is Med-PaLM 2 (ER). See a full comparison of 3 papers with code.

This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge.

It consists of about 16,000 multiple-choice questions spanning 57 academic subjects including mathematics, philosophy, law, and medicine. It is one of the ...

We're on a journey to advance and democratize artificial intelligence through open source and open science.

See what others are saying about this dataset · What have you used this dataset for? · How would you describe this dataset?

This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge. The test spans subjects in the humanities, social ...

MMLU-Medical [33] : MMLU was originally designed to assess the world knowledge of models across various subjects including mathematics, physics, history, and ...

A subset of six tasks that are related to biomedicine are selected from MMLU, including anatomy, clinical knowledge, professional medicine, human genetics, ...

Recent advancements in large language models (LLMs) such as ChatGPT and LLaMA have hinted at their potential to revolutionize medical applications, ...