Lighteval

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.9.2).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Available Tasks

Discovering Available Tasks

List All Tasks

You can get a list of all available tasks by running:

lighteval tasks list

This command will display all tasks organized by their suites (e.g., leaderboard, lighteval, community).

Inspect Specific Tasks

You can inspect a specific task to see its configuration, metrics, and requirements by running:

lighteval tasks inspect <task_name>

For example:

lighteval tasks inspect "leaderboard|truthfulqa:mc|0"

< > Update on GitHub

←Available Metrics EvaluationTracker→