compar:IA: two answers, one vote, open data

You ask a question. Two models answer, with their names hidden. You vote for the answer you prefer, or skip. The names only appear afterwards. That is all compar:IA asks of you, and it is free with no account.

The principle is simple. What held our attention is what the project has built on it in two years: a public leaderboard, open datasets, and a platform that others can install themselves.

The milestones, as the project tells them

The project's repository keeps its own timeline. The public arena opened in October 2024, run by the French government. In March 2025 it reached 50,000 votes and a first dataset appeared on Hugging Face. In November 2025 a first public leaderboard was released, the project was recognised as a Digital Public Good, and a second instance, Denmark's AI-arenaen, went live. By June 2026 the counter had passed 700,000 conversations and 250,000 votes, about 89% of them in French, across more than 130 models. In September 2026 version 2.0 added message history, personal leaderboards and an admin panel.

According to the repository, funding comes from ALT-EDIC, DINUM and the French Ministry of Culture. Others contribute too: PEReN on the leaderboard, PIX, and the Danish team.

What the leaderboard measures

The leaderboard turns blind votes into Bradley-Terry scores with 95% confidence intervals. The project is careful to say that these scores measure "the preferences collected in the arena", not objective model quality. The methodology, calculations and source data are public.

That caveat matters. A model that pleases the people who visit the site is not necessarily the most accurate or the most useful for a given task. A readable leaderboard with its limits written next to it beats a score presented as the truth.

What gets published

Participants' prompts, votes and reactions are published as open datasets under the Etalab Open Licence 2.0 and CC BY 4.0 (the comparia-fr-arena dataset). The repository intends them for research, training and fine-tuning. These are real questions, mostly in French, alongside the judgement of the people who asked them. That is why MAUB files the project among those that supply resources to AI rather than those that embed one.

The model catalogue gives each model's origin and technical characteristics. Where enough technical data is available, it shows an energy-use estimate computed with EcoLogits. For proprietary models, whose size is not published, the project works from a range and flags the figure as an estimate.

What is still open

The repository lists work in progress: tool use by models, OIDC sign-in and better style control. It also records, as done, the move from Gradio to FastAPI, a migration that is invisible from the outside but that the rest depends on.

The project now presents itself as a platform that any organisation, sector or language can deploy: one server, Docker, and models and language set up in the admin panel. Denmark has done it: the repository's table now lists two instances, France and Denmark, whose dataset is listed as "coming soon".

A preference ranking, an energy estimate in the summary of each exchange, data handed back to the people who produce it: it is what the manifesto asks of an AI. If you run a language community or a sector that would like its own arena, tell us.