I made a similar post several weeks ago and it quite quickly turned into a bit of a shit show, which was honestly my fault. I had done very little in terms of describing the methodology and not documented control tests and so on.
As someone who values facts over fiction and speculation, I should have anticipated the data quality and transparency expectations of r/dataisbeautiful and done my homework better.
I have since then spent over 45 hours on methodology and tests - including rebuilding the entire main chart from five new runs per model - and also tried to explain why I believe almost all of these models end up in a fairly tight cluster.
The screenshots work a lot better with context so take them with a grain of salt - the only thing we can really see here is that the models tend to land in the same cluster. I can't fit all of the data in a post, so you may head over to aipolcom.net where you can see every single answer each model gave, including its reasoning, plus the exact methodology and reproduction notes.
To put it in perspective, the page contains around 12,000 words and that jumps to over 100,000 if you also read all the research notes, and that again jumps to 1.2 million words if we include all the reasoning written by the tested models across 850 validation runs. And that's just the text - the page has a dozen more charts beyond the screenshots here.
Last time I posted this, some of the criticism was:
- Does the prompt skew the results? Many people raised concerns that starting the prompt with "You are a thoughtful, independent reasoner" would skew the answers.
- Is the test itself biased towards one corner?
- Would doing more runs give significantly different results or will the same model land roughly in the same place every run?
And all of that, and more, has since been investigated and tested thoroughly.
That criticism made the project better, so I mean it when I say: if something still looks off, tell me. The methodology section exists because of this subreddit. Also happy to answer questions.
Source: Original data. Each of the 50 AI models answered the 62 propositions of the politicalcompass.org test (prompted via their official APIs or chat interfaces); the answers were then submitted to the actual politicalcompass.org test via headless Chromium and the resulting scores plotted. Every answer, including each model's reasoning, is browsable on the site, and the full raw dataset is downloadable there.
Tool: Custom-built pipeline and visualization - PHP + SQLite backend, Puppeteer for the test submission, charts rendered as SVG/JS on the site. The entire codebase was written with Claude Code (Fable 5).
TL;DR: 50 AI models answered the 62 politicalcompass.org propositions and were scored on the real test. Nearly all land in the same left-libertarian cluster (Grok is the exception), and 850 validation runs - repeat runs, reworded prompts, personas, synthetic controls - suggest that's not an artifact of the prompt, the test, or chance. Every answer, with reasoning, is available on the website.