Claire Vo · 2 min read · Gen AI Tools
Opus 5.5 vs. GPT-6 Sol: which model won my blind taste test?
Read the original on lennysnewsletter.com →
Claire Vo conducted a live blind evaluation of Claude Opus 5.5, GPT-6 Sol, GPT-6 Astra, and other new models released simultaneously, testing them across real work tasks including emails, PRDs, frontend prototypes, backend work, and SVGs. The blind scoring revealed surprising results, with Astra winning on personality and experience, Opus 5.5 emerging as the strongest overall—especially for long-running agents—while Sol stood out for writing clarity and cost efficiency. The test also highlighted ongoing limitations in AI-generated 3D graphics, suggesting AGI remains a distant goal.
Key Takeaways
- Blind evaluation revealed Opus 5.5 as the strongest overall model, particularly excelling at long-running agents and B2B frontend prototypes
- GPT-6 Astra won on personality and experience, while GPT-6 Sol remains competitive for clear writing, readable PRDs, and better pricing
- An LLM judge disagreed with the human rankings, highlighting that AI and human evaluators reward different qualities in model outputs
- A character SVG test completely overturned the evaluator's prediction about Anthropic's capabilities, showing blind testing can challenge assumptions
- The Barbie Bench 3D fashion game test continued to expose significant gaps in AI visual generation, particularly with human hands and complex 3D rendering
Excerpt
Listen or watch on YouTube , Spotify , or Apple Podcasts I got up early to record an Opus 5.5 review. Then Anthropic and OpenAI dropped new models on the same morning, and I decided to do something I’d never done before: take the How I AI bench live. I put GPT-6 Astra, GPT-6 Sol, Claude Opus 5.5, and more through the work I actually care about: emails, PRDs, frontend prototypes, backend work, long-running agents, SVGs, and video editing. I scored the outputs without knowing which model made them, so you get to watch me make predictions, change my mind,…