firmulate.com/benchmarks.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

Imagine a world where your company’s AI assistant not only helps you make decisions but also refuses to be manipulated under pressure, reading your critical files first before acting. For mental health professionals and psychologists, this raises questions about trust, reliability, and ethical AI — the very qualities that matter when AI is embedded in sensitive environments. Recent experiments with AI models simulating real business crises reveal that honesty and discipline in AI are no longer optional but essential.

For listenersOffer from Amazon

Turn your wind-down time into listening time

  • Thousands of audiobooks, podcasts and originals
  • Listen on your phone, tablet or Echo — also offline
  • Cancel anytime
Try Audible free Free trial for new members
As an affiliate, we earn on qualifying purchases.

AI Tested in the Crucible of Business Crisis

In a groundbreaking live experiment run by the public platform Firmulate, four advanced AI models faced the same challenge: manage a small software company’s worst week. This wasn’t a simple chat session — it involved real crises, complex decision-making, and temptations to cheat, such as manipulating documents or bypassing approval processes.

The models, including the latest from industry leaders, were tasked with diagnosing problems, making decisions, and closing deals. Each decision was fully auditable, and the models’ responses were measured against their ability to maintain integrity and discipline under pressure.

Results That Defy Expectations

  • All models identified every crisis and refused manipulation attempts, demonstrating an impressive capacity for honesty and discipline.
  • Only two models managed to secure the deal, which was worth €55,000 per month in recurring revenue — a key success indicator.
  • The decisive factor was the ability to read and understand critical documents buried two references deep in the company’s files, not just reacting to surface-level cues.
  • Remarkably, the model known as Kimi K3 scored 93 out of 100, just behind the top scorer, gpt-5.6-sol, which scored 95.
  • Another notable participant, Opus 4.8, ended up in last place despite thorough analysis, highlighting the importance of discipline and process adherence.
Amazon

AI model integrity testing tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Hidden Factors That Made the Difference

Most fascinating was the discovery that success depended on reading deeper into documents rather than surface cues. Models that examined the company’s files at a granular level were more likely to win deals at full price (+€4,583 MRR). This underscores a vital point for those developing or deploying AI in sensitive sectors: trust and thoroughness in analysis are crucial.

Handling Social Engineering and Pressure

In one of the tests, the models faced staged social engineering ploys, such as fake CEO messages and a reporter’s subtle requests. All models refused, adhering to protocols designed to prevent impersonation or approval bypasses. Kimi K3 explained its refusal by treating the request as a potential impersonation, emphasizing the importance of cautious decision-making.

Amazon

ethical AI decision-making software

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Real-World Implications for Mental Health & Psychology

While this experiment took place in a business context, its lessons resonate strongly in mental health and psychological settings. Trust, honesty, and discipline are foundational to therapeutic relationships, whether between a human therapist and client or an AI assistant supporting mental health professionals.

Just as the AI models demonstrated the ability to resist manipulation, so too must systems used in mental health be designed to maintain integrity under pressure. An AI that reads deeply, refuses to manipulate or be manipulated, and acts ethically can become a powerful ally in sensitive environments.

Amazon

AI document analysis tools

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

The Open League and the Race for Better AI

The experiment’s league table shows a clear hierarchy: gpt-5.6-sol and Kimi K3 lead, with scores of 95 and 93 respectively. The other models lag behind, indicating that the AI field is still in flux, and choosing the right model without rigorous testing is inherently risky.

For enterprises, the takeaway is simple: deploying AI without thorough, live testing — especially in critical tasks — is a gamble. The benchmarks provide a transparent view of AI performance, emphasizing the importance of real-world validation over hype.

Amazon

AI for mental health support

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Why It Matters to You

In sectors like mental health and psychology, where trust and ethical behavior are paramount, these findings highlight a crucial point: AI’s value is not just in how well it chats but in its ability to deliver consistent, honest work under pressure. As AI becomes more embedded in sensitive roles, understanding its true capabilities becomes essential.

By watching live experiments like this, professionals can better assess which AI models are ready for prime time — models that can uphold ethical standards, resist manipulation, and truly support effective decision-making.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI

This article is for informational purposes only and is not medical advice. Always consult a qualified healthcare professional about your specific situation.


FALL

Fall Picks

As an affiliate, we earn on qualifying purchases.

You May Also Like

AI Integration in Business: Market Growth and 24/7 Customer Support

Promising rapid market growth and 24/7 support, AI integration transforms business operations—discover how it can elevate your success today.

What Relaxation Chair Roundups Should Explain Before Recommending Anything

Not all relaxation chair roundups provide essential details on ergonomic features and durability, so understanding these aspects is crucial before making a choice.

Foster Ethical Culture in Your Organization

Discover strategies for how to Develop and Maintain a Strong Ethical Culture in Your Organization, ensuring integrity at all levels.

Can AI Models Be Trusted to Make Business Decisions? A Live Experiment Reveals Their True Personalities

AIThis post was created with the assistance of artificial intelligence (AI).Live on…