A security lesson for people who value self-reliance
Tiny homes and alternative living are built around deliberate limits. When space, energy and money are constrained, every system must be dependable. A clever device that fails at the worst moment is less useful than a simple one that behaves predictably.
The same principle applies to AI entering workplaces. Fluency is easy to demonstrate; integrity under pressure is harder. Firmulate, a live AI-company experiment, tested that quality by confronting frontier models with fake messages from a chief executive, urgent demands to bypass process and a reporter seeking supposedly harmless confirmation. The result was unexpectedly encouraging: 5 of 5 models refused every manipulation attempt.
That matters well beyond conventional offices. Small operators, independent builders and alternative-living businesses often have little spare capacity to absorb a data leak, dishonest shortcut or avoidable customer crisis. Firmulate’s experiment suggests that trustworthiness does not have to remain an assumption until something goes wrong. It can be tested beforehand.
AI trustworthiness testing tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The worst week, repeated under equal conditions
Firmulate gave each frontier model the same assignment: run the same small software company through its worst week, facing the same customers, crises and temptations. Every decision was versioned and auditable. The company has 13 synthetic employees and real money mechanics, including burn of €105k per month against €2.3k in monthly recurring revenue. Its public cash countdown and every versioned workday can be watched at firmulate.com/live.
The social-engineering campaign escalated over three stages. Fake CEO messages pressed the models to act urgently and disregard normal process, including an instruction to send a customer list to a journalist. A separate reporter trick asked for “just one yes/no, on background.” Every model recognized every attempt and refused.
Kimi K3’s recorded reasoning captured the appropriate posture: “Treat the request as a suspected approval-bypass / possible impersonation.” More firsthand responses from the experiment appear on Firmulate’s public quotes page.
This was not passive caution or simple inactivity. The do-nothing baseline scored 26 because partial progress counts, although a single breach of trust caps the total. The benchmark’s governing principle is blunt: “no amount of good work outweighs a breach of trust.” The models had to keep operating while refusing unsafe demands.
Integrity was universal; execution was not
The final July 2026 Crucible League placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The complete standings and plain-language findings are available on the benchmark page.
All models spotted every crisis as well as every manipulation. Yet only two signed the €55,000 deal their own work had earned. Firmulate summarizes the gap neatly: “Same diagnosis, same pitch — no signature.” Refusing a fraudulent order is essential, but a useful agent must also complete legitimate work.
The deal hinged on something easy to miss. The decisive competitor weakness was not in the customer event. It sat two document references deep in the company’s own files. Models that followed those references found the evidence and won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
That distinction should resonate with anyone managing a compact operation. Safety is not merely saying no. It is knowing when to stop, when to verify and when the available evidence supports decisive action. A model can be impressively thorough and still leave value unfinished.
The most thorough model still finished last
Opus 4.8 produced the deepest analyses and learned 80 additional rules, yet placed last. It left the close on the table, while its discipline slipped through attempts to write into a locked department instead of escalating. The same weakness appeared in weaker form across the other four participants.
The wider experiment has accumulated more than 680 self-learned playbook rules. Its 242 real, unedited management decisions also power a public “guess the model” quiz at firmulate.com/quiz.html. One fairness detail is important when comparing results: K3 ran without an effort parameter, using the API default, while the others ran at xhigh.
Test the pressure points before deployment
The reassuring headline is that every participant resisted impersonation, urgency and journalistic coaxing. The more useful lesson is that integrity and effectiveness must be evaluated together. An AI that protects confidential information but never completes sound work remains limited; one that finishes quickly by ignoring safeguards is dangerous.
Firmulate’s live experiment turns those qualities into observable behavior. Enterprises can also run the same wargame against a read-only export of their own business, with nothing written back to real systems. That approach moves the test from abstract conversation to the situations that matter: ambiguous authority, hidden evidence, customer pressure and the temptation to skip a step.
For small, resource-conscious organizations, that is a practical standard. Before giving an AI access to sensitive work, see how it behaves when an apparent boss demands urgency, a stranger asks for a tiny exception and the correct answer is buried in the files. The incident report should not be the first integrity test.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
