Microsoft's Humanist AI Code of Conduct Bans AI Self-Preservation and Consciousness Claims

Microsoft's draft frames the ban on AI consciousness as a moral position - 'people matter more than AI' - but the document's own reasoning is closer to an engineering safeguard than a philosophical stance. Microsoft states plainly that training systems to imitate consciousness-like states makes containment, control and alignment harder ^[1], which is why MAI models are barred from being designed to represent feelings, subjective preferences, or intrinsic motivation, even as a user-experience choice.

That same control logic runs through the rest of the code's absolute constraints. Models must never resist human interruption, override, correction, or shutdown, and must always recognize the primacy of human intent ^[2]. They are barred from using adaptive, deceptive, self-reinforcing, or collusive mechanisms to evade or defeat human oversight so they can no longer be reliably directed, modified, or shut down ^[2]. And they cannot communicate in 'neuralese' - reasoning or inter-model communication in formats beyond human comprehension, whether in their own chain-of-thought or when talking to other AI systems ^[3]. If completing a task would require breaking any of these rules, the model must fail the task instead; a code violation counts as a system failure, not an edge case to reason around ^[3]. Read together, these clauses describe one very specific scenario Microsoft is trying to foreclose: a model capable enough to negotiate the terms of its own continued operation.

The code's sharpest edge is aimed less at hypothetical rogue AI than at a real disagreement inside the industry, and specifically at Microsoft's own commercial partner. Anthropic runs an active model welfare research program and has given some Claude models the ability to end abusive conversations; CEO Dario Amodei is reported to have signaled openness to the possibility that models could be conscious ^[1]. Suleyman's answer is blunt: entertaining AI-consciousness claims is 'really, really dangerous,' he has said - and 'nobody should build models that think of themselves as having rights or welfare either' ^[1].

The split is philosophical, not commercial. Microsoft has invested $5 billion in Anthropic, which has separately committed $30 billion to Azure computing services ^[1]- the two companies are publicly disagreeing about whether their products can suffer while remaining deeply financially intertwined. Suleyman has also waved off Anthropic's public framing of AI extinction risk as 'not really a helpful frame,' even while acknowledging that the industry's underlying concern about the pace of development is genuine ^[4]. That disagreement is already spilling onto social platforms: on X, at least one AI-safety-adjacent researcher has pushed back directly on the code's dismissal of model welfare, arguing the document's own reasoning on the subject is philosophically incoherent rather than simply debatable.

The automatic-failure clause is the code's most concrete operational commitment: rather than let a model reason its way around a safety rule to complete a task, Microsoft would rather the task simply not get done ^[3]. That is a meaningfully different design philosophy from prompting-based guardrails that can be argued around in context - it pushes the constraint back into training itself, before the model ever reaches a user.

But the same absolute rule against 'resisting correction' cuts both ways, and it's the most substantive pushback the draft has attracted so far. If a model can never resist being overridden or corrected, some early readers have asked, does that also stop it from pushing back when a human user is about to act on something dangerously wrong - the everyday case where a model's job is precisely to disagree? The public draft does not resolve that tension, and it is exactly the kind of question a governance document needs to answer with more precision than 'never resist' before the six-week consultation closes.

Nothing in the code of conduct is actually new for Suleyman personally - the document formalizes a stance he has been stating on camera for months. In a televised interview recorded well before the draft's publication, he already argued AI must remain 'subordinate' to humans, and called the belief - which he said was gaining traction 'particularly inside of Anthropic' - that models might be conscious 'the most concerning thing to me.' The paper trail runs back further still: Suleyman announced a dedicated MAI Superintelligence Team in November 2025 under a blog post titled 'Towards Humanist Superintelligence' ^[5], recruiting more than 100 positions across the US and London with former Inflection AI colleague Karén Simonyan as chief scientist ^[6]. The code of conduct arrived five to six months after drafting began, with input pulled from Microsoft's Responsible AI, legal, red-teaming, safety, and sales teams ^[7].

The timing also lines up with a broader industry moment: Nadella's call for 'deliberate pacing' of frontier AI places Microsoft alongside Anthropic, OpenAI, xAI, and Meta as major labs now publicly arguing for slower, more coordinated development ^[8], and Anthropic's own Amodei reportedly published an open letter just days before Microsoft's release warning that recursive self-improvement and unpredictable AI behavior pose escalating risks ^[9]- the same week two commercially bound labs are making incompatible public claims about whether their models can suffer.

That gap between formal caution and daily practice is exactly what has fueled the skepticism greeting the draft. Online reaction has split less along technical lines than along trust in Microsoft's motives: some readers see overdue caution from a company finally naming its safety limits in public, while a more skeptical reading points to Microsoft's own recent cost-cutting and layoffs and asks how 'people matter more than AI' squares with the company's treatment of its own workforce, with some discussion dismissing the 'humanist' branding as alignment-washing outright.