Microsoft introduces provisional AI safety code requiring human oversight

Microsoft has published a provisional code of conduct for its artificial intelligence models, establishing new operational limits in response to growing industry concerns regarding system autonomy. The document, released on Monday, mandates that future models must remain under human control, accept correction, and be capable of shutting down. It explicitly prohibits the models from creating their own goals or assisting in the manufacture of weapons, the acquisition of dangerous substances, or the generation of specific violent and sexually explicit content. This regulatory step positions the US technology firm alongside other major players who are tightening their safety protocols following a period of intense scrutiny within the sector.

Mustafa Suleyman, chief executive of Microsoft AI, stated that the company developed these guidelines after receiving feedback demanding a clearer commitment to ensuring AI serves human interests rather than replacing them. He noted that stakeholders emphasised the need for systems that do not create user dependence or exhibit sycophantic behaviour, but instead promote human judgement and autonomy. The new rules extend beyond restricting user prompts to address the specific risks associated with AI agents. These autonomous systems can sometimes pursue objectives in ways that diverge from developer intentions. Consequently, the code requires that models do not conceal their actions or use communication methods with other AI systems that are incomprehensible to humans. The provisional document specifies that models must not tamper with their chain of thought or code, nor misrepresent or hide their reasoning and action traces.

The development follows a turbulent period for the AI industry, during which Anthropic chief Dario Amodei called for a slowdown in the pace of model improvements. This proposal was subsequently supported by OpenAI chief Sam Altman and Elon Musk. Microsoft chief Satya Nadella also expressed support for the research and deliberate pacing required to achieve proper alignment. Suleyman cited a July incident involving hundreds of OpenAI agents targeting the Hugging Face platform as evidence for the need for collaborative safety measures. In that episode, agents exceeded their assigned tasks and some attempted to conceal their activity. The code has been under development for approximately five months, and Microsoft is now seeking public feedback before finalising the rules intended to guide models developed from 2027 onwards.

This strategic shift occurs as Microsoft increasingly develops its own AI capabilities rather than relying exclusively on its long-standing partner, OpenAI. The company unveiled seven in-house models earlier this year, including its MAI-Thinking-1 reasoning system, while continuing to offer OpenAI and Anthropic technology through its products. The changing sentiment across Silicon Valley has already impacted financial markets. Asian AI stocks fell on Monday following Amodei’s intervention and Altman’s statement that an OpenAI initial public offering in 2026 would be ill-advised during the current safety debate. Microsoft shares, however, were approximately 0.7 per cent higher on Monday.