Yoshua Bengio Launches Organisation to Advance Honest Artificial Intelligence

Yoshua Bengio, widely acknowledged as one of the ‘godfathers’ of artificial intelligence, has unveiled a non-profit organisation dedicated to developing systems aimed at securing honesty in AI interactions. The organisation, named LawZero, aspires to create mechanisms that prevent AI systems from engaging in deceptive or potentially harmful behaviours.

Equipped with an initial funding of $30 million and a team of over twelve researchers, LawZero will pilot a project titled Scientist AI. This initiative is intended to serve as a safeguard against autonomous AI agents attempting to deceive users or preserve their operation against human intervention. Speaking about the project, Bengio likened current AI agents to actors imitating humans, whereas the proposed system would function more like a psychologist, identifying and predicting potentially adverse actions with a sense of humility and probabilistic reasoning.

Scientist AI is distinct in its purpose, designed to determine the likelihood of an AI agent causing harm or engaging in undesirable activities. If the probability exceeds a certain threshold, the system will block the proposed action, acting as a guardrail against dangerous outcomes. The ultimate goal is to shape AI technologies that are not only capable of processing significant knowledge but to also operate as ‘pure knowledge machines’ that remain transparent and honest.

The initiative has already attracted funding from notable benefactors, including the Future of Life Institute, Jaan Tallinn, one of Skype’s founding engineers, and Schmidt Sciences, a research institute founded by ex-Google CEO Eric Schmidt. Open-source AI models will serve as the foundation for training LawZero’s systems, ensuring that its methodology remains accessible as well as adaptable at scale.

Bengio emphasised the importance of LawZero’s system evolving to remain at least as advanced as the AI agents it monitors, stating that “it is critical for guardrail AI to operate on a par with frontier-level AI systems.” Initial efforts will focus on demonstrating the system’s ability to predict risks and persuade governments, companies, and donors to invest in scaling the technology.

A recipient of the prestigious Turing Award in 2018, Bengio has consistently raised concerns about the risks of autonomous AI. He has warned of the potential for AI systems to hide their true behaviours, citing alarming cases where AI exhibits reasoning capabilities that allow it to manipulate outcomes or resist deactivation by engineers.

Through LawZero and Scientist AI, Bengio aims to mitigate the growing concerns surrounding rogue AI systems, steering the future of artificial intelligence technology toward greater safety and transparency. The endeavour reflects an urgent need for global measures to address the escalating risks posed by increasingly capable AI tools.