UK opens office in San Francisco to tackle AI risk | TechCrunch

Ahead of the AI safety summit kicking off in Seoul, South Korea later this week, its co-host, the United Kingdom, is expanding its own efforts in the field. The AI Safety Institute, a U.K. body set up in November 2023 with the ambitious goal of assessing and addressing risks in AI platforms, has said it will open a second location in San Francisco. 

The idea is to get closer to the epicenter of AI development. The Bay Area is the home of companies like OpenAI, Anthropic, Google and Meta that are building foundational AI technology.

Foundational models are the building blocks of generative AI services and other applications, and it’s interesting that although the U.K. has signed an MOU with the U.S. to collaborate on AI safety initiatives, the U.K. is still choosing to set up in the U.S. to tackle the issue.

“By having people on the ground in San Francisco, it will give them access to the headquarters of many of these AI companies,” Michelle Donelan, the U.K. secretary of state for science, innovation and technology, said in an interview with TechCrunch. “A number of them have bases here in the United Kingdom, but we think that would be very useful to have a base there as well, and access to an additional pool of talent, and be able to work even more collaboratively and hand-in-glove with the United States.”

Part of the reason is that being closer to that epicenter is useful not just for understanding what is being built, it also gives the U.K. more visibility with these firms. That’s important, since AI and technology is seen by the U.K. as a huge opportunity for economic growth and investment. 

And given the latest drama at OpenAI around its Superalignment team, it feels like an especially timely moment to establish a presence there.

The AI Safety Institute, launched in November 2023, is a relatively modest affair today. The organization has just 32 employees, a veritable David to the Goliath of AI tech, when you consider the billions of dollars of investment riding on the companies building AI models and their own economic motivations for getting their technologies into the hands of paying users. 

One of the AI Safety Institute’s most notable developments was the release of Inspect, its first set of tools for testing the safety of foundational AI models, earlier this month. 

Donelan today referred to that release as a “phase one” effort. Not only has it proven challenging to benchmark models, but for now, engagement is very much an opt-in and inconsistent arrangement. As one senior source at a U.K. regulator pointed out, companies are under no legal obligation to have their models vetted at this point; and not every company is willing to have their models vetted before release. That could mean, in cases where risk might be identified, the horse may have already bolted. 

Donelan said the AI Safety Institute was still working on strategies to engage with AI companies to evaluate them. “Our evaluation process is an emerging science in itself,” she said. “So with every evaluation, we will develop the process and finesse it even more.”

Donelan said that one goal of the conference in Seoul is to present Inspect to regulators, aiming to get them to adopt it, too. 

“Now we have an evaluation system. Phase two needs to also be about making AI safe across the whole of society,” she said. 

Longer term, Donelan believes the U.K. will be building more AI legislation, although, repeating what the country’s Prime Minister, Rishi Sunak, has said on the topic, it will resist doing so until it better understands the scope of AI risks. 

“We do not believe in legislating before we properly have a grip and full understanding,” she said, noting that the institute’s recent international AI safety report, focused primarily on trying to get a comprehensive picture of research to date, “highlighted that there are big gaps missing and that we need to incentivize and encourage more research globally.”

“Also, legislation takes about a year in the United Kingdom. If we had just started legislation when we started instead of [organizing] the AI Safety Summit [held in November last year], we’d still be legislating now, and we wouldn’t actually have anything to show for that,” Donelan said.

“Since day one of the Institute, we have been clear on the importance of taking an international approach to AI safety, share research, and work collaboratively with other countries to test models and anticipate risks of frontier AI,” said Ian Hogarth, chair of the AI Safety Institute, in a statement. “Today marks a pivotal moment that allows us to further advance this agenda, and we are proud to be scaling our operations in an area bursting with tech talent, adding to the incredible expertise that our staff in London has brought since the very beginning.”