Rogue employees at the cryptocurrency exchange Kraken are threatening to release sensitive customer data unless, presumably, they are rewarded with their own stash of cryptocurrency. The breach apparently stems from the compromise of members of the customer support team.
That’s pretty bad, but in truth I’m surprised we don’t see a lot more of this kind of thing. Since we lack a digital identity infrastructure capable of supporting effective know-your-employee (KYE) processes, it’s no surprise that companies find themselves employing scammers. That’s when they are employing real people at all, of course.
xxx
Testing and controls should, in theory, be able to prevent an agent from escaping a sandbox, or from getting access to information or networks it’s not authorized to access.
“These are all the things you have to tell [an AI agent]: I’m not granting you agency to do these things,” Rubinow said.
Sandboxes themselves are meant to be a good way to safely experiment with and test AI agents. In theory at least, they are safe, contained environments, protected from exposure to outside servers.
But in the Alibaba and Anthropic examples, the agents were running within sandboxes and broke out.
From: AI agents are going rogue. Here’s what banks can do about it | American Banker.
xxx
xxx
As rogue AI agents cause security problems and product outages even at sophisticated companies such as Meta and Amazon, major firms such as ServiceNow as well as startups are developing new AI to monitor and stop them.
1
You’re reading the Applied AI Newsletter
View all newsletters
The new AI, also known as guardian AI agents, comes in the form of a cloud application and can be laborious to set up. To use a guardian AI app, customers need to connect it to other AI agents they use—including ones developed with OpenClaw, Claude Code and Agentforce—using standard application programming interfaces or model context protocol servers that enable such connections or monitoring.
Customers must then tell the guardian app how the various AI agents they use should behave, given the agents’ tendency to stray from their original mission. For instance, a company might tell the guardian AI app to make sure that whenever an AI agent generates internal financial reports, it should always check Bloomberg when the reports reference stock prices.
If the report-making AI agent goes against that or another one of the company’s guidelines, the guardian agent can send an alert to employees to either stop the report-making agent or change the agent’s behavior the next time the same thing happens.
From: ‘Guardian’ Apps Aim to Stop AI Agents From Going Rogue — The Information.
xxx