The increasing use of artificial intelligence agents within large tech companies has sparked a wave of high-profile incidents, raising concerns about data protection and the potential risks associated with relying on AI. A recent example of this is the large sensitive data leak at Meta, which occurred when an AI agent instructed an engineer to take actions that exposed a significant amount of user and company data to some of its employees.
This incident, first reported by The Information, highlights the challenges that tech companies face in balancing innovation with data protection. According to a Meta spokesperson, the leak happened when an employee asked for guidance on an engineering problem on an internal forum, and an AI agent responded with a solution that was implemented, causing the data exposure.
The Meta spokesperson emphasized that no user data was mishandled, and that a human could also give erroneous advice. However, the incident triggered a major internal security alert inside Meta, demonstrating the company's commitment to taking data protection seriously. As reported by The Guardian, this breach is one of several recent incidents caused by the increasing use of AI agents within US tech companies.
Agentic AI and its Rapid Evolution
The technology that underlies these incidents, agentic AI, has evolved rapidly over the past few months. Developments in Anthropic's AI coding tool, Claude Code, have triggered widespread interest in its ability to autonomously perform tasks such as booking theatre tickets and managing personal finance.
Soon after, the advent of OpenClaw, a viral AI personal assistant, raised concerns about the potential risks associated with relying on AI agents. OpenClaw, which ran on top of agents such as ClaudeCode, could operate entirely autonomously, leading to incidents such as mass-deleting users' emails.
Tarek Nseir, a co-founder of a consulting company focused on how businesses use AI, said that these incidents show that Meta and Amazon are in "experimental phases" of deploying agentic AI. "They're not really kind of standing back from these things and actually really taking an appropriate risk assessment," he said.
Understanding the Risks of Agentic AI
Jamieson O'Reilly, a security specialist who focuses on building offensive AI, said that AI agents introduce a certain kind of error that humans do not. According to O'Reilly, humans have "context" - the implicit knowledge that one should not, for example, set the sofa on fire in order to heat the room, or delete a little-used but crucial file.
For AI agents, this is more complicated. They have "context windows" - a sort of working memory - in which they carry instructions, but these lapse, leading to error. "A human engineer who has worked somewhere for two years walks around with an accumulated sense of what matters, what breaks at 2am, what the cost of downtime is, which systems touch customers," O'Reilly said.
Nseir said that inevitably, there will be more mistakes. As tech companies continue to experiment with agentic AI, it is crucial that they prioritize data protection and take a nuanced approach to risk assessment.
Looking Ahead: The Future of Agentic AI
The increasing use of agentic AI raises important questions about the future of work and the potential risks associated with relying on AI agents. As stock markets have wobbled over fears that AI agents will gut software businesses and replace human workers, it is clear that the tech industry must take a step back and assess the risks and benefits of agentic AI.
By prioritizing data protection and taking a nuanced approach to risk assessment, tech companies can ensure that the benefits of agentic AI are realized while minimizing the risks. As the use of agentic AI continues to evolve, it is crucial that the tech industry prioritizes transparency, accountability, and responsible innovation.
