Artificial Intelligence has crossed the threshold from lab experiment to live production. Large language models now sit behind customer-facing chat services, internal knowledge tools, developer workflows and management dashboards. The pace of adoption has been remarkable, but the security practices around these systems? These have not kept up.
That gap matters. Businesses rolling out LLMs and the infrastructure that supports them are quietly adding attack surfaces that look nothing like the ones their security teams are used to. And as a breach disclosed just this month demonstrates, the consequences of getting this wrong are already playing out in public.
The AI attack surface is different
A traditional web application is, at heart, code running on a server. An LLM deployment is something else entirely. It typically involves:
- Hosted or third-party models accessed via API
- Integration layers connecting the model to internal systems and data sources
- Training and fine-tuning pipelines that handle sensitive data at scale
- User input channels that are inherently unpredictable
- OAuth grants and SaaS tooling that extend trust to external vendors
Each of these is a potential entry point. The OWASP Foundation has been publishing research on the emerging threat landscape, flagging prompt injection, insecure output handling and data leakage as the most pressing risks. But the real-world attack chains we are starting to see go well beyond prompt manipulation.
A timely case study: the Vercel breach
Earlier this month, web infrastructure provider Vercel disclosed that attackers had gained access to internal systems and a subset of customer credentials. The stolen data was subsequently listed on BreachForums with an asking price of two million dollars.
The attack did not begin at Vercel. It began at Context.ai, a small AI productivity tool. A Context.ai employee had been infected with the Lumma Stealer malware, reportedly after downloading Roblox cheat scripts, which harvested their corporate credentials, including Google Workspace logins and keys for services such as Supabase, Datadog and Authkit.
From there, the attackers compromised OAuth tokens that Context.ai held on behalf of its users. A Vercel employee had installed the Context.ai browser extension and signed in with their enterprise Google account, granting it broad “Allow All” permissions.
Vercel’s internal OAuth configuration had permitted this. The attacker used the compromised token to take over the employee’s Google Workspace account, then pivoted into Vercel’s internal environments and extracted environment variables that were not marked as sensitive.
The lesson here is clear. No novel AI exploitation technique was needed. An infostealer, a poorly scoped OAuth grant and a lack of oversight over third-party AI tooling was all it took. This is the supply chain risk that AI adoption introduces, not in theory, but in practice.
Prompt injection and model manipulation
Prompt injection remains the most widely discussed risk in LLM security, and for good reason.
These attacks work by crafting inputs that cause the model to deviate from its intended behaviour ignoring system instructions, revealing configuration details, or producing outputs that should never reach a user.
In isolation, that is a nuisance. When the model is connected to internal APIs, databases or workflow automation, it becomes something more serious: a manipulated prompt could trigger actions, retrieve records or escalate privileges in ways the system’s designers never intended.
What makes prompt injection particularly difficult to manage is that it does not exploit a code vulnerability in the traditional sense. It exploits the way language models interpret and weigh-up natural language input. That is ultimately, what puts it outside the scope of most conventional security tooling.
Data exposure and unintended leakage
LLMs are hungry for data. Businesses routinely connect them to internal knowledge bases, document repositories, CRM systems and support ticket archives to make them useful. Which means, the model has access to information that must be carefully controlled.
Without robust access controls and data handling design, there is a genuine risk that:
- Sensitive data surfaces in model responses to unauthorised users
- Training or fine-tuning data is inadvertently disclosed
- Backend integrations expose more than intended when queried through the model
This creates an inherent design flaw that security teams need to test for explicitly.
Insecure integrations and API risk
Most AI deployments are structured using APIs. The model connects to applications, data sources and orchestration layers through API calls, and every one of those connections needs to be secured.
Poorly configured APIs can allow attackers to access AI services without authorisation, tamper with inputs and outputs, or extract data from connected systems.
The Vercel breach is instructive here as well: it was OAuth tokens and API-level trust relationships that gave the attackers their route in.
API security is not a new discipline, but in AI deployments the number of integration points, and the breadth of permissions they require, tends to be significantly higher than in a conventional application stack.
Why traditional security testing falls short
Standard vulnerability scanning will not flag a prompt injection risk. Automated application security tools will not assess whether a model’s outputs could be manipulated to trigger downstream actions.
Securing AI systems calls for a broader assessment approach, one that considers:
- How models interact with data and what they can access
- How user inputs can be manipulated to alter model behaviour
- How model outputs are consumed by other systems
- How access to AI services and the third-party tools around them is governed
- How OAuth grants and API keys are scoped, monitored and revoked
Building a secure AI deployment
Businesses bringing AI into production should be thinking about security from the outset, not bolting it on after the first incident. Key areas of focus include:
- Access control: Restrict who and what can interact with models and connected systems. Apply the principle of least privilege to every integration.
- OAuth and third-party governance: Audit which AI tools employees have connected to corporate accounts. Review the permissions granted and remove anything unnecessary. The Vercel breach could have been contained had that single OAuth grant been scoped more tightly.
- Input validation: Implement controls around how user inputs are processed. This includes filtering, monitoring and testing for prompt injection patterns.
- Output handling: Ensure that model responses cannot trigger unintended actions in downstream systems. Treat model output as untrusted input.
- Data governance: Maintain clear policies on what data models can access, store and surface. Classify and label data before connecting it to an LLM.
- Monitoring and anomaly detection: Track usage patterns, flag unusual queries and maintain audit logs. If something goes wrong, you need to know quickly.
From Experimentation to Operational Security
The shift from pilot to production is where risk compounds. An AI tool trialled by a small team with limited data access is one thing. The same tool integrated into core business processes, connected to sensitive systems and used by hundreds of employees is quite another.
How Prism Infosec Can Help
Prism Infosec works with businesses like yours to identify security risks across modern technology environments, including applications, cloud platforms and emerging AI systems.
Through penetration testing and security assessments, we help businesses understand how AI integrations may be introducing vulnerabilities, extending trust boundaries or exposing sensitive data.
Whether you are mid-deployment or still planning, it is far better to find these issues before an attacker does.
If your business is deploying AI technologies, get in touch to discuss how we can help secure your environment.
