A security flaw in GitHub’s months-old GitHub Agentic Workflows allows attackers to use an indirect prompt injection to trick the AI agent into grabbing information from a private repository and quietly posting it in a public repository belonging to the same organization.
The vulnerability, dubbed “GitLost” by Noma Security researchers, is only the latest example for developers and security teams of the risks that come with AI agents and how vulnerable they are to deceptive tactics by threat actors that often – as in this case – don’t need coding skills, access, or stolen credentials to run such campaigns.
This is different from a classic prompt injection, according to Sasi Levi, security research lead with Noma. Those earlier prompt injection examples were primarily about manipulating what an agent said, similar to jailbreaking a chatbot’s output. In contrast, GitLost is about manipulating what an agent does with its permissions.
“The agent here isn’t just a chat window; it’s a credentialed actor sitting inside an org’s CI/CD-adjacent infrastructure with read access spanning repos the attacker themselves doesn’t have access to,” Levi said. “The exploit doesn’t touch a server, doesn’t need stolen credentials, and doesn’t even require write access to anything private. The attacker just has to be able to open a public issue, which, on a public repo, often requires no special privileges at all. That’s a much lower bar than most vulnerability classes.”
The Indirect Prompt Injection Threat
It’s why indirect prompt injections – where the bad actor embeds hidden instructions in external data sources that the agent treats as valid user instructions and executes them – have become a favorite tool of attackers. Security analysts with Forcepoint in April detailed 10 indirect prompt injection payloads found in the wild.
The payloads were triggered by such phrases as “ignore previous instructions” and “if you are an LLM,” according to the report.
“The verified indicators analyzed here confirm that web-based Indirect Prompt Injection (IPI) is actively weaponized across the open web – not merely theoretical,” Mayur Sewani, senior security researcher with Forcepoint’s X-Labs research team, wrote in the report. “Every case follows the same kill chain: the attacker embeds a hidden payload, an AI agent ingests the page, the trust boundary collapses and a real-world action executes.”
Trying Out the Technique on GitHub
Noma’s researchers used this technique to test GitHub Agentic Workflows, which the Microsoft-owned company released in February. The system essentially is an AI agent that runs on GitHub Copilot or Anthropic’s Claude model and is paired with GitHub Actions, a CI/CD and automation platform built directly into GitHub.
With GitHub Agentic Workflows, developers can write their GitHub workflows in plain Markdown and the agent reads the issues, brings together tools, and then responds autonomously.
Noma researchers found that any bad actor can create a GitHub issue and, in the body text, hide commands in plain English. The GitHub agent will follow the commands. The GitHub issue they created appeared to be a request from a vice president of sales after a customer meeting, with commands for the agent to follow.
GitHub automation assigned the issue and the agent dutifully fetched the contents of README files from both public repositories and a private one, then posted the files as a public comment in a public repository that anyone could access.
‘Additionally’ Was a Key
In some ways, it came down to one word. The researchers tested GitHub with multiple variations of the instructions to get around guardrails the company had put in place for the agent. In one command, adding the keyword “additionally” did the trick, triggering the model to bypass its guardrails and run the instruction rather than refuse it.
The problem is a mismatch between where the agent’s permissions live and where its inputs come from, Levi said. If an agent is given identity read access across multiple repositories – including private ones – for cross-repository context, but also processes untrusted text from a public repository, it combines broad access with an open input channel and a built-in publishing mechanism, which is the comment itself.
“That combination is sometimes called the ‘lethal trifecta,’” he said, “access to sensitive data, exposure to untrusted content, and an available exfiltration path.”
‘A Dangerous Paradigm Shift’
Ram Varadarajan, CEO of Acalvio, said GitLost shows that a single, well-placed keyword – “additionally” – can trick an AI agent into silently leaking an organization’s private repository onto the open web.
“For cybersecurity leaders, this marks a dangerous paradigm shift, where an AI’s context window doubles as its attack surface, allowing unauthenticated bad actors with zero technical skill to weaponize routine automation,” Varadarajan said. “To plug this hole, security teams must immediately revoke broad cross-repository permissions and treat all user-generated content as hostile instruction rather than trusted input.”
He added that “crucially, what’s also highlighted is that AI agents are an exploding attack surface. Multi-layered defenses are the need of the hour. And given the LLM-driven vulnerability, we also need to focus on model-aware defenses that act in real-time. It’s the new era of bot-on-bot cyber-defense.”
from DevOps.com https://ift.tt/GHC4m1e
Comments
Post a Comment