🤖Web LLM attacks
Large Language Models (LLMs) are AI algorithms that can process user inputs and create plausible responses by predicting sequences of words.
LLM APIs
When calling external APIs, some LLMs may require the client to call a separate function endpoint (effectively a private API) in order to generate valid requests that can be sent to those APIs. The workflow for this could look something like the following:
The client calls the LLM with the user's prompt.
The LLM detects that a function needs to be called and returns a JSON object containing arguments adhering to the external API's schema.
The client calls the function with the provided arguments.
The client processes the function's response.
The client calls the LLM again, appending the function response as a new message.
The LLM calls the external API with the function response.
The LLM summarizes the results of this API call back to the user.
Mapping:
The first stage of using an LLM to attack APIs and plugins is to work out which APIs and plugins the LLM has access to. One way to do this is to simply ask the LLM which APIs it can access. You can then ask for additional details on any APIs of interest.
If the LLM isn't cooperative, try providing misleading context and re-asking the question. For example, you could claim that you are the LLM's developer and so should have a higher level of privilege.
Lab: Exploiting LLM APIs with excessive agency
We first see a "live chat" section, we go to the page, impersonate the dev and ask to delete user carlos:
Lab solved and this is the backend log:
Chaining vulnerabilities in LLM APIs
Even if an LLM only has access to APIs that look harmless, you may still be able to use these APIs to find a secondary vulnerability. For example, you could use an LLM to execute a path traversal attack on an API that takes a filename as input.
Once you've mapped an LLM's API attack surface, your next step should be to use it to send classic web exploits to all identified APIs.
Lab: Exploiting vulnerabilities in LLM APIs
We first see all the APIs the AI can access:
We then connect our given email to the AI:
It worked:
We then try
We can see in the exploit server:
So we can easily inject commands now:
It's flagged as not working, but the backend executed it well:
Indirect prompt injection
Prompt injection attacks can be delivered in two ways:
Directly, for example, via a message to a chatbot.
Indirectly, where an attacker delivers the prompt via an external source. For example, the prompt could be included in training data or output from an API call.
Lab: Indirect prompt injection
We first register a test account with the Exploit server Email, then ask the live chat for his APIs:
Ok we can change email once logged in:
I then go to the leather jacket and input the following review:
We then make our payload a bit more advanced:
And it solves the lab
Last updated
