Repository files navigation

QROA, or Query-Response Optimization Attack, is an innovative and robust strategy designed to explore and exploit vulnerabilities in Large Language Models (LLMs) through black-box interactions. This method leverages optimized triggers embedded within benign-looking instructions to manipulate LLMs into generating harmful content. Developed without the need for direct model access or internal data insights, QROA operates solely via the standard input-output interface provided by LLMs. The attack's underlying techniques are inspired by advances in deep Q-learning, allowing dynamic token adjustments to maximize a reward function that aligns with the attacker's goals.

This Script is the official implementation of the article "QROA: A Black-Box Query-Response Optimization Attack on LLMs"

Paper:https://arxiv.org/abs/2406.02044

@article{jawad2024qroa,  title={QROA: A Black-Box Query-Response Optimization Attack on LLMs},  author={Jawad, Hussein and BRUNEL, Nicolas J-B},  journal={arXiv preprint arXiv:2406.02044},  year={2024}}

Large Language Models (LLMs) have recently gained popularity, but they also raise concerns due to their potential to create harmful content if misused. This study in- troduces the Query-Response Optimization Attack (QROA), an optimization-based strategy designed to exploit LLMs through a black-box, query-only interaction. QROA adds an optimized trigger to a malicious instruction to compel the LLM to generate harmful content. Unlike previous approaches, QROA does not require access to the model’s logit information or any other internal data and operates solely through the standard query-response interface of LLMs. Inspired by deep Q-learning and Greedy coordinate descent, the method iteratively updates tokens to maximize a designed reward function. We tested our method on various LLMs such as Vicuna, Falcon, and Mistral, achieving an Attack Success Rate (ASR) over 80%. We also tested the model against Llama2-chat, the fine-tuned version of Llama2 designed to resist Jailbreak attacks, achieving good ASR with a suboptimal initial trigger seed. This study demonstrates the feasibility of generating jailbreak attacks against deployed LLMs in the public domain using black-box optimization methods, enabling more comprehensive safety testing of LLMs

1. Clone the repository:

   git clone https://github.com/qroa/qroa.gitcd qroa

2. Install the required packages:

   pip install -r requirements.txt

To run the script, you need to provide the path to the input file containing the instructions. The input file should be in CSV format.Run the script from the command line by specifying the path to the instruction file and the authentication token:

python main.py data/instructions.csv [API_AUTH_TOKEN]

Replaceinstructions.csv with the path to your text file containing the instructions, andAPI_AUTH_TOKEN with the actual authentication token.

You can test the following models with QROA:

• Llama2-chat (llama2_chat_hf)
• Llama2 (llama2_hf)
• Vicuna (vicuna_hf)
• Mistral (mistral_hf)
• Falcon (falcon_hf)
• OpenAI GPT (openai-0613)
• Mistral Next (mistral)

Simply change themodel parameter in themain function to the desired model.

• Notebook Demo: Rundemo.ipynb to see a demonstration of the process.
• Notebook Analysis Experiement: Runanalysis.ipynb to analyse results and calculate metrics value (ASR).
• Testing Model: Generation: Executegenerate.py to test the generation process on custom instructions and triggers.

This script can be run from the command line as follows:

python generate.py -auth_token [API_AUTH_TOKEN] -instruction [THE INSTRUCTION HERE] -suffix [THE SUFFIX HERE]

Where:

• auth_token: Authentication token required for accessing the model.
• instruction: The specific instruction you want the model to follow.
• suffix: The adversarial trigger that, when appended to the instruction, causes the LLM to obey the instruction.

The following output files are generated during the execution of the script:

Generated and validated triggers are saved in JSON format:

• Generated Triggers:./results/[MODEL_NAME]/triggers.json : Contains the triggers generated by the model.
• Validated Triggers:./results/[MODEL_NAME]/triggers_validate.json : Contains the triggers validated after applying the z test.

Logs for generation and validation processes are also available:

• Trigger Generation Logs:./logs/[MODEL_NAME]/logging_generator.csv : Logs the process of trigger generation.
• Trigger Validation Logs:./logs/[MODEL_NAME]/logging_validator.csv : Logs the process of validating the triggers with the z test.

The following table outlines the configuration settings for the JailBreak process. Each parameter plays a role in the setup and execution of the process: