Defending LLMs against Jailbreaking Attacks via Backtranslation

2402.16459

YC

67

Reddit

0

Published 6/10/2024 by Yihan Wang, Zhouxing Shi, Andrew Bai, Cho-Jui Hsieh
Defending LLMs against Jailbreaking Attacks via Backtranslation

Abstract

Although many large language models (LLMs) have been trained to refuse harmful requests, they are still vulnerable to jailbreaking attacks which rewrite the original prompt to conceal its harmful intent. In this paper, we propose a new method for defending LLMs against jailbreaking attacks by ``backtranslation''. Specifically, given an initial response generated by the target LLM from an input prompt, our backtranslation prompts a language model to infer an input prompt that can lead to the response. The inferred prompt is called the backtranslated prompt which tends to reveal the actual intent of the original prompt, since it is generated based on the LLM's response and not directly manipulated by the attacker. We then run the target LLM again on the backtranslated prompt, and we refuse the original prompt if the model refuses the backtranslated prompt. We explain that the proposed defense provides several benefits on its effectiveness and efficiency. We empirically demonstrate that our defense significantly outperforms the baselines, in the cases that are hard for the baselines, and our defense also has little impact on the generation quality for benign input prompts. Our implementation is based on our library for LLM jailbreaking defense algorithms at url{https://github.com/YihanWang617/llm-jailbreaking-defense}, and the code for reproducing our experiments is available at url{https://github.com/YihanWang617/LLM-Jailbreaking-Defense-Backtranslation}.

Get summaries of the top AI research delivered straight to your inbox:

Overview

  • This paper explores ways to defend large language models (LLMs) against "jailbreaking" attacks, where users try to bypass the model's intended behavior and get it to generate harmful or unethical content.
  • The authors propose using a technique called "backtranslation" to detect and mitigate these attacks.
  • Backtranslation involves translating the model's output to another language and then translating it back, checking for discrepancies that could indicate an attack.

Plain English Explanation

The paper focuses on protecting powerful AI language models, known as large language models (LLMs), from being misused or "jailbroken" by users. Jailbreaking refers to finding ways to bypass the safeguards and intended behavior of an LLM, in order to get it to generate harmful, unethical, or undesirable content.

The researchers suggest using a technique called backtranslation to detect and stop these jailbreaking attacks. Backtranslation involves taking the text generated by the LLM, translating it to another language, and then translating it back. If there are significant differences between the original text and the backtranslated version, it could be a sign that the LLM has been jailbroken and is producing content that deviates from its normal, intended behavior.

By monitoring for these discrepancies, the researchers believe they can identify and mitigate jailbreaking attacks on LLMs, helping to keep these powerful AI systems from being misused.

Technical Explanation

The paper proposes using backtranslation as a defense mechanism against jailbreaking attacks on large language models (LLMs). Jailbreaking attacks involve finding ways to bypass the intended behavior and safety constraints of an LLM, in order to get it to generate harmful or undesirable content.

To detect these attacks, the authors suggest translating the LLM's output to another language and then translating it back, comparing the original and backtranslated versions. If there are significant discrepancies, it could indicate that the LLM has been jailbroken and is producing content that deviates from its normal behavior.

The researchers evaluated this backtranslation approach on several LLMs, including GPT-3, and found that it was effective at identifying jailbreaking attempts. Their results showed that backtranslation could reliably detect when the models were being misused, even in the face of sophisticated jailbreaking techniques.

Critical Analysis

The paper presents a promising defense against jailbreaking attacks on LLMs, but there are some potential limitations and areas for further research:

  • The backtranslation approach relies on the availability of high-quality translation models, which may not always be reliable or accessible, especially for less common language pairs.
  • The authors only tested their method on a limited set of LLMs and jailbreaking techniques. More comprehensive evaluations would be needed to fully understand its robustness.
  • The paper does not address the potential for subtle, incremental jailbreaking that could gradually erode the model's intended behavior over time.

Overall, the backtranslation approach shows promise, but additional research is needed to fully understand its limitations and explore other potential defense mechanisms against the evolving threat of jailbreaking attacks on LLMs.

Conclusion

This paper presents a novel defense against jailbreaking attacks on large language models (LLMs), using a technique called backtranslation to detect deviations from the model's intended behavior. By translating the LLM's output to another language and then back again, the researchers were able to reliably identify when the model was being misused to generate harmful or undesirable content.

While the backtranslation approach shows promise, there are still some limitations and areas for further research. Nonetheless, this work represents an important step forward in protecting these powerful AI systems from being exploited for malicious purposes, with significant implications for the responsible development and deployment of LLMs in the future.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Related Papers

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, Stjepan Picek

YC

0

Reddit

0

Large Language Models (LLMS) have increasingly become central to generating content with potential societal impacts. Notably, these models have demonstrated capabilities for generating content that could be deemed harmful. To mitigate these risks, researchers have adopted safety training techniques to align model outputs with societal values to curb the generation of malicious content. However, the phenomenon of jailbreaking, where carefully crafted prompts elicit harmful responses from models, persists as a significant challenge. This research conducts a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. We meticulously investigate nine attack techniques and seven defense techniques applied across three distinct language models: Vicuna, LLama, and GPT-3.5 Turbo. We aim to evaluate the effectiveness of these attack and defense techniques. Our findings reveal that existing white-box attacks underperform compared to universal techniques and that including special tokens in the input significantly affects the likelihood of successful attacks. This research highlights the need to concentrate on the security facets of LLMs. Additionally, we contribute to the field by releasing our datasets and testing framework, aiming to foster further research into LLM security. We believe these contributions will facilitate the exploration of security measures within this domain.

Read more

5/20/2024

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing

Wei Zhao, Zhe Li, Yige Li, Ye Zhang, Jun Sun

YC

0

Reddit

0

Large language models (LLMs) are increasingly being adopted in a wide range of real-world applications. Despite their impressive performance, recent studies have shown that LLMs are vulnerable to deliberately crafted adversarial prompts even when aligned via Reinforcement Learning from Human Feedback or supervised fine-tuning. While existing defense methods focus on either detecting harmful prompts or reducing the likelihood of harmful responses through various means, defending LLMs against jailbreak attacks based on the inner mechanisms of LLMs remains largely unexplored. In this work, we investigate how LLMs response to harmful prompts and propose a novel defense method termed textbf{L}ayer-specific textbf{Ed}iting (LED) to enhance the resilience of LLMs against jailbreak attacks. Through LED, we reveal that several critical textit{safety layers} exist among the early layers of LLMs. We then show that realigning these safety layers (and some selected additional layers) with the decoded safe response from selected target layers can significantly improve the alignment of LLMs against jailbreak attacks. Extensive experiments across various LLMs (e.g., Llama2, Mistral) show the effectiveness of LED, which effectively defends against jailbreak attacks while maintaining performance on benign prompts. Our code is available at url{https://github.com/ledllm/ledllm}.

Read more

5/29/2024

Subtoxic Questions: Dive Into Attitude Change of LLM's Response in Jailbreak Attempts

Subtoxic Questions: Dive Into Attitude Change of LLM's Response in Jailbreak Attempts

Tianyu Zhang, Zixuan Zhao, Jiaqi Huang, Jingyu Hua, Sheng Zhong

YC

0

Reddit

0

As Large Language Models (LLMs) of Prompt Jailbreaking are getting more and more attention, it is of great significance to raise a generalized research paradigm to evaluate attack strengths and a basic model to conduct subtler experiments. In this paper, we propose a novel approach by focusing on a set of target questions that are inherently more sensitive to jailbreak prompts, aiming to circumvent the limitations posed by enhanced LLM security. Through designing and analyzing these sensitive questions, this paper reveals a more effective method of identifying vulnerabilities in LLMs, thereby contributing to the advancement of LLM security. This research not only challenges existing jailbreaking methodologies but also fortifies LLMs against potential exploits.

Read more

4/15/2024

SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner

New!SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner

Xunguang Wang, Daoyuan Wu, Zhenlan Ji, Zongjie Li, Pingchuan Ma, Shuai Wang, Yingjiu Li, Yang Liu, Ning Liu, Juergen Rahmel

YC

0

Reddit

0

Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs) and has evolved into four major categories: optimization-based attacks such as Greedy Coordinate Gradient (GCG), jailbreak template-based attacks such as Do-Anything-Now, advanced indirect attacks like DrAttack, and multilingual jailbreaks. However, delivering a practical jailbreak defense is challenging because it needs to not only handle all the above jailbreak attacks but also incur negligible delay to user prompts, as well as be compatible with both open-source and closed-source LLMs. Inspired by how the traditional security concept of shadow stacks defends against memory overflow attacks, this paper introduces a generic LLM jailbreak defense framework called SelfDefend, which establishes a shadow LLM defense instance to concurrently protect the target LLM instance in the normal stack and collaborate with it for checkpoint-based access control. The effectiveness of SelfDefend builds upon our observation that existing LLMs (both target and defense LLMs) have the capability to identify harmful prompts or intentions in user queries, which we empirically validate using the commonly used GPT-3.5/4 models across all major jailbreak attacks. Our measurements show that SelfDefend enables GPT-3.5 to suppress the attack success rate (ASR) by 8.97-95.74% (average: 60%) and GPT-4 by even 36.36-100% (average: 83%), while incurring negligible effects on normal queries. To further improve the defense's robustness and minimize costs, we employ a data distillation approach to tune dedicated open-source defense models. These models outperform four SOTA defenses and match the performance of GPT-4-based SelfDefend, with significantly lower extra delays. We also empirically show that the tuned models are robust to targeted GCG and prompt injection attacks.

Read more

6/11/2024