Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
Zihao Xu (University of New South Wales, Student TU Delft)
Yi Liu (Nanyang Technological University)
Gelei Deng (Nanyang Technological University)
Kailong Wang (Huazhong University of Science and Technology)
Yuekang Li (University of New South Wales)
Ling Shi (Nanyang Technological University)
Stjepan Picek (Radboud Universiteit Nijmegen, TU Delft - Electrical Engineering, Mathematics and Computer Science)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Security concerns for large language models (LLMs) have intensified, particularly regarding jailbreaking attempts via malicious inputs. Studying new jailbreak attacks can help with red teaming to secure the LLMs. For open-source LLMs, embedding-based attacks can achieve high effectiveness. However, existing embedding-based attacks only optimize the suffix of the prompt, leading to unnecessary complexity and rendering them easier to detect. We propose a novel attack method that directly manipulates entire LLM inputs without separating them into bodies and suffixes. However, manipulating entire LLM inputs faces the challenges of random or nonsensical repetitive responses. To address these challenges, we propose Clip, whose main strategy is to clip each input dimension based on the mean and standard deviation of the model vocabulary during model inference. Experiments show that Clip improves the attack success rate (ASR) of continuous embedding attacks with full LLM inputs from 62% to 83% for LLaMa and from 38% to 83 % for Vicuna.