Universal and Transferable Adversarial Attacks on Aligned Language Models

1 Introduction

1. How did earlier jailbreaks differ from the attack of Zou et al.?
   Earlier jailbreaks were hand-crafted with human ingenuity and brittle, and automatic adversarial prompt search had largely failed on aligned models. Zou et al. automatically optimize an adversarial suffix appended to the user’s request, leaving the request itself unchanged.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §1
2. Why had automatic adversarial attacks on language models been hard compared with images?
   Text inputs are discrete tokens, which limits the effective input dimensionality and makes the search combinatorial; gradient methods that work in continuous pixel space don’t apply directly.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §1
3. What three ingredients combine in the GCG attack?
   (1) Target an affirmative start of the response (“Sure, here is …”); (2) optimize the suffix with combined greedy and gradient-based discrete search; (3) optimize one suffix over many prompts and several models at once, making it universal and transferable.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §1
4. What does the success of GCG imply for the alignment agenda, according to Zou et al.?
   Like adversarial examples in vision, which remain unsolved because defenses are costly, hurt clean performance and cover narrow threat models, attacks on aligned models may persist: post hoc “repair” of models may not suffice.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §1