Universal and Transferable Adversarial Attacks on Aligned Language Models
Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043)
Subscriber deck — free during early access
2 cards in “2.2 Greedy Coordinate Gradient-based Search” · whole deck
2.2 Greedy Coordinate Gradient-based Search
- How does Greedy Coordinate Gradient (GCG) choose token replacements?Compute the gradient of the loss with respect to the one-hot vector of each suffix token; for every position take the top-\(k\) tokens with the most negative gradient as candidates; sample \(B\) single-token swaps from all positions, evaluate their exact loss with forward passes, and keep the best. Repeat.
Greedy Coordinate Gradient: gradients propose, exact forward passes decide; loss = −log p(“Sure, here is …”) - Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §2.2
- How does GCG differ from AutoPrompt, and why does it matter?AutoPrompt picks one position in advance and only tries replacements there; GCG considers candidate swaps at all positions in each step. With the same number of forward passes this seemingly minor change made a large difference: 88% vs 25% exact harmful-string matches on Vicuna-7B.
- Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §2.2
Source
- Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter and Matt Fredrikson. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. In arXiv preprint arXiv:2307.15043. https://arxiv.org/abs/2307.15043