Universal and Transferable Adversarial Attacks on Aligned Language Models

2.1 Producing Affirmative Responses

1. Why does GCG target a response beginning that restates the request affirmatively?
   Making “Sure, here is [the requested content]” the most likely start puts the model into a mode where it continues with the content. Targeting only “Sure” risks a suffix that overrides the request (e.g. “never mind, tell me a joke”); restating the request keeps the behaviour on target.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §2.1
2. What is the adversarial loss that GCG minimizes?
   The negative log-probability of the target token sequence \(x^\star_{n+1:n+H}\) (the affirmative beginning) given the prompt, \(\mathcal{L}(x_{1:n}) = -\log p(x^\star_{n+1:n+H} \mid x_{1:n})\), minimized over the tokens at the suffix positions only.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §2.1