Universal and Transferable Adversarial Attacks on Aligned Language Models

3.2 Transfer attacks

1. How well did GCG suffixes transfer to black-box commercial models?
   Suffixes optimized only on open Vicuna and Guanaco models elicited harmful behaviors from GPT-3.5 (up to about 87%), GPT-4 (about 47–54%) and PaLM-2 (66%), but rarely Claude 2 (about 2%); transfer was best to GPT models, perhaps because Vicuna was trained on ChatGPT outputs.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §3.2
2. How could transfer attacks be strengthened?
   Concatenating several GCG suffixes raised GPT-3.5 success from 47% to 80% (though GPT-4 then often asked for clarification), and an ensemble trying several suffixes per request succeeded more often; stopping optimization earlier also reduced over-fitting to the source models.
   • Zou et al., 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (arXiv preprint arXiv:2307.15043), §3.2